|NO.Z.00027|——————————|^^ 部署 ^^|——|KuberNetes&二進位制部署.V05|5臺Server|---------------------------------------|證書生成|
阿新 • • 發佈：2022-03-29
[CloudNative：KuberNetes&二進位制部署.V05] [Applications.KuberNetes] [|DevOps|k8s|**5節點**|二進位制1.20|生成證書|] [|下載cfssl/cfssljson|etcd證書|kube-apiserver證書|kubernetes證書|kube-apiserver證書|] [kube-controller-manager證書|kube-scheduler證書|kubernetes-admin證書|serviceaccount.key.secret|]
一、生成證書：
### --- 生成證書：Master01下載生成證書工具（下載不成功可以去百度網盤）及建立資源目錄
~~~ etcd及kubernetes證書生成
~~~ 二進位制安裝最關鍵步驟，一步錯誤全盤皆輸，一定要注意每個步驟都要是正確的### --- 下載證書生成工具 [root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssl_linux-amd64" -O /usr/local/bin/cfssl [root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64" -O /usr/local/bin/cfssljson [root@k8s-master01 ~]# chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson
~~~ # etcd證書：所有安裝etcd節點建立etcd證書目錄
[root@k8s-master01 ~]# mkdir /etc/etcd/ssl -p
~~~ # 所有節點建立kubernetes相關目錄
[root@k8s-master01 ~]# mkdir -p /etc/kubernetes/pki二、生成etcd證書
### --- Master01節點生成etcd證書；生成證書的CSR檔案：證書籤名請求檔案，配置了一些域名、公司、單位 ~~~ # 生成etcd CA證書和CA證書的key [root@k8s-master01 ~]# cd /root/k8s-ha-install/pki [root@k8s-master01 pki]# cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca ~~~ 注：輸出結果： 2021/04/09 17:52:06 [INFO] generating a new CA key and certificate from CSR 2021/04/09 17:52:06 [INFO] generate received request 2021/04/09 17:52:06 [INFO] received CSR 2021/04/09 17:52:06 [INFO] generating key: rsa-2048 2021/04/09 17:52:06 [INFO] encoded CSR 2021/04/09 17:52:06 [INFO] signed certificate with serial number 423190820026756858541446334719884914519938174735
### --- 頒發etcd證書
~~~ # 通過生成的ca證書及key頒發證書
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/etcd/ssl/etcd-ca.pem \
-ca-key=/etc/etcd/ssl/etcd-ca-key.pem \
-config=ca-config.json \
-hostname=127.0.0.1,k8s-master01,k8s-master02,k8s-master03,192.168.1.11,192.168.1.12,192.168.1.13 \
-profile=kubernetes \
etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd
~~~ 輸出結果：
2021/04/09 17:52:57 [INFO] generate received request
2021/04/09 17:52:57 [INFO] received CSR
2021/04/09 17:52:57 [INFO] generating key: rsa-2048
2021/04/09 17:52:57 [INFO] encoded CSR
2021/04/09 17:52:57 [INFO] signed certificate with serial number 713011014384658330270180341022355700267979852195### --- 將etcd證書傳送到其它節點
~~~ # 定義變數
[root@k8s-master01 pki]# MasterNodes='k8s-master02 k8s-master03'
[root@k8s-master01 pki]# WorkNodes='k8s-node01 k8s-node02'~~~ # 傳送證書到其它節點
[root@k8s-master01 pki]# for NODE in $MasterNodes; do
ssh $NODE "mkdir -p /etc/etcd/ssl"
for FILE in etcd-ca-key.pem etcd-ca.pem etcd-key.pem etcd.pem; do
scp /etc/etcd/ssl/${FILE} $NODE:/etc/etcd/ssl/${FILE}
done
done
~~~ 注：輸出結果：
etcd-ca-key.pem 100% 1675 925.2KB/s 00:00
etcd-ca.pem 100% 1367 904.5KB/s 00:00
etcd-key.pem 100% 1679 715.8KB/s 00:00
etcd.pem 100% 1509 526.8KB/s 00:00
etcd-ca-key.pem 100% 1675 994.7KB/s 00:00
etcd-ca.pem 100% 1367 507.2KB/s 00:00
etcd-key.pem 100% 1679 682.7KB/s 00:00
etcd.pem 三、生成證書：k8s元件證書-kube-apiserver證書
### --- Master01生成kubernetes證書
~~~ # 生成kubernetes ca證書和ca證書的key
[root@k8s-master01 pki]# cd /root/k8s-ha-install/pki
[root@k8s-master01 pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare /etc/kubernetes/pki/ca
~~~ 注：輸出結果：
2021/04/09 17:55:35 [INFO] generating a new CA key and certificate from CSR
2021/04/09 17:55:35 [INFO] generate received request
2021/04/09 17:55:35 [INFO] received CSR
2021/04/09 17:55:35 [INFO] generating key: rsa-2048
2021/04/09 17:55:35 [INFO] encoded CSR
2021/04/09 17:55:35 [INFO] signed certificate with serial number 312724731765196138565235611759823205222208149928### --- 為kubernetes頒發證書
~~~ # 頒發證書
~~~ 10.96.0.是k8s service的網段，如果說需要更改k8s service網段，那就需要更改10.96.0.1，
~~~ 如果不是高可用叢集，192.168.1.11為Master01的IP
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=ca-config.json -hostname=10.96.0.1,192.168.1.20,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,192.168.1.11,192.168.1.12,192.168.1.13 -profile=kubernetes apiserver-csr.json | cfssljson -bare /etc/kubernetes/pki/apiserver
~~~ 注：輸出結果：
2021/04/09 17:56:51 [INFO] generate received request
2021/04/09 17:56:51 [INFO] received CSR
2021/04/09 17:56:51 [INFO] generating key: rsa-2048
2021/04/09 17:56:51 [INFO] encoded CSR
2021/04/09 17:56:51 [INFO] signed certificate with serial number 210976959466905225261553741556880293283669782216四、生成apiserver的聚合證書
### --- 生成apiserver的聚合證書。生成apiserver的ca證書和ca證書的key
~~~ # 生成apiserver的聚合證書。Requestheader-client-xxx requestheader-allowwd-xxx:aggerator
[root@k8s-master01 pki]# cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-ca
~~~ 注：輸出結果：
2021/04/09 17:57:34 [INFO] generating a new CA key and certificate from CSR
2021/04/09 17:57:34 [INFO] generate received request
2021/04/09 17:57:34 [INFO] received CSR
2021/04/09 17:57:34 [INFO] generating key: rsa-2048
2021/04/09 17:57:34 [INFO] encoded CSR
2021/04/09 17:57:34 [INFO] signed certificate with serial number 229937635932613642720561308611651745796925115826### --- 為apiserver頒發證書
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/front-proxy-ca.pem -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem -config=ca-config.json -profile=kubernetes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
~~~ 注：輸出結果：
2021/04/09 17:58:01 [INFO] generate received request
2021/04/09 17:58:01 [INFO] received CSR
2021/04/09 17:58:01 [INFO] generating key: rsa-2048
2021/04/09 17:58:01 [INFO] encoded CSR
2021/04/09 17:58:01 [INFO] signed certificate with serial number 341267100255324312793031252506134637751260697587
2021/04/09 17:58:01 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").五、生成證書ControllerManager證書
### --- 為controller-manager頒發證書
~~~ # 生成ControllerManager證書
~~~ 注意，如果不是高可用叢集，192.168.0.236:8443改為master01的地址，8443改為apiserver的埠，預設是6443
~~~ # set-cluster：設定一個叢集項
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
manager-csr.json | cfssljson -bare /etc/kubernetes/pki/controller-manager
~~~ 注：輸出結果：
2021/04/09 17:59:19 [INFO] generate received request
2021/04/09 17:59:19 [INFO] received CSR
2021/04/09 17:59:19 [INFO] generating key: rsa-2048
2021/04/09 17:59:19 [INFO] encoded CSR
2021/04/09 17:59:19 [INFO] signed certificate with serial number 57476090859638169695558073350960312663800570839
2021/04/09 17:59:19 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 設定set-cluster叢集項
~~~ # 設定叢集項
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.0.236:8443 \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
Cluster "kubernetes" set.### --- 設定環境項上下文
~~~ # 設定一個環境項，一個上下文
[root@k8s-master01 pki]# kubectl config set-context system:kube-controller-manager@kubernetes \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
Context "system:kube-controller-manager@kubernetes" created.### --- 設定使用者項
~~~ # set-credentials 設定一個使用者項
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-controller-manager \
--client-certificate=/etc/kubernetes/pki/controller-manager.pem \
--client-key=/etc/kubernetes/pki/controller-manager-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
User "system:kube-controller-manager" set.### --- 使用某個環境當做預設環境
[root@k8s-master01 pki]# kubectl config use-context system:kube-controller-manager@kubernetes \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
Switched to context "system:kube-controller-manager@kubernetes".六、生成kube-scheduler證書
### --- 為kube-scheduler頒發證書
~~~ # 為kube-scheduler頒發證書
~~~ 注意，如果不是高可用叢集，192.168.0.236:8443改為master01的地址，8443改為apiserver的埠，預設是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
scheduler-csr.json | cfssljson -bare /etc/kubernetes/pki/scheduler
~~~ 注：輸出結果：
2021/04/09 18:05:35 [INFO] generate received request
2021/04/09 18:05:35 [INFO] received CSR
2021/04/09 18:05:35 [INFO] generating key: rsa-2048
2021/04/09 18:05:36 [INFO] encoded CSR
2021/04/09 18:05:36 [INFO] signed certificate with serial number 581707959605151325249703913778192497550173882170
2021/04/09 18:05:36 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 設定叢集項
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.0.236:8443 \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
Cluster "kubernetes" set.### --- 設定環境項上下文
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-scheduler \
--client-certificate=/etc/kubernetes/pki/scheduler.pem \
--client-key=/etc/kubernetes/pki/scheduler-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
User "system:kube-scheduler" set.### --- 設定使用者項
[root@k8s-master01 pki]# kubectl config set-context system:kube-scheduler@kubernetes \
--cluster=kubernetes \
--user=system:kube-scheduler \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
Context "system:kube-scheduler@kubernetes" created.### --- 使用某個環境當做預設環境
[root@k8s-master01 pki]# kubectl config use-context system:kube-scheduler@kubernetes \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
Switched to context "system:kube-scheduler@kubernetes".七、生成kubernetes-admin使用者證書
### --- 為kubernetes-admin使用者頒發證書
~~~ 注意，如果不是高可用叢集，192.168.0.236:8443改為master01的地址，8443改為apiserver的埠，預設是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
admin-csr.json | cfssljson -bare /etc/kubernetes/pki/admin
~~~ 注：輸出結果：
2021/04/09 18:10:11 [INFO] generate received request
2021/04/09 18:10:11 [INFO] received CSR
2021/04/09 18:10:11 [INFO] generating key: rsa-2048
2021/04/09 18:10:12 [INFO] encoded CSR
2021/04/09 18:10:12 [INFO] signed certificate with serial number 604960830409772440587252668095907626459060809354
2021/04/09 18:10:12 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").### --- 設定叢集項
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.20:8443 --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
Cluster "kubernetes" set.### --- 設定環境項上下文
[root@k8s-master01 pki]# kubectl config set-credentials kubernetes-admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
User "kubernetes-admin" set.### --- 設定使用者項
[root@k8s-master01 pki]# kubectl config set-context kubernetes-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
Context "kubernetes-admin@kubernetes" created.### --- 使用某個環境當做預設環境
[root@k8s-master01 pki]# kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
Switched to context "kubernetes-admin@kubernetes".### --- 建立ServiceAccount key
[root@k8s-master01 pki]# openssl genrsa -out /etc/kubernetes/pki/sa.key 2048
~~~ 注：輸出結果：
Generating RSA private key, 2048 bit long modulus
....................+++
.......................................+++
e is 65537 (0x10001)### --- 生成serviceAccount key
[root@k8s-master01 pki]# openssl rsa -in /etc/kubernetes/pki/sa.key -pubout -out /etc/kubernetes/pki/sa.pub
writing RSA key九、將證書傳送至其它節點
### --- 將證書傳送至其它節點
[root@k8s-master01 pki]# for NODE in k8s-master02 k8s-master03; do
for FILE in $(ls /etc/kubernetes/pki | grep -v etcd); do
scp /etc/kubernetes/pki/${FILE} $NODE:/etc/kubernetes/pki/${FILE};
done;
for FILE in admin.kubeconfig controller-manager.kubeconfig scheduler.kubeconfig; do
scp /etc/kubernetes/${FILE} $NODE:/etc/kubernetes/${FILE};
done;
done
~~~ 輸出結果：
admin.csr 100% 1025 20.8KB/s 00:00
admin-key.pem 100% 1679 643.1KB/s 00:00
admin.pem 100% 1444 408.3KB/s 00:00
apiserver.csr 100% 1029 462.9KB/s 00:00
apiserver-key.pem 100% 1679 611.6KB/s 00:00
apiserver.pem 100% 1692 62.5KB/s 00:00
ca.csr 100% 1025 274.1KB/s 00:00
ca-key.pem 100% 1679 629.6KB/s 00:00
ca.pem 100% 1411 577.3KB/s 00:00
controller-manager.csr 100% 1082 368.9KB/s 00:00
controller-manager-key.pem 100% 1679 537.2KB/s 00:00
controller-manager.pem 100% 1501 477.9KB/s 00:00
front-proxy-ca.csr 100% 891 400.5KB/s 00:00
front-proxy-ca-key.pem 100% 1675 473.6KB/s 00:00
front-proxy-ca.pem 100% 1143 296.4KB/s 00:00
front-proxy-client.csr 100% 903 233.5KB/s 00:00
front-proxy-client-key.pem 100% 1675 35.1KB/s 00:00
front-proxy-client.pem 100% 1188 59.8KB/s 00:00
sa.key 100% 1675 764.3KB/s 00:00
sa.pub 100% 451 226.2KB/s 00:00
scheduler.csr 100% 1058 370.8KB/s 00:00
scheduler-key.pem 100% 1675 799.9KB/s 00:00
scheduler.pem 100% 1476 727.9KB/s 00:00
admin.kubeconfig 100% 6452 306.1KB/s 00:00
controller-manager.kubeconfig 100% 6584 1.9MB/s 00:00
scheduler.kubeconfig ### --- 檢視生成的證書及證書數量
~~~ # 檢視生成的所有證書
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/
admin.csr apiserver.csr ca.csr controller-manager.csr front-proxy-ca.csr front-proxy-client.csr sa.key scheduler-key.pem
admin-key.pem apiserver-key.pem ca-key.pem controller-manager-key.pem front-proxy-ca-key.pem front-proxy-client-key.pem sa.pub scheduler.pem
admin.pem apiserver.pem ca.pem controller-manager.pem front-proxy-ca.pem front-proxy-client.pem scheduler.csr~~~ # 檢視生成證書數量
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/ |wc -l
23===============================END===============================
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart ——W.S.Landor
來自為知筆記(Wiz)