|NO.Z.00052|——————————|^^ 部署 ^^|——|KuberNetes&二進位制部署.V05|3臺Server|---------------------------------------|證書生成|
阿新 • • 發佈：2022-03-29
[CloudNative：KuberNetes&二進位制部署.V05] [Applications.KuberNetes] [|DevOps|k8s|**3節點**|二進位制1.20|生成證書|] [|下載cfssl/cfssljson|etcd證書|kube-apiserver證書|kubernetes證書|kube-apiserver證書|] [kube-controller-manager證書|kube-scheduler證書|kubernetes-admin證書|serviceaccount.key.secret|]
一、生成證書：
### --- Master01下載生成證書工具（下載不成功可以去百度網盤）及建立資源目錄
~~~ etcd及kubernetes證書生成
~~~ 二進位制安裝最關鍵步驟，一步錯誤全盤皆輸，一定要注意每個步驟都要是正確的### ---下載證書生成工具 [root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssl_linux-amd64" -O /usr/local/bin/cfssl [root@k8s-master01 ~]# wget "https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64" -O /usr/local/bin/cfssljson [root@k8s-master01 ~]# chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson
~~~ # etcd證書：所有安裝etcd節點建立etcd證書目錄
[root@k8s-master01 ~]# mkdir /etc/etcd/ssl -p
~~~ # 所有節點建立kubernetes相關目錄
[root@k8s-master01 ~]# mkdir -p /etc/kubernetes/pki二、生成etcd證書
### --- Master01節點生成etcd證書；生成證書的CSR檔案：證書籤名請求檔案，配置了一些域名、公司、單位 ~~~ # 生成etcd CA證書和CA證書的key [root@k8s-master01 ~]# cd /root/k8s-ha-install/pki [root@k8s-master01 pki]# cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /etc/etcd/ssl/etcd-ca ~~~ 注：輸出結果： 2021/05/12 19:15:31 [INFO] generating a new CA key and certificate from CSR 2021/05/12 19:15:31 [INFO] generate received request 2021/05/12 19:15:31 [INFO] received CSR 2021/05/12 19:15:31 [INFO] generating key: rsa-2048 2021/05/12 19:15:32 [INFO] encoded CSR 2021/05/12 19:15:32 [INFO] signed certificate with serial number 417879652597954519889260948756440442182907581235
### --- 頒發etcd證書
~~~ # 通過生成的ca證書及key頒發證書
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/etcd/ssl/etcd-ca.pem \
-ca-key=/etc/etcd/ssl/etcd-ca-key.pem \
-config=ca-config.json \
-hostname=127.0.0.1,k8s-master01,k8s-node01,k8s-node02,192.168.1.11,192.168.1.14,192.168.1.15 \
-profile=kubernetes \
etcd-csr.json | cfssljson -bare /etc/etcd/ssl/etcd
~~~ 注：輸出結果：
2021/05/12 19:20:27 [INFO] generate received request
2021/05/12 19:20:27 [INFO] received CSR
2021/05/12 19:20:27 [INFO] generating key: rsa-2048
2021/05/12 19:20:27 [INFO] encoded CSR
2021/05/12 19:20:27 [INFO] signed certificate with serial number 452010686264797775985430527541917102604725591793### --- 將etcd證書傳送到其它節點
~~~ # 定義變數
[root@k8s-master01 pki]# MasterNodes='k8s-node01 k8s-node02'
[root@k8s-master01 pki]# WorkNodes='k8s-node01 k8s-node02'~~~ # 傳送證書到其它節點
[root@k8s-master01 pki]# for NODE in $MasterNodes; do
ssh $NODE "mkdir -p /etc/etcd/ssl"
for FILE in etcd-ca-key.pem etcd-ca.pem etcd-key.pem etcd.pem; do
scp /etc/etcd/ssl/${FILE} $NODE:/etc/etcd/ssl/${FILE}
done
done
~~~ 注：輸出結果：
etcd-ca-key.pem 100% 1679 877.4KB/s 00:00
etcd-ca.pem 100% 1367 648.2KB/s 00:00
etcd-key.pem 100% 1675 634.7KB/s 00:00
etcd.pem 100% 1501 350.7KB/s 00:00
etcd-ca-key.pem 100% 1679 468.6KB/s 00:00
etcd-ca.pem 100% 1367 387.5KB/s 00:00
etcd-key.pem 100% 1675 404.1KB/s 00:00
etcd.pem
三、生成證書：k8s元件證書-kube-apiserver證書
### --- Master01生成kubernetes證書
~~~ # 生成kubernetes ca證書和ca證書的key
[root@k8s-master01 pki]# cd /root/k8s-ha-install/pki
[root@k8s-master01 pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare /etc/kubernetes/pki/ca
~~~ 注：輸出結果：
2021/05/12 19:24:28 [INFO] generating a new CA key and certificate from CSR
2021/05/12 19:24:28 [INFO] generate received request
2021/05/12 19:24:28 [INFO] received CSR
2021/05/12 19:24:28 [INFO] generating key: rsa-2048
2021/05/12 19:24:28 [INFO] encoded CSR
2021/05/12 19:24:28 [INFO] signed certificate with serial number 447109712814672408133045353535582932352630134506
### --- 為kubernetes頒發證書
~~~ # 頒發證書
~~~ 10.96.0.是k8s service的網段，如果說需要更改k8s service網段，那就需要更改10.96.0.1，
~~~ 如果不是高可用叢集，192.168.1.11為Master01的IP
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=ca-config.json -hostname=10.96.0.1,192.168.1.11,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,192.168.1.11,192.168.1.14,192.168.1.15 -profile=kubernetes apiserver-csr.json | cfssljson -bare /etc/kubernetes/pki/apiserver
~~~ 注：輸出結果：
2021/05/12 19:27:37 [INFO] generate received request
2021/05/12 19:27:37 [INFO] received CSR
2021/05/12 19:27:37 [INFO] generating key: rsa-2048
2021/05/12 19:27:37 [INFO] encoded CSR
2021/05/12 19:27:37 [INFO] signed certificate with serial number 84788908146128667480104666726419859131741151671四、生成apiserver的聚合證書
### --- 生成apiserver的聚合證書。生成apiserver的ca證書和ca證書的key
~~~ # 生成apiserver的聚合證書。Requestheader-client-xxx requestheader-allowwd-xxx:aggerator
[root@k8s-master01 pki]# cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-ca
~~~ 注：輸出結果：
netes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
2021/05/12 19:28:52 [INFO] generating a new CA key and certificate from CSR
2021/05/12 19:28:52 [INFO] generate received request
2021/05/12 19:28:52 [INFO] received CSR
2021/05/12 19:28:52 [INFO] generating key: rsa-2048
2021/05/12 19:28:52 [INFO] encoded CSR
2021/05/12 19:28:52 [INFO] signed certificate with serial number 516217615073867303946934109541825127048193138588
### --- 為apiserver頒發證書
[root@k8s-master01 pki]# cfssl gencert -ca=/etc/kubernetes/pki/front-proxy-ca.pem -ca-key=/etc/kubernetes/pki/front-proxy-ca-key.pem -config=ca-config.json -profile=kubernetes front-proxy-client-csr.json | cfssljson -bare /etc/kubernetes/pki/front-proxy-client
2021/05/12 19:28:52 [INFO] generate received request
2021/05/12 19:28:52 [INFO] received CSR
2021/05/12 19:28:52 [INFO] generating key: rsa-2048
2021/05/12 19:28:53 [INFO] encoded CSR
2021/05/12 19:28:53 [INFO] signed certificate with serial number 696831972351009499497638617028756244195237115056
2021/05/12 19:28:53 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
~~~ 注：輸出結果：
五、生成證書ControllerManager證書
### --- 為controller-manager頒發證書
~~~ # 生成ControllerManager證書
~~~ 注意：如果不是高可用叢集，192.168.1.11:8443改為master01的地址，8443改為apiserver的埠，預設是6443
~~~ #set-cluster：設定一個叢集項
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
manager-csr.json | cfssljson -bare /etc/kubernetes/pki/controller-manager
~~~ 注：輸出結果：
2021/05/12 19:31:16 [INFO] generate received request
2021/05/12 19:31:16 [INFO] received CSR
2021/05/12 19:31:16 [INFO] generating key: rsa-2048
2021/05/12 19:31:16 [INFO] encoded CSR
2021/05/12 19:31:16 [INFO] signed certificate with serial number 384930359684285771093349021380105968424292629747
2021/05/12 19:31:16 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
### --- 設定set-cluster叢集項
~~~ # 設定叢集項
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.1.11:6443 \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
Cluster "kubernetes" set.
### --- 設定環境項上下文
~~~ # 設定一個環境項，一個上下文
[root@k8s-master01 pki]# kubectl config set-context system:kube-controller-manager@kubernetes \
--cluster=kubernetes \
--user=system:kube-controller-manager \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
Context "system:kube-controller-manager@kubernetes" created.
### --- 設定使用者項
~~~ # set-credentials 設定一個使用者項
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-controller-manager \
--client-certificate=/etc/kubernetes/pki/controller-manager.pem \
--client-key=/etc/kubernetes/pki/controller-manager-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
User "system:kube-controller-manager" set.
### --- 使用某個環境當做預設環境
~~~ # 使用某個環境當做預設環境
[root@k8s-master01 pki]# kubectl config use-context system:kube-controller-manager@kubernetes \
--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig
~~~ 注：輸出結果：
Switched to context "system:kube-controller-manager@kubernetes".
六、生成kube-scheduler證書
### --- 為kube-scheduler頒發證書
~~~ # 注意，如果不是高可用叢集，192.168.1.11:8443改為master01的地址，8443改為apiserver的埠，預設是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
scheduler-csr.json | cfssljson -bare /etc/kubernetes/pki/scheduler
~~~ 注：輸出結果：
2021/05/12 19:36:56 [INFO] generate received request
2021/05/12 19:36:56 [INFO] received CSR
2021/05/12 19:36:56 [INFO] generating key: rsa-2048
2021/05/12 19:36:56 [INFO] encoded CSR
2021/05/12 19:36:56 [INFO] signed certificate with serial number 331467505877315472816673290342178535942330545986
2021/05/12 19:36:56 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
### --- 設定叢集項
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=https://192.168.1.11:6443 \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
Cluster "kubernetes" set.
### --- 設定環境項上下文
[root@k8s-master01 pki]# kubectl config set-credentials system:kube-scheduler \
--client-certificate=/etc/kubernetes/pki/scheduler.pem \
--client-key=/etc/kubernetes/pki/scheduler-key.pem \
--embed-certs=true \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
User "system:kube-scheduler" set.
### --- 設定使用者項
[root@k8s-master01 pki]# kubectl config set-context system:kube-scheduler@kubernetes \
--cluster=kubernetes \
--user=system:kube-scheduler \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
Context "system:kube-scheduler@kubernetes" created.
### --- 使用某個環境當做預設環境
[root@k8s-master01 pki]# kubectl config use-context system:kube-scheduler@kubernetes \
--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
~~~ 注：輸出結果：
Switched to context "system:kube-scheduler@kubernetes".
七、生成kubernetes-admin使用者證書
### --- 為kubernetes-admin使用者頒發證書
~~~ # 注意，如果不是高可用叢集，192.168.1.11:8443改為master01的地址，8443改為apiserver的埠，預設是6443
[root@k8s-master01 pki]# cfssl gencert \
-ca=/etc/kubernetes/pki/ca.pem \
-ca-key=/etc/kubernetes/pki/ca-key.pem \
-config=ca-config.json \
-profile=kubernetes \
admin-csr.json | cfssljson -bare /etc/kubernetes/pki/admin
~~~ 注：輸出結果：
2021/05/12 19:39:12 [INFO] generate received request
2021/05/12 19:39:12 [INFO] received CSR
2021/05/12 19:39:12 [INFO] generating key: rsa-2048
2021/05/12 19:39:12 [INFO] encoded CSR
2021/05/12 19:39:12 [INFO] signed certificate with serial number 267138241499797848649434576652196091163365718803
2021/05/12 19:39:12 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
### --- 設定叢集項
[root@k8s-master01 pki]# kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.11:6443 --kubeconfig=/etc/kubernetes/admin.kubeconfig
--client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=/etc/kubernetes/admin.kubeconfig
kubectl config set-context kubernetes-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig
kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
Cluster "kubernetes" set.
### --- 設定環境項上下文
[root@k8s-master01 pki]# kubectl config set-credentials kubernetes-admin --client-certificate=/etc/kubernetes/pki/admin.pem --client-key=/etc/kubernetes/pki/admin-key.pem --embed-certs=true --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
User "kubernetes-admin" set.
### --- 設定使用者項
[root@k8s-master01 pki]# kubectl config set-context kubernetes-admin@kubernetes --cluster=kubernetes --user=kubernetes-admin --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
Context "kubernetes-admin@kubernetes" created.
### --- 使用某個環境當做預設環境
[root@k8s-master01 pki]# kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=/etc/kubernetes/admin.kubeconfig
~~~ 注：輸出結果：
Switched to context "kubernetes-admin@kubernetes".八、建立ServiceAccount key——>secret
### --- 建立ServiceAccount key
[root@k8s-master01 pki]# openssl genrsa -out /etc/kubernetes/pki/sa.key 2048
~~~ 輸出結果：
Generating RSA private key, 2048 bit long modulus
.......................................+++
..........+++
e is 65537 (0x10001)
### --- 生成serviceAccount key
[root@k8s-master01 pki]# openssl rsa -in /etc/kubernetes/pki/sa.key -pubout -out /etc/kubernetes/pki/sa.pub
writing RSA key
九、將證書傳送至其它節點
### --- 將證書傳送至其它節點
[root@k8s-master01 pki]# for NODE in k8s-node01 k8s-node02; do
for FILE in $(ls /etc/kubernetes/pki | grep -v etcd); do
scp /etc/kubernetes/pki/${FILE} $NODE:/etc/kubernetes/pki/${FILE};
done;
for FILE in admin.kubeconfig controller-manager.kubeconfig scheduler.kubeconfig; do
scp /etc/kubernetes/${FILE} $NODE:/etc/kubernetes/${FILE};
done;
done
~~~ 輸出結果：
admin.csr 100% 1025 357.0KB/s 00:00
admin-key.pem 100% 1679 734.2KB/s 00:00
admin.pem 100% 1444 552.1KB/s 00:00
apiserver.csr 100% 1029 246.7KB/s 00:00
apiserver-key.pem 100% 1675 583.3KB/s 00:00
apiserver.pem 100% 1692 631.7KB/s 00:00
ca.csr 100% 1025 248.2KB/s 00:00
ca-key.pem 100% 1675 593.3KB/s 00:00
ca.pem 100% 1411 707.0KB/s 00:00
controller-manager.csr 100% 1082 292.6KB/s 00:00
controller-manager-key.pem 100% 1675 316.4KB/s 00:00
controller-manager.pem 100% 1501 540.6KB/s 00:00
front-proxy-ca.csr 100% 891 453.1KB/s 00:00
front-proxy-ca-key.pem 100% 1679 64.9KB/s 00:00
front-proxy-ca.pem 100% 1143 575.4KB/s 00:00
front-proxy-client.csr 100% 903 248.0KB/s 00:00
front-proxy-client-key.pem 100% 1679 408.5KB/s 00:00
front-proxy-client.pem 100% 1188 296.9KB/s 00:00
sa.key 100% 1679 387.5KB/s 00:00
sa.pub 100% 451 156.3KB/s 00:00
scheduler.csr 100% 1058 465.2KB/s 00:00
scheduler-key.pem 100% 1679 238.3KB/s 00:00
scheduler.pem 100% 1476 372.8KB/s 00:00
admin.kubeconfig 100% 6452 143.4KB/s 00:00
controller-manager.kubeconfig 100% 6580 1.5MB/s 00:00
scheduler.kubeconfig 100% 6512 1.5MB/s 00:00
admin.csr 100% 1025 116.5KB/s 00:00
admin-key.pem 100% 1679 78.4KB/s 00:00
admin.pem 100% 1444 365.7KB/s 00:00
apiserver.csr 100% 1029 274.5KB/s 00:00
apiserver-key.pem 100% 1675 196.8KB/s 00:00
apiserver.pem 100% 1692 338.6KB/s 00:00
ca.csr 100% 1025 115.5KB/s 00:00
ca-key.pem 100% 1675 393.6KB/s 00:00
ca.pem 100% 1411 143.1KB/s 00:00
controller-manager.csr 100% 1082 139.4KB/s 00:00
controller-manager-key.pem 100% 1675 157.9KB/s 00:00
controller-manager.pem 100% 1501 277.9KB/s 00:00
front-proxy-ca.csr 100% 891 201.4KB/s 00:00
front-proxy-ca-key.pem 100% 1679 214.3KB/s 00:00
front-proxy-ca.pem 100% 1143 167.7KB/s 00:00
front-proxy-client.csr 100% 903 169.0KB/s 00:00
front-proxy-client-key.pem 100% 1679 393.0KB/s 00:00
front-proxy-client.pem 100% 1188 235.9KB/s 00:00
sa.key 100% 1679 94.5KB/s 00:00
sa.pub 100% 451 55.0KB/s 00:00
scheduler.csr 100% 1058 371.9KB/s 00:00
scheduler-key.pem 100% 1679 325.1KB/s 00:00
scheduler.pem 100% 1476 201.7KB/s 00:00
admin.kubeconfig 100% 6452 1.1MB/s 00:00
controller-manager.kubeconfig 100% 6580 687.2KB/s 00:00
scheduler.kubeconfig
### --- 檢視生成的證書及證書數量
~~~ # 檢視生成的所有證書
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/
admin.csr apiserver.csr ca.csr controller-manager.csr front-proxy-ca.csr front-proxy-client.csr sa.key scheduler-key.pem
admin-key.pem apiserver-key.pem ca-key.pem controller-manager-key.pem front-proxy-ca-key.pem front-proxy-client-key.pem sa.pub scheduler.pem
admin.pem apiserver.pem ca.pem controller-manager.pem front-proxy-ca.pem front-proxy-client.pem scheduler.csr
~~~ # 檢視生成證書數量
[root@k8s-master01 pki]# ls /etc/kubernetes/pki/ |wc -l
23Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart ——W.S.Landor
來自為知筆記(Wiz)