第9章：Bootstrap Token方式增加Node
阿新 • • 發佈：2020-08-23
Bootstrap Token方式增加Node
參考資料：
https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens
https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping
二進位制搭建 K8s 詳細步驟：
https://mp.weixin.qq.com/s/VYtyTU9_Dw9M5oHtvRfseA
Ansible自動化部署K8s叢集：https://github.com/lizhenliang/ansible-install-k8s/
角色 IP
Master 192.168.31.61
Node1 192.168.31.62
Node2 192.168.31.63
Node3 192.168.31.64
1、準備新節點環境 準備新節點環境
提前安裝好 Docker。
scp /tmp/k8s/docker/* [email protected]:/usr/bin/ scp /usr/lib/systemd/system/docker.service [email protected]:/usr/lib/systemd/system/
scp -r /etc/docker/daemon.json [email protected]:/etc/docker/
拷貝已部署好的 Node 相關檔案到新節點 Node3：
#複製 kubernetes 檔案 scp -r /opt/kubernetes/ root@192.168.31.73:/opt/ scp -r /usr/lib/systemd/system/{kubelet,kube-proxy}.service root@192.168.31.73:/usr/lib/systemd/system/ #複製 cni 網路 scp -r /opt/cni/ root@192.168.31.73:/opt
刪除 kubelet 證書和 kubeconfig 檔案：
#刪除複製過來的舊的 kubelet 證書，後面新加入節點，生成新的證書 [root@k8s-node2 ~]# cd /opt/kubernetes/ssl/ [root@k8s-node2 ssl]# ls ca.pem kubelet-client-current.pem kubelet.key kube-proxy.pem kubelet-client-2020-08-23-01-07-26.pem kubelet.crt kube-proxy-key.pem [root@k8s-node2 ssl]# rm -f kubelet* [root@k8s-node2 ssl]# ls ca.pem kube-proxy-key.pem kube-proxy.pem #刪除複製過來舊的bootstrap.kubeconfig 檔案，kubelet.kubeconfig （這個簽發成功，後面會自動生成） [root@k8s-node2 ssl]# cd ../cfg/ [root@k8s-node2 cfg]# ls bootstrap.kubeconfig kubelet-config.yml kube-proxy.conf kube-proxy.kubeconfig kubelet.conf kubelet.kubeconfig kube-proxy-config.yml [root@k8s-node2 cfg]# rm -f kubelet.kubeconfig bootstrap.kubeconfig
注：這幾個檔案是證書申請審批後自動生成的，每個 Node 不同，必須刪除重新生成。
修改主機名
對新加入的node 進行改名 node2 和新節點名字一致 [root@k8s-node2 cfg]# vim kubelet.conf [root@k8s-node2 cfg]# vim kube-proxy-config.yml
2 、確認 啟用bootstrap-token
預設已經啟用。
# cat /opt/kubernetes/cfg/kube-apiserver.conf … --enable-bootstrap-token-auth=true …
3、使用 Secret儲存 Bootstrap Token
注：expiration 為 token 過期時間，當前時間向後推幾天隨意
[root@k8s-master1 chp9]# cat token-secret.yaml apiVersion: v1 kind: Secret metadata: # Name MUST be of form "bootstrap-token-<token id>" name: bootstrap-token-07401b namespace: kube-system # Type MUST be 'bootstrap.kubernetes.io/token' type: bootstrap.kubernetes.io/token stringData: # Human readable description. Optional. description: "The default bootstrap token generated by 'kubeadm init'." # Token ID and secret. Required. token-id: 07401b token-secret: f395accd246ae52d # Expiration. Optional. expiration: 2020-10-10T03:22:11Z # Allowed usages. usage-bootstrap-authentication: "true" usage-bootstrap-signing: "true" # Extra groups to authenticate the token as. Must start with "system:bootstrappers:" auth-extra-groups: system:bootstrappers:worker,system:bootstrappers:ingress
[root@k8s-master1 c9]# kubectl apply -f secret-token.yml secret/bootstrap-token-07401b created [root@k8s-master1 chp9]# kubectl get secrets -n kube-system NAME TYPE DATA AGE bootstrap-token-07401b bootstrap.kubernetes.io/token 7 17s
4、建立RBAC角色繫結，允許 kubelet tls bootstrap建立CSR請求
[root@k8s-master1 chp9]# cat rbac.yml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: create-csrs-for-bootstrapping subjects: - kind: Group name: system:bootstrappers apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: system:node-bootstrapper apiGroup: rbac.authorization.k8s.io [root@k8s-master1 chp9]# kubectl apply -f bootstrap.yml clusterrolebinding.rbac.authorization.k8s.io/create-csrs-for-bootstrapping created
5、 kubelet配置 bootstrap kubeconfig檔案
在Node3上操作
apiVersion: v1 kind: Config clusters: - cluster: certificate-authority: /opt/kubernetes/ssl/ca.pem server: https://192.168.31.71:6443 name: bootstrap contexts: - context: cluster: bootstrap user: kubelet-bootstrap name: bootstrap current-context: bootstrap preferences: {} users: - name: kubelet-bootstrap user: token: 07401b.f395accd246ae52d
配置檔案指定 kubeconfig 檔案，預設已經配置：
[root@k8s-node2 ssl]# cat /opt/kubernetes/cfg/kubelet.conf KUBELET_OPTS="--logtostderr=false \ --v=4 \ --log-dir=/opt/kubernetes/logs \ --hostname-override=k8s-node2 \ --network-plugin=cni \ --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \ --bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \ --config=/opt/kubernetes/cfg/kubelet-config.yml \ --cert-dir=/opt/kubernetes/ssl \ --pod-infra-container-image=lizhenliang/pause-amd64:3.0"
啟動並設定開機啟動 ：
systemctl daemon-reload
systemctl enable kubelet.service
systemctl start kubelet
6、在 Master節點頒發證書
[root@k8s-master1 c9]# kubectl get csr NAME AGE SIGNERNAME REQUESTOR CONDITION node-csr-Ur8gGqJexjA3yGk5k1wWC8y6Q076x4IKHfQjTkx8k3g 34m kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Approved,Issued node-csr-YVpk8Sax7vSJ_R-J_MAQDOY6jbzWtR4q9xzVXKxPCMM 28s kubernetes.io/kube-apiserver-client-kubelet system:bootstrap:07401b Pending node-csr-ps3O0TCveqWMEdANu0Psg1qq-WyFR0pWuaFCKAPupXM 34m kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Approved,Issued node-csr-ycjy-hf7hO1ZA7fzPIRfUY45htHe_Djh_bY8wgfMqKI 34m kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Approved,Issued
[root@k8s-master1 c9]# kubectl certificate approve node-csr-YVpk8Sax7vSJ_R-J_MAQDOY6jbzWtR4q9xzVXKxPCMM certificatesigningrequest.certificates.k8s.io/node-csr-YVpk8Sax7vSJ_R-J_MAQDOY6jbzWtR4q9xzVXKxPCMM approved
[root@k8s-master1 c9]# kubectl get csr NAME AGE SIGNERNAME REQUESTOR CONDITION node-csr-Ur8gGqJexjA3yGk5k1wWC8y6Q076x4IKHfQjTkx8k3g 35m kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Approved,Issued node-csr-YVpk8Sax7vSJ_R-J_MAQDOY6jbzWtR4q9xzVXKxPCMM 53s kubernetes.io/kube-apiserver-client-kubelet system:bootstrap:07401b Approved,Issued node-csr-ps3O0TCveqWMEdANu0Psg1qq-WyFR0pWuaFCKAPupXM 35m kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Approved,Issued node-csr-ycjy-hf7hO1ZA7fzPIRfUY45htHe_Djh_bY8wgfMqKI 35m kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Approved,Issued [root@k8s-master1 c9]# kubectl get node NAME STATUS ROLES AGE VERSION k8s-master1 Ready <none> 37m v1.18.6 k8s-node1 Ready <none> 37m v1.18.6 k8s-node2 Ready <none> 3m48s v1.18.6