kubeadm安裝Kubernetes，叢集狀態檢測時，master一直處於notready狀態

[root@master ~]# kubectl get nodes
NAME     STATUS     ROLES    AGE    VERSION
master   NotReady   master   32m    v1.16.0
node1    NotReady   <none>   8m2s   v1.16.0

找問題，先檢視pods狀態

[root@master ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                             READY   STATUS    RESTARTS   AGE
kube-system   coredns-58cc8c89f4-pkx82         0/1     Pending   0          34m
kube-system   coredns-58cc8c89f4-sddpq         0/1     Pending   0          34m
kube-system   etcd-master                      1/1     Running   0          34m
kube-system   kube-apiserver-master            1/1     Running   0          34m
kube-system   kube-controller-manager-master   1/1     Running   0          33m
kube-system   kube-proxy-4fj8z                 1/1     Running   0          10m
kube-system   kube-proxy-v54nh                 1/1     Running   0          34m
kube-system   kube-scheduler-master            1/1     Running   0          34m

 

發現coredns一直處於pending狀態，再進一步看kuberctl.services日誌

[root@master ~]# journalctl -f -u kubelet.service
-- Logs begin at Mon 2019-09-23 22:55:58 CST. --
Sep 24 01:33:44 master kubelet[6213]: E0924 01:33:44.107180    6213 kubelet.go:2187] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Sep 24 01:33:46 master kubelet[6213]: W0924 01:33:46.416805    6213 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d
 

看到是網路的問題，應該是fu

[root@master ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml
clusterrole.rbac.authorization.k8s.io/flannel unchanged
clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
serviceaccount/flannel unchanged
configmap/kube-flannel-cfg unchanged
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"

 

解決辦法，換個連結

[root@master ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel configured
clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
serviceaccount/flannel unchanged
configmap/kube-flannel-cfg unchanged
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created

檢視node狀態，發現還是notready。檢視日誌

journalctl -f -u kubelet.service 
-- Logs begin at Mon 2019-09-23 22:55:58 CST. --
Sep 24 14:36:17 master kubelet[6213]: E0924 14:36:17.023967    6213 kubelet.go:2187] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Sep 24 14:36:19 master kubelet[6213]: W0924 14:36:19.214442    6213 cni.go:202] Error validating CNI config &{cbr0  false [0xc0006a22a0 0xc0006a2360] [123 10 32 32 34 110 97 109 101 34 58 32 34 99 98 114 48 34 44 10 32 32 34 112 108 117 103 105 110 115 34 58 32 91 10 32 32 32 32 123 10 32 32 32 32 32 32 34 116 121 112 101 34 58 32 34 102 108 97 110 110 101 108 34 44 10 32 32 32 32 32 32 34 100 101 108 101 103 97 116 101 34 58 32 123 10 32 32 32 32 32 32 32 32 34 104 97 105 114 112 105 110 77 111 100 101 34 58 32 116 114 117 101 44 10 32 32 32 32 32 32 32 32 34 105 115 68 101 102 97 117 108 116 71 97 116 101 119 97 121 34 58 32 116 114 117 101 10 32 32 32 32 32 32 125 10 32 32 32 32 125 44 10 32 32 32 32 123 10 32 32 32 32 32 32 34 116 121 112 101 34 58 32 34 112 111 114 116 109 97 112 34 44 10 32 32 32 32 32 32 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 32 123 10 32 32 32 32 32 32 32 32 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 32 116 114 117 101 10 32 32 32 32 32 32 125 10 32 32 32 32 125 10 32 32 93 10 125 10]}: [plugin flannel does not support config version ""]
Sep 24 14:36:19 master kubelet[6213]: W0924 14:36:19.214617    6213 cni.go:237] Unable to update cni config: no valid networks found in /etc/cni/net.d

發現報錯plugin flannel does not support config version ，修改配置檔案

vim /etc/cni/net.d/10-flannel.conflist
//加上cni的版本號
//檔案內容如下
{
  "name": "cbr0",
  "cniVersion": "0.2.0",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}

修改後，執行

systemctl daemon-reload 

再檢視叢集狀態，發現master正常了，處於ready狀態；但是node
1節點還是處於notready狀態

[root@master ~]# kubectl get nodes
NAME     STATUS     ROLES    AGE   VERSION
master   Ready      master   14h   v1.16.0
node1    NotReady   <none>   13h   v1.16.0

解決辦法：去到node1，檢視kubectl日誌，發現還是報錯no valid networks found in /etc/cni/net.d
將node1中也加上cni的版本號，重新啟動，即可看到叢集狀態變為正常

[root@master ~]# kubectl get nodes
NAME     STATUS   ROLES    AGE   VERSION
master   Ready    master   14h   v1.16.0
node1    Ready    <none>   14h   v1.16.0

參考資料：
kubeadm安裝Kubernetes 1.14最佳https://www.kubernetes.org.cn/5462.html
kubernetes安裝過程中錯誤（kube-dns 狀態一直是Pending，master節點是NotReady）
https://blog.csdn.net/u013355826/article/details/82786649
How to fix Flannel CNI plugin. Error: [plugin flannel does not support config version “”]
https://stackoverflow.com/questions/58037620/how-to-fix-flannel-cni-plugin-error-plugin-flannel-does-not-support-config-ve
kubernetes nodes notready 解決思路
https://blog.csdn.net/qq_21816375/article/details/80222689