[Kubernetes]The connection to the server <master>:6443 was refused - did you specify the right host or port?
阿新 • • 發佈：2018-11-07
不知道怎麼用著用著，使用kubectl 時報錯，錯誤如下：
[email protected]2-2:~# kubectl get pod The connection to the server 107.105.136.28:6443 was refused - did you specify the right host or port?
$systemctl status kubelet
Mar 07 20:42:25 server1 kubelet[14619]: E0307 20:42:25.908557 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://107.105.136.28:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:26 server1 kubelet[14619]: E0307 20:42:26.718263 14619 eviction_manager.go:238] eviction manager: unexpected err: failed to getnode info: node "server1" not found Mar 07 20:42:26 server1 kubelet[14619]: E0307 20:42:26.906939 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: Failed to list *v1.Service: Get https://107.105.136.28:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refusedMar 07 20:42:26 server1 kubelet[14619]: E0307 20:42:26.907933 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: Failed to list *v1.Node: Get https://107.105.136.28:6443/api/v1/nodes?fieldSelector=metadata.name%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:26 server1 kubelet[14619]: E0307 20:42:26.908994 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://107.105.136.28:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:27 server1 kubelet[14619]: E0307 20:42:27.392766 14619 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed pulling image "gcr.io/google_containers/pause-amd64:3.0": Error response from daemon: Get https://gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded w Mar 07 20:42:27 server1 kubelet[14619]: E0307 20:42:27.392788 14619 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "kube-apiserver-server1_kube-system(7c74aa0f4b9044a62ba3fc2b222b6a49)" failed: rpc error: code = Unknown desc = failed pulling image "gcr.io/google_containers/pause-amd64:3.0": Error response from daemon: Get https://gcr.io/v2/: net/http: request c Mar 07 20:42:27 server1 kubelet[14619]: E0307 20:42:27.392795 14619 kuberuntime_manager.go:647] createPodSandbox for pod "kube-apiserver-server1_kube-system(7c74aa0f4b9044a62ba3fc2b222b6a49)" failed: rpc error: code = Unknown desc = failed pulling image "gcr.io/google_containers/pause-amd64:3.0": Error response from daemon: Get https://gcr.io/v2/: net/http: request Mar 07 20:42:27 server1 kubelet[14619]: E0307 20:42:27.392823 14619 pod_workers.go:186] Error syncing pod 7c74aa0f4b9044a62ba3fc2b222b6a49 ("kube-apiserver-server1_kube-system(7c74aa0f4b9044a62ba3fc2b222b6a49)"), skipping: failed to "CreatePodSandbox" for "kube-apiserver-server1_kube-system(7c74aa0f4b9044a62ba3fc2b222b6a49)" with CreatePodSandboxError: "CreatePodSandbo Mar 07 20:42:27 server1 kubelet[14619]: E0307 20:42:27.907221 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: Failed to list *v1.Service: Get https://107.105.136.28:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:27 server1 kubelet[14619]: E0307 20:42:27.908220 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: Failed to list *v1.Node: Get https://107.105.136.28:6443/api/v1/nodes?fieldSelector=metadata.name%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:27 server1 kubelet[14619]: E0307 20:42:27.909242 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://107.105.136.28:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:28 server1 kubelet[14619]: I0307 20:42:28.239272 14619 kubelet_node_status.go:273] Setting node annotation to enable volume controller attach/detach Mar 07 20:42:28 server1 kubelet[14619]: I0307 20:42:28.246581 14619 kubelet_node_status.go:82] Attempting to register node server1 Mar 07 20:42:28 server1 kubelet[14619]: E0307 20:42:28.246744 14619 kubelet_node_status.go:106] Unable to register node "server1" with API server: Post https://107.105.136.28:6443/api/v1/nodes: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:28 server1 kubelet[14619]: E0307 20:42:28.907491 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: Failed to list *v1.Service: Get https://107.105.136.28:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:28 server1 kubelet[14619]: E0307 20:42:28.908477 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: Failed to list *v1.Node: Get https://107.105.136.28:6443/api/v1/nodes?fieldSelector=metadata.name%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:28 server1 kubelet[14619]: E0307 20:42:28.909606 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://107.105.136.28:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:29 server1 kubelet[14619]: E0307 20:42:29.907804 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: Failed to list *v1.Service: Get https://107.105.136.28:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:29 server1 kubelet[14619]: E0307 20:42:29.908776 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: Failed to list *v1.Node: Get https://107.105.136.28:6443/api/v1/nodes?fieldSelector=metadata.name%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:29 server1 kubelet[14619]: E0307 20:42:29.909785 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://107.105.136.28:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:30 server1 kubelet[14619]: E0307 20:42:30.908131 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: Failed to list *v1.Service: Get https://107.105.136.28:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:30 server1 kubelet[14619]: E0307 20:42:30.909155 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: Failed to list *v1.Node: Get https://107.105.136.28:6443/api/v1/nodes?fieldSelector=metadata.name%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:30 server1 kubelet[14619]: E0307 20:42:30.910228 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://107.105.136.28:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:31 server1 kubelet[14619]: E0307 20:42:31.908495 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:465: Failed to list *v1.Service: Get https://107.105.136.28:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:31 server1 kubelet[14619]: E0307 20:42:31.909392 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:474: Failed to list *v1.Node: Get https://107.105.136.28:6443/api/v1/nodes?fieldSelector=metadata.name%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:31 server1 kubelet[14619]: E0307 20:42:31.910461 14619 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://107.105.136.28:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dserver1&limit=500&resourceVersion=0: dial tcp 107.105.136.28:6443: getsockopt: connection refused Mar 07 20:42:32 server1 kubelet[14619]: E0307 20:42:32.018611 14619 event.go:209] Unable to write event: 'Patch https://107.105.136.28:6443/api/v1/namespaces/default/events/server1.1519a1c427a76a94: dial tcp 107.105.136.28:6443: getsockopt: connection refused' (may retry after sleeping)
6443是Kubernetes叢集APIServer的埠，因此猜測是APIServer掛掉了，因此檢視正在執行的程序
$ps -aux | grep "kube"
發現果然沒有有關的程序，因此重啟相關的服務
$systemctl restart kube-apiserver $systemctl restart kube-controller-manager $systemctl restart kube-scheduler
檢視狀態，可以使用kubectl命令了
[email protected]2-3:~# kubectl get componentstatus NAME STATUS MESSAGE ERROR scheduler Healthy ok controller-manager Healthy ok etcd-1 Healthy {"health":"true"} etcd-2 Healthy {"health":"true"} etcd-0 Healthy {"health":"true"}