(轉)Kubernetes之kubectl常用命令使用指南:2:故障排查
kubectl是一個用於操作kubernetes叢集的命令列介面,通過利用kubectl的各種命令可以實現各種功能,是在使用kubernetes中非常常用的工具。這裡我們會通過一些簡單的例項來展現其中一些高頻命令的使用方法。
常用命令
kubectl故障排查相關,本文將會簡單介紹一下如下命令
項番 | 命令 | 說明 |
---|---|---|
No.1 | version | 顯示客戶端和伺服器側版本資訊 |
No.2 | api-versions | 以group/version的格式顯示伺服器側所支援的API版本 |
No.3 | explain | 顯示資源文件資訊 |
No.4 | get | 取得確認物件資訊列表 |
No.5 | describe | 取得確認物件的詳細資訊 |
No.6 | logs | 取得pod中容器的log資訊 |
No.7 | exec | 在容器中執行一條命令 |
No.8 | cp | 從容器考出或向容器考入檔案 |
No.9 | attach | Attach到一個執行中的容器上 |
事前準備
kubectl version
version命令用於確認客戶端和伺服器側的版本資訊,不同的版本的情況變化可能很大,所以故障排除時首先也需要確認的是現場環境的版本資訊。
從下面可以清楚地看到,本文驗證時所使用的版本為1.5.2
[root@ku8-1 tmp]# kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:52:34Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
叢集構成
一主三從的Kubernetes叢集
項番 | 型別 | Hostname | IP |
---|---|---|---|
No.1 | Master | ku8-1 | 192.168.32.131 |
No.1 | Node | ku8-2 | 192.168.32.132 |
No.1 | Node | ku8-3 | 192.168.32.133 |
No.1 | Node | ku8-4 | 192.168.32.134 |
[root@ku8-1 tmp]# kubectl get nodes
NAME STATUS AGE
192.168.32.132 Ready 12m
192.168.32.133 Ready 11m
192.168.32.134 Ready 11m
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
kubectl api-versions
使用api-versions命令可以列出當前版本的kubernetes的伺服器端所支援的api版本資訊。
[root@ku8-1 tmp]# kubectl api-versions
apps/v1beta1
authentication.k8s.io/v1beta1
authorization.k8s.io/v1beta1
autoscaling/v1
batch/v1
certificates.k8s.io/v1alpha1
extensions/v1beta1
policy/v1beta1
rbac.authorization.k8s.io/v1alpha1
storage.k8s.io/v1beta1
v1
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
kubectl explain
使用kubectl explain可以和kubectl help一樣進行輔助的功能確認,使用它可以瞭解各個部分的說明和組成部分。比如如下可以看到對rc的說明,在故障排除時作用並不具有太大作用,到是可以多讀讀加深一下對各個部分的理解。
[root@ku8-1 ~]# kubectl explain rc
DESCRIPTION:
ReplicationController represents the configuration of a replication controller.
FIELDS:
apiVersion <string>
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#resources
kind <string>
Kind is a string value representing the REST resource this object
represents. Servers may infer this from the endpoint the client submits
requests to. Cannot be updated. In CamelCase. More info:
http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#types-kinds
metadata <Object>
If the Labels of a ReplicationController are empty, they are defaulted to
be the same as the Pod(s) that the replication controller manages. Standard
object's metadata. More info:
http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata
spec <Object>
Spec defines the specification of the desired behavior of the replication
controller. More info:
http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status
status <Object>
Status is the most recently observed status of the replication controller.
This data may be out of date by some window of time. Populated by the
system. Read-only. More info:
http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status
[root@ku8-1 ~]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
explain命令能夠確認的資訊類別
其所能支援的類別如下:
類別 |
---|
clusters (僅對federation apiservers有效) |
componentstatuses (縮寫 cs) |
configmaps (縮寫 cm) |
daemonsets (縮寫 ds) |
deployments (縮寫 deploy) |
endpoints (縮寫 ep) |
events (縮寫 ev) |
horizontalpodautoscalers (縮寫 hpa) |
ingresses (縮寫 ing) |
jobs |
limitranges (縮寫 limits) |
namespaces (縮寫 ns) |
networkpolicies |
nodes (縮寫 no) |
persistentvolumeclaims (縮寫 pvc) |
persistentvolumes (縮寫 pv) |
pods (縮寫 po) |
podsecuritypolicies (縮寫 psp) |
podtemplates |
replicasets (縮寫 rs) |
replicationcontrollers (縮寫 rc) |
resourcequotas (縮寫 quota) |
secrets |
serviceaccounts (縮寫 sa) |
services (縮寫 svc) |
statefulsets |
storageclasses |
thirdpartyresources |
事前準備
剩下的一些命令需要事前作一些準備,我們還是用上篇文章所用的yaml檔案建立mysql和sonarqube的Deployment和pod。
yaml檔案準備
[root@ku8-1 tmp]# ls yamls
mysql.yaml sonar.yaml
[root@ku8-1 tmp]# cat yamls/mysql.yaml
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: mysql
spec:
replicas: 1
template:
metadata:
labels:
name: mysql
spec:
containers:
- name: mysql
image: 192.168.32.131:5000/mysql:5.7.16
ports:
- containerPort: 3306
protocol: TCP
env:
- name: MYSQL_ROOT_PASSWORD
value: "hello123"
[root@ku8-1 tmp]# cat yamls/sonar.yaml
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: sonarqube
spec:
replicas: 1
template:
metadata:
labels:
name: sonarqube
spec:
containers:
- name: sonarqube
image: 192.168.32.131:5000/sonarqube:5.6.5
ports:
- containerPort: 9000
protocol: TCP
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
啟動
[root@ku8-1 tmp]# kubectl create -f yamls/
deployment "mysql" created
deployment "sonarqube" created
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
kubectl get
使用get命令確認所創建出來的pod和deployment的資訊
確認pod
可以看到創建出來的pod的所有資訊,也可以使用Kubectl get po進行確認
[root@ku8-1 tmp]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-478535978-1dnm2 1/1 Running 0 34s
sonarqube-3574384362-m7mdq 1/1 Running 0 34s
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
確認deployment
可以看到創建出來的deployment的所有資訊
[root@ku8-1 tmp]# kubectl get deployments
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
mysql 1 1 1 1 41s
sonarqube 1 1 1 1 41s
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
如果希望得到更加詳細一點的資訊,可以加上-o wide引數,比如對pods可以看到此pod在哪個node上執行,此pod的叢集IP是多少也被一併顯示了
[root@ku8-1 tmp]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE
mysql-478535978-1dnm2 1/1 Running 0 2m 172.200.44.2 192.168.32.133
sonarqube-3574384362-m7mdq 1/1 Running 0 2m 172.200.59.2 192.168.32.134
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
確認node資訊
顯示node的資訊
[root@ku8-1 tmp]# kubectl get nodes -o wide
NAME STATUS AGE EXTERNAL-IP
192.168.32.132 Ready 6h <none>
192.168.32.133 Ready 6h <none>
192.168.32.134 Ready 6h <none>
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
確認namespace資訊
列出所有的namespace
[root@ku8-1 tmp]# kubectl get namespaces
NAME STATUS AGE
default Active 6h
kube-system Active 6h
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
get命令能夠確認的資訊類別
使用node/pod/event/namespaces等結合起來,能夠獲取叢集基本資訊和狀況, 其所能支援的類別如下:
類別 |
---|
clusters (僅對federation apiservers有效) |
componentstatuses (縮寫 cs) |
configmaps (縮寫 cm) |
daemonsets (縮寫 ds) |
deployments (縮寫 deploy) |
endpoints (縮寫 ep) |
events (縮寫 ev) |
horizontalpodautoscalers (縮寫 hpa) |
ingresses (縮寫 ing) |
jobs |
limitranges (縮寫 limits) |
namespaces (縮寫 ns) |
networkpolicies |
nodes (縮寫 no) |
persistentvolumeclaims (縮寫 pvc) |
persistentvolumes (縮寫 pv) |
pods (縮寫 po) |
podsecuritypolicies (縮寫 psp) |
podtemplates |
replicasets (縮寫 rs) |
replicationcontrollers (縮寫 rc) |
resourcequotas (縮寫 quota) |
secrets |
serviceaccounts (縮寫 sa) |
services (縮寫 svc) |
statefulsets |
storageclasses |
thirdpartyresources |
kubectl describe
確認node詳細資訊
一般使用get命令取得node資訊,然後使用describe確認詳細資訊。
[root@ku8-1 tmp]# kubectl get nodes
NAME STATUS AGE
192.168.32.132 Ready 6h
192.168.32.133 Ready 6h
192.168.32.134 Ready 6h
[root@ku8-1 tmp]# kubectl describe node 192.168.32.132
Name: 192.168.32.132
Role:
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/hostname=192.168.32.132
Taints: <none>
CreationTimestamp: Wed, 28 Jun 2017 23:06:22 -0400
Phase:
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
OutOfDisk False Thu, 29 Jun 2017 05:52:07 -0400 Wed, 28 Jun 2017 23:06:22 -0400 KubeletHasSufficientDisk kubelet has sufficient disk space available
MemoryPressure False Thu, 29 Jun 2017 05:52:07 -0400 Wed, 28 Jun 2017 23:06:22 -0400 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 29 Jun 2017 05:52:07 -0400 Wed, 28 Jun 2017 23:06:22 -0400 KubeletHasNoDiskPressure kubelet has no disk pressure
Ready True Thu, 29 Jun 2017 05:52:07 -0400 Wed, 28 Jun 2017 23:06:34 -0400 KubeletReady kubelet is posting ready status
Addresses: 192.168.32.132,192.168.32.132,192.168.32.132
Capacity:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 1
memory: 2032128Ki
pods: 110
Allocatable:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 1
memory: 2032128Ki
pods: 110
System Info:
Machine ID: 22718f24279240be9fe0c469187f901a
System UUID: 9F584D56-F5B3-FAB8-3985-938D67451312
Boot ID: fe3b2606-37ee-4b07-8de2-438fe29bf765
Kernel Version: 3.10.0-514.el7.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://1.13.1
Kubelet Version: v1.5.2
Kube-Proxy Version: v1.5.2
ExternalID: 192.168.32.132
Non-terminated Pods: (0 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.
CPU Requests CPU Limits Memory Requests Memory Limits
------------ ---------- --------------- -------------
0 (0%) 0 (0%) 0 (0%) 0 (0%)
No events.
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
確認pod
確認某一pod詳細資訊
[root@ku8-1 tmp]# kubectl describe pod mysql-478535978-1dnm2
Name: mysql-478535978-1dnm2
Namespace: default
Node: 192.168.32.133/192.168.32.133
Start Time: Thu, 29 Jun 2017 05:04:21 -0400
Labels: name=mysql
pod-template-hash=478535978
Status: Running
IP: 172.200.44.2
Controllers: ReplicaSet/mysql-478535978
Containers:
mysql:
Container ID: docker://47ef1495e86f4b69414789e81081fa55b837dafe9e47944894e7cb3733700410
Image: 192.168.32.131:5000/mysql:5.7.16
Image ID: docker-pullable://192.168.32.131:5000/mysql@sha256:410b279f6827492da7a355135e6e9125849f62eeca76429974a534f021852b58
Port: 3306/TCP
State: Running
Started: Thu, 29 Jun 2017 05:04:22 -0400
Ready: True
Restart Count: 0
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-dzs1w (ro)
Environment Variables:
MYSQL_ROOT_PASSWORD: hello123
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-dzs1w:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-dzs1w
QoS Class: BestEffort
Tolerations: <none>
No events.
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
確認deployment詳細資訊
確認某一deployment的詳細資訊
[root@ku8-1 tmp]# kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
mysql 1 1 1 1 1h
sonarqube 1 1 1 1 1h
[root@ku8-1 tmp]# kubectl describe deployment mysql
Name: mysql
Namespace: default
CreationTimestamp: Thu, 29 Jun 2017 05:04:21 -0400
Labels: name=mysql
Selector: name=mysql
Replicas: 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 1 max unavailable, 1 max surge
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: mysql-478535978 (1/1 replicas created)
No events.
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
describe命令能夠確認的資訊
describe命令所能支援的類別如下:
類別 |
---|
clusters (僅對federation apiservers有效) |
componentstatuses (縮寫 cs) |
configmaps (縮寫 cm) |
daemonsets (縮寫 ds) |
deployments (縮寫 deploy) |
endpoints (縮寫 ep) |
events (縮寫 ev) |
horizontalpodautoscalers (縮寫 hpa) |
ingresses (縮寫 ing) |
jobs |
limitranges (縮寫 limits) |
namespaces (縮寫 ns) |
networkpolicies |
nodes (縮寫 no) |
persistentvolumeclaims (縮寫 pvc) |
persistentvolumes (縮寫 pv) |
pods (縮寫 po) |
podsecuritypolicies (縮寫 psp) |
podtemplates |
replicasets (縮寫 rs) |
replicationcontrollers (縮寫 rc) |
resourcequotas (縮寫 quota) |
secrets |
serviceaccounts (縮寫 sa) |
services (縮寫 svc) |
statefulsets |
storageclasses |
thirdpartyresources |
#kubectl logs
類似於docker logs,使用kubectl logs能夠取出pod中映象的log,也是故障排除時候的重要資訊
[root@ku8-1 tmp]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-478535978-1dnm2 1/1 Running 0 1h
sonarqube-3574384362-m7mdq 1/1 Running 0 1h
[root@ku8-1 tmp]# kubectl logs mysql-478535978-1dnm2
Initializing database
...
2017-06-29T09:04:37.081939Z 0 [Note] Event Scheduler: Loaded 0 events
2017-06-29T09:04:37.082097Z 0 [Note] mysqld: ready for connections.
Version: '5.7.16' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
kubectl exec
exec命令用於到容器中執行一條命令,比如下述命令用於到mysql的映象中執行hostname命令
[root@ku8-1 tmp]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-478535978-1dnm2 1/1 Running 0 1h
sonarqube-3574384362-m7mdq 1/1 Running 0 1h
[root@ku8-1 tmp]# kubectl exec mysql-478535978-1dnm2 hostname
mysql-478535978-1dnm2
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
更為常用的方式則是登陸到pod中,在有條件的時候,進行故障發生時的現場確認,這種方式是最為直接有效和快速,但是對許可權要求也較多。
[root@ku8-1 tmp]# kubectl exec -it mysql-478535978-1dnm2 sh
# hostname
mysql-478535978-1dnm2
#
- 1
- 2
- 3
- 4
kubectl cp
用於pod和外部的檔案交換,比如如下示例瞭如何在進行內外檔案交換。
在pod中建立一個檔案message.log
[root@ku8-1 tmp]# kubectl exec -it mysql-478535978-1dnm2 sh
# pwd
/
# cd /tmp
# echo "this is a message from `hostname`" >message.log
# cat message.log
this is a message from mysql-478535978-1dnm2
# exit
[root@ku8-1 tmp]#
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
拷貝出來並確認
[root@ku8-1 tmp]# kubectl cp mysql-478535978-1dnm2:/tmp/message.log message.log
tar: Removing leading `/' from member names
[root@ku8-1 tmp]# cat message.log
this is a message from mysql-478535978-1dnm2
[root@ku8-1 tmp]#
[root@ku8-1 tmp]# echo "information added in `hostname`" >>message.log
[root@ku8-1 tmp]# cat message.log
this is a message from mysql-478535978-1dnm2
information added in ku8-1
[root@ku8-1 tmp]# kubectl cp message.log mysql-478535978-1dnm2:/tmp/message.log
[root@ku8-1 tmp]#
更改message.log並拷貝回pod
[root@ku8-1 tmp]# echo "information added in `hostname`" >>message.log
[root@ku8-1 tmp]# cat message.log
this is a message from mysql-478535978-1dnm2
information added in ku8-1
[root@ku8-1 tmp]# kubectl cp message.log mysql-478535978-1dnm2:/tmp/message.log
[root@ku8-1 tmp]#
[root@ku8-1 tmp]# echo "information added in `hostname`" >>message.log
[root@ku8-1 tmp]# cat message.log
this is a message from mysql-478535978-1dnm2
information added in ku8-1
[root@ku8-1 tmp]# kubectl cp message.log mysql-478535978-1dnm2:/tmp/message.log
[root@ku8-1 tmp]#
確認更改後的資訊
[root@ku8-1 tmp]# kubectl exec mysql-478535978-1dnm2 cat /tmp/message.log
this is a message from mysql-478535978-1dnm2
information added in ku8-1
[root@ku8-1 tmp]#
[root@ku8-1 tmp]# kubectl exec mysql-478535978-1dnm2 cat /tmp/message.log
this is a message from mysql-478535978-1dnm2
information added in ku8-1
[root@ku8-1 tmp]#
kubectl attach
類似於docker attach的功能,用於取得實時的類似於kubectl logs的資訊
[root@ku8-1 tmp]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-478535978-1dnm2 1/1 Running 0 1h
sonarqube-3574384362-m7mdq 1/1 Running 0 1h
[root@ku8-1 tmp]# kubectl attach sonarqube-3574384362-m7mdq
If you don't see a command prompt, try pressing enter.
[root@ku8-1 tmp]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-478535978-1dnm2 1/1 Running 0 1h
sonarqube-3574384362-m7mdq 1/1 Running 0 1h
[root@ku8-1 tmp]# kubectl attach sonarqube-3574384362-m7mdq
If you don't see a command prompt, try pressing enter.
kubectl cluster-info
使用cluster-info和cluster-info dump也能取出一些資訊,尤其是你需要看整體的全部資訊的時候一條命令一條命令的執行不如kubectl cluster-info dump來的快一些
[root@ku8-1 tmp]# kubectl cluster-info
Kubernetes master is running at http://localhost:8080
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@ku8-1 tmp]#
[root@ku8-1 tmp]# kubectl cluster-info
Kubernetes master is running at http://localhost:8080
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@ku8-1 tmp]#
總結
這篇文章中介紹了九個kubectl的常用命令,利用它們在故障確認和排查中非常有效。