40.1 kubernetes
阿新 • • 發佈：2018-12-04
Master:
kube-apiserver:6443,8080
kube-scheduler,也是apiserver的客戶端，排程節點上未使用的pod
kube-controller-manager:包含多個控制器如pod控制器，serveice控制器，node控制器等，執行為一個程序
etcd:k8s儲存資源的型別
Node:
kubelet:apiserver的客戶端，執行pod的配置建立容器
docker
kubernetes API API groups 為了便於獨立進行版本演進，kubernetes將API劃分了稱為"API群組"的邏輯集合，每個群組的REST路勁為"/apis/$GROUP_NAME/$VERSION",例如/apis/apps/v1; 核心組core使用簡化的REST路勁/api/v1 每個群組可同時存在多個不通級別的版本，主要包括alpha、beta和stable三個，使用的級別標識有如vlalpha1、v1beat2和v1等； 命令kubectl api-versions
[[email protected] ~ ]#kubectl api-versions
admissionregistration.k8s.io/v1beta1
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1
apiregistration.k8s.io/v1beta1
apps/v1
apps/v1beta1
apps/v1beta2
authentication.k8s.io/v1
組名/版本號
API資源型別 kubernetes系統把管理的絕大多數事物都抽象成了資源，它們分別代表著不同的事務型別，例如Node、Service、Pod、Controller等等； 每種型別均可通過"屬性賦值"進行例項化，從而構建出"物件(Objects)"; kubernetes Object are persistent entities in the Kubernetes system. kubernetes uses these entities to represent the state of your cluster. 物件主要用於描述在叢集中執行的"應用程式(Pod)",以及應用程式相關的控制(Controllers)、配置(ConfigMap和Secret)、服務暴露(Service和Ingress)、儲存(Volume)等； 使用者使用這些物件來規劃、部署、配置、維護和監控應用程式並記錄執行日誌； 每種型別的資源物件都支援響應的一組方法(管理操作)，它們可用標準的HTTP Verb進行表示，例如PUT、delete、POST和GET等；
和解迴圈(Reconciliation Loop) 客戶端向API server提交POST請求以建立物件 通過JSON格式的body提交 YAML格式需要實現完成向JSON的轉換 物件配置資訊保存於etcd中，其定義出的狀態也稱為"期望的狀態(spec)"控制器負責將其建立為kubernetes叢集上的具體物件(活動物件)，並確保其當前狀態(status)與使用者定義的期望狀態相同； status由控制器自行維護，而spec則由使用者進行提交 活動物件在執行過程中因節點故障燈原因可能會在某一時刻導致其status不在吻合於spec 控制器通過和解迴圈(loop)不間斷地監控著相關物件的當前狀態，在物件的當前狀態發生改變時執行合適的操作讓其當前狀態不限接近期望的狀態
圖9-30-2
資源物件的配置格式
API server接受和返回的所有JSON物件都遵循同一個模式，它們都具有"kind"和“apiVersion”欄位，用於標識物件所屬的資源型別、API群組及相關的版本
大多數的物件或型別的資源還需要具有三個巢狀的欄位metadata、spec和status
metadata欄位為資源提供元資料資訊，例如名稱、隸屬的名稱空間和標籤等；
spec使用者定義使用者期望的狀態，不同的資源型別，其狀態的意義各有不同，例如Pod資源最為核心的功能在於執行容器；
status則記錄著活動物件的當前狀態資訊，它由kubernetes系統自行維護，對使用者來說為只讀欄位；
"kubectl api-resoures"命令可以獲取叢集支援使用的所有資源型別
[[email protected] ~ ]#kubectl run myapp --image=ikubernetes/myapp:v1 --replicas=2 --expose --port=80
service/myapp created
deployment.apps/myapp created
[[email protected] ~ ]#kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-6865459dff-4w4hb 0/1 ContainerCreating 0 20s
myapp-6865459dff-x2c82 0/1 ContainerCreating 0 20s
[[email protected] ~ ]#kubectl get pods myapp-6865459dff-4w4hb -o yaml
[[email protected] ~ ]#mkdir manifests
[[email protected] ~ ]#cd manifests/
[[email protected] ~ ]#vim pod-demo.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-demo
namespace: default
spec:
containers:
- name: nginx
image: nginx:1.10-alpine
- name: busybox
image: busybox:latest
imagePullPolicy: IfNotPresent #映象獲取策略
command: ['/bin/sh','-c','sleep 600']
[[email protected] ~ ]#kubectl create -f pod-demo.yaml
pod/pod-demo created
[[email protected] manifests ]#kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-6865459dff-4w4hb 1/1 Running 0 11m
myapp-6865459dff-x2c82 1/1 Running 0 11m
nginx-5c89469986-5tqj2 0/1 ImagePullBackOff 0 1h
nginx-5c89469986-z4mfd 0/1 Unknown 0 3d
pod-demo 0/2 ContainerCreating 0 30s
tomcat-667d6c9-2rzhc 1/1 Unknown 1 3d
tomcat-667d6c9-4qwfn 1/1 Unknown 1 3d
tomcat-667d6c9-glk7r 1/1 Running 0 1h
tomcat-667d6c9-xx6hj 1/1 Running 0 1h
[[email protected] ~ ]#kubectl get pods pod-demo -o yaml
====
What is Pod?
A group of one or more containers that are always co-located and co-scheduled that share the context
Containers in a pod share the same IP address,ports,hostname and storage
MOdeled like a virtual machine
Each container represents one process
Tightly coupled with other containers in the same pod
Pods are scheduled in Nodes
Fundamental unit of deployment in Kuernetes
===
What is Pod?
Containers within the same pod communicate each other using IPC;
Containers can find each other via localhost;
Each container inherits the same of the pod
Each pod has an IP address in a flat shared networking space
Volumes are shared by containers in a pod
圖9-30-3
kubectl
API server:
陳述式命令：kubectl run/expose/delete/get/…
陳述式配置清單：
create -f /PATH/TO/SOMEFILE
delete -f
replace -f
宣告式配置清單：(打補丁)
apply -f /PATH/TO/SOMEFILE
patch -p “”
[[email protected] ~ ]#kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-6865459dff-4w4hb 1/1 Running 0 47m
myapp-6865459dff-x2c82 1/1 Running 0 47m
nginx-5c89469986-5tqj2 0/1 ImagePullBackOff 0 1h
nginx-5c89469986-z4mfd 0/1 Unknown 0 3d
pod-demo 2/2 Running 3 37m
[[email protected] ~ ]#cd manifests/
[[email protected] manifests ]#ls
pod-demo.yaml
[[email protected] manifests ]#kubectl delete -f pod-demo.yaml
pod "pod-demo" deleted
谷歌搜尋kubernetes api reference第一個連結
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.12/
#名稱空間預設為default
[[email protected] manifests ]#kubectl get namespaces
NAME STATUS AGE
default Active 3d
kube-public Active 3d
kube-system Active 3d
#通過命令建立名稱空間
[[email protected] manifests ]#kubectl create namespace dev
namespace/dev created
[[email protected] manifests ]#kubectl get namespaces
NAME STATUS AGE
default Active 3d
dev Active 2s
kube-public Active 3d
kube-system Active 3d
[[email protected] manifests ]#kubectl get ns dev -o yaml
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: 2018-09-30T09:33:20Z
name: dev
resourceVersion: "135006"
selfLink: /api/v1/namespaces/dev
uid: de6349d4-c493-11e8-8632-000c293bcf92
spec:
finalizers:
- kubernetes
status:
phase: Active
#通過配置檔案建立名稱空間
[[email protected] manifests ]#vim ns-demo.yaml
apiVersion: v1
kind: Namespace
metadata:
name: product
[[email protected] manifests ]#kubectl create -f ns-demo.yaml
namespace/product created
[[email protected] manifests ]#kubectl get namespaces
NAME STATUS AGE
default Active 3d
dev Active 1m
kube-public Active 3d
kube-system Active 3d
product Active 4s
[[email protected] manifests ]#vim myapp-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
namespace: dev
spec:
containers:
- {"name":"myapp-container","image":"ikubernetes/myapp:v1"}
等同於一下兩行
- name: myapp-container
image: ikubernetes/myapp:v1
[[email protected] manifests ]#kubectl create -f myapp-pod.yaml
pod/myapp-pod created
[[email protected] manifests ]#kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-6865459dff-4w4hb 1/1 Running 0 1h
myapp-6865459dff-x2c82 1/1 Running 0 1h
nginx-5c89469986-5tqj2 0/1 ImagePullBackOff 0 2h
nginx-5c89469986-z4mfd 0/1 Unknown 0 3d
tomcat-667d6c9-2rzhc 1/1 Unknown 1 3d
tomcat-667d6c9-4qwfn 1/1 Unknown 1 3d
tomcat-667d6c9-glk7r 1/1 Running 0 2h
tomcat-667d6c9-xx6hj 1/1 Running 0 2h
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 12s
[[email protected] manifests ]#kubectl exec -it myapp-pod -n dev -- sh
/ # netstat -tnl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
[[email protected] manifests ]#kubectl logs myapp-pod -n dev
管理Pod物件的容器
映象獲取策略:imagePullPolicy
定義暴露的埠:ports
自定義執行的容器命令:command和args
環境變數:env
共享節點的網路名稱空間:hostNetwork
安全上下文:securityContext
imagePullPolicy: IfNotPresent #映象獲取策略，本地沒有自動下載
imagePullPolicy: Always #從倉庫獲取
imagePullPolicy: Never #映象獲取從不獲取，需要手動獲取
定義DNAT規則
[[email protected] ~ ]#kubectl explain pod
[[email protected] ~ ]#kubectl explain pod.spec
[[email protected] ~ ]#kubectl explain pod.spec.containers
[[email protected] ~ ]#vim pod-demo.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-demo
namespace: default
spec:
containers:
- name: nginx
image: nginx:1.10-alpine
- name: busybox
image: busybox:latest
imagePullPolicy: IfNotPresent #映象獲取策略
command: ['/bin/sh','-c','sleep 600']
ports:
- name: http
containerPort: 80
[[email protected] manifests ]#kubectl delete -f pod-demo.yaml
pod "pod-demo" deleted
[[email protected] manifests ]#kubectl apply -f myapp-pod.yml
[[email protected] ~ ]#vim pod-demo.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-demo
namespace: default
spec:
containers:
- name: nginx
image: nginx:1.10-alpine
- name: busybox
image: busybox:latest
imagePullPolicy: IfNotPresent #映象獲取策略
command: ['/bin/sh']
args: ["-c","while true; do echo $(HOST_ID) >> /tmp/date.txt; sleep 1; done"]
env:
- name: HOST_ID
value: "My_Pod"
[[email protected] manifests ]#kubectl delete -f pod-demo.yaml
pod "pod-demo" deleted
[[email protected] manifests ]#kubectl apply -f myapp-pod.yml
標籤選擇器(Label Selector)
標籤選擇器用於表達標籤的查詢條件或選擇標準，kubernetes API目前支援兩個選擇器
基於等值關係(equality-based)
操作符有=、==和！=三種，其中前兩個意義相同，都表示"等值"關係，最後一個表示"不等"關係
基於集合關係(set-based)
KEY in (VALUE,VALUE2,...)
KEY not in (VALUE1,VALUE2,...)
KEY:所有存在此鍵名標籤的資源
!KEY:所有不存在此鍵名標籤的資源。
使用標籤選擇器時還將遵循以下邏輯：
同時制定的多個選擇器之間的邏輯關係為“與”操作；
使用空值的標籤選擇器意味著每個資源物件豆漿被選中
空的標籤選擇器將無法選出任何資源。
[[email protected] ~ ]#kubectl get pods -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
myapp-pod 1/1 Running 0 1h <none>
[[email protected] ~ ]#kubectl label pods myapp-pod -n dev dhy=haiyang
pod/myapp-pod labeled
[[email protected] ~ ]#kubectl get pods -n dev --show-labels
NAME READY STATUS RESTARTS AGE LABELS
myapp-pod 1/1 Running 0 2h dhy=haiyang
[[email protected] ~ ]#kubectl get pods -n dev -l dhy=haiyang
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 2h
[[email protected] ~ ]#kubectl get pods -n dev -l \!dhy
No resources found.
[[email protected] ~ ]#vim pod-demo.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-demo
namespace: default
labels:
app: dhyapp
relase: beta
annotations:
dhyannotatons/environment: Development
[[email protected] ~ ]#kubectl describe pods -n dev myapp-pod
[[email protected] ~ ]#kubectl annotate pods myapp-pod dhyannotations.com/environ=haiyang --overwrite
Pod生命週期
Pod物件總是應該處於其生命週期過程中一下幾個相位(phase)之一
Pending:API Server建立了Pod資源物件並已存入etcd中，但它尚未被排程完成，或仍處於從倉庫下載映象的過程中；
Running:Pod已經被排程至某節點，並且所有容器都已經被kubelet建立完成；
Succeeded:Pod中的所有容器都已經成功終止並且不會被重啟；
Faild:所有容器都已經終止，但至少有一個容器終止失敗，即容器返回了非0值的退出狀態或已經被系統終止；
Unknown:API Server無法正常獲取到Pod物件的狀態資訊，通常是由於其無法與所在工作節點的kubelet同行所致
Pod生命週期中的重要階段
初始化容器
生命週期鉤子函式
postStart
preStop
容器探測
探測型別
存活狀態探測：liveness porbe
就緒狀態探測：readiness probe
探測行為
ExecAction
TCPSocketAction
HTTPGetAction
#啟動後鉤子
[[email protected] ~ ]#kubectl explain pods.spec.containers.lifecycle
[[email protected] manifests ]#vim podlift.yaml
apiVersion: v1
kind: Pod
metadata:
name: podlifecycle
namespace: dev
spec:
containers:
- name: bbox-container
image: busybox
imagePullPolicy: IfNotPresent
command: ["/bin/sh","-c",sleep 3600]
lifecycle:
postStart:
exec:
command: ["/bin/sh","-c"," mkdir -p /data/html"]
[[email protected] manifests ]#kubectl apply -f podlift.yaml
pod/podlifecycle created
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 3h
podlifecycle 1/1 Running 0 40s
[[email protected] manifests ]#kubectl exec podlifecycle -n dev -- ls /
bin
data
dev
etc
home
proc
root
sys
tmp
usr
var
===
存活狀態探測：liveness porbe
[[email protected] ~ ]#kubectl explain pods.spec.containers.livenessProbe
[[email protected] manifests ]#vim podliveness.yaml
apiVersion: v1
kind: Pod
metadata:
name: podliveness.dhy
namespace: dev
spec:
containers:
- name: bbox-container-liveness
image: ikubernetes/myapp:v1
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
port: 80
failureThreshold: 2
[[email protected] manifests ]#kubectl exec podliveness.dhy -n dev -it -- sh
/ # cd /usr/share/nginx/html/
/usr/share/nginx/html # ls
50x.html index.html
/usr/share/nginx/html # rm -f index.html
/usr/share/nginx/html # exit
#重啟一次了
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 3h
podlifecycle 1/1 Running 0 18m
podliveness.dhy 1/1 Running 1 5m
[[email protected] manifests ]#kubectl apply -f podliveness.yaml
pod/podliveness.dhy created
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 3h
podlifecycle 1/1 Running 0 14m
podliveness.dhy 1/1 Running 0 1m
#檢視詳細資訊
[[email protected] manifests ]#kubectl describe pods podliveness.dhy -n dev
====
就緒狀態探測：readiness probe
[[email protected] manifests ]#vim podreadinessprobe.yaml
apiVersion: v1
kind: Pod
metadata:
name: podreadiness.dhy
namespace: dev
spec:
containers:
- name: bbox-container-liveness
image: ikubernetes/myapp:v1
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
port: 80
failureThreshold: 2
readinessProbe:
exec:
command: ["/bin/sh","-c","ls /data/html"]
periodSeconds: 2
[[email protected] manifests ]#kubectl apply -f podreadinessprobe.yaml
[[email protected] manifests ]#kubectl apply -f podreadinessprobe.yaml
pod/podreadiness.dhy created
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 3h
podlifecycle 1/1 Running 0 24m
podliveness.dhy 1/1 Running 1 11m
podreadiness.dhy 0/1 Running 0 11s
[[email protected] manifests ]#kubectl exec podreadiness.dhy -n dev -- mkdir -p /data/html
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 3h
podlifecycle 1/1 Running 0 26m
podliveness.dhy 1/1 Running 1 12m
podreadiness.dhy 1/1 Running 0 1m
容器的重啟策略
Pod物件因容器程式奔潰或容器申請超出限制的資源等原因都可能導致其被終止，此時是否應該重建它，則取決於其重啟策略(restartPolicy)屬性的定義
Always:但凡Pod物件終止就將其重啟，此為預設設定
OnFailure: 僅在Pod物件出現錯誤時方才將其重啟；
Never:從不重啟；
資源需求及資源限制
容器的計算資源配額
CPU屬於可壓縮(compressible)型資源，即資源額度可按需收縮，而記憶體(當前)則是不可壓縮型資源，對其執行收縮操作可能會導致某種程度的問題
CPU資源的計量方式
一個核心相當於1000個微核心，即1=1000m,0.5=500m
記憶體資源的計量方式
預設單位為位元組，也可以使用E、P、T、G、M和K字尾單位，或Ei、Pi、Ti、Gi、Mi和Ki形式的單位字尾
圖9-30-4
Pod服務質量類別
根據Pod物件的requests和limit屬性，Kubernetes把Pod物件歸類到BestEffort、Burstable和Guarantee三個服務質量類別(Quality of Service,Qos)類別下
Guaranteed：每個容器都為CPU資源設定了具有相同值的requests和limits屬性，以及每個容器都為記憶體資源設定了具有相同值的requests和limits屬性的pod資源會自動歸屬此類別，這類pod資源具有最高優先順序
Burstable:至少有一個容器設定了CPU或記憶體資源的requests屬性，但不滿足Guaranteed類別要求的pod資源自動歸屬此類別，它們具有中等優先順序
BestEffort：未為任何一個容器設定requests或limits屬性的pod資源自動歸屬此類別，它們的優先順序為最低級別
[email protected] manifests ]#vim podresource.yaml
apiVersion: v1
kind: Pod
metadata:
name: podresource.dhy
namespace: dev
spec:
containers:
- name: bbox-container-liveness
image: ikubernetes/myapp:v1
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
port: 80
failureThreshold: 2
resources:
requests:
cpu: 200m
memory: 32Mi
limits:
cpu: 2
memory: 1Gi
[[email protected] manifests ]#kubectl apply -f podresource.yaml
pod/podresource.dhy created
[[email protected] manifests ]#kubectl describe pods podresource.dhy -n dev
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
圖9-30-4
ReplicaSet
A ReplicaSet ensures that a spenified number of pod replicas are running at any given time.
ReplicaSet Spen
Pod Template
Pod Selector
Replicas
Working with ReplicaSets
Deleting a ReplicaSet and its Pods
Deleting just a ReplicaSet
Isolating pods from a ReplicaSet
Scaling a ReplicaSet
[[email protected] manifests ]#vim apiController.yaml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: rs-demo
namespace: dev
spec:
replicas: 2
selector:
matchLabels:
app: myapp
env: dev
template:
metadata:
labels:
app: myapp
env: dev
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
[[email protected] manifests ]#kubectl apply -f apiController.yaml
replicaset.apps/rs-demo created
[[email protected] manifests ]#kubectl get rs -n dev
NAME DESIRED CURRENT READY AGE
rs-demo 2 2 2 11s
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 1 1d
podlifecycle 1/1 Running 11 22h
podliveness.dhy 1/1 Running 2 22h
podreadiness.dhy 0/1 Running 1 22h
podresource.dhy 1/1 Running 1 22h
rs-demo-4z9qw 1/1 Running 0 2m
rs-demo-qm6b7 1/1 Running 0 2m
2個rs-demo，如果要3個，需要更改配置檔案，並且重新提交。或者線上修改
[[email protected] manifests ]#kubectl edit rs rs-demo -n dev
spec:
replicas: 3
NAME READY STATUS RESTARTS AGE
rs-demo-4z9qw 1/1 Running 0 8m
rs-demo-6j5s6 1/1 Running 0 18s
rs-demo-qm6b7 1/1 Running 0 8m
修改配置版本為v2，則需要刪除一個，配置檔案建立補足一個為v2的版本
kubectl delete pods rs-demo-qm6b7 -n dev
這樣不太方便，可以通過Deployment自動管理
===
Deployment
A Deployment controller provides declarative updates for Pods and ReplicaSets
you describe a desired state in a Deployment object,and the Deployment controller changes the actual state to the desired state at a controlled rate.
The following are typical use cases for Deployments:
Create a Deployment to rollout a ReplicaSet
Declare the new state of the Pods by updating the PodTemplateSpec of the Deployment.
Rollback to an earlier Deployment revision if the current state of the Deployment is not stable
Scale up the Deployment to facilitate more load
Pause the Deployment to apply multiple fixes to its PodTemplateSpec and then resume it to start a new rollout. #canary釋出
Use the status of the Deployment as an indicator that a rollout has stuck.
Clean up older ReplicaSets that you don't need anymore
[[email protected] manifests ]#vim myapp1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: rs-demo
namespace: dev
spec:
replicas: 2
selector:
matchLabels:
app: myapp-pod
env: dev
template:
metadata:
labels:
app: myapp-pod
env: dev
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
[[email protected] manifests ]#kubectl apply -f myapp1.yaml
deployment.apps/rs-demo created
[[email protected] manifests ]#kubectl get deploy -o wide -n dev
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
rs-demo 2 2 2 2 17s myapp ikubernetes/myapp:v1 app=myapp-pod,env=dev
[[email protected] manifests ]#kubectl get rs -n dev
NAME DESIRED CURRENT READY AGE
rs-demo 3 3 3 27m
rs-demo-589c774986 2 2 2 1m
[[email protected] manifests ]#kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 1 1d
podlifecycle 1/1 Running 11 23h
podliveness.dhy 1/1 Running 2 23h
podreadiness.dhy 0/1 Running 1 22h
podresource.dhy 1/1 Running 1 22h
rs-demo-4z9qw 1/1 Running 0 28m
rs-demo-589c774986-gqkgn 1/1 Running 0 2m
rs-demo-589c774986-wnwtd 1/1 Running 0 2m
rs-demo-6j5s6 1/1 Running 0 20m
rs-demo-hj8kb 1/1 Running 0 15m