Kebernetes學習總結(5) Ingress
Ingress相關組件:
1) Ingress controlle
kubernetes中的controller有很多,比如CronJob、DeamonSet、Deployment、ReplicationSet、StatefulSet等,它們的作用就是監控集群的變化,使集群始終保持期望的最終狀態(yml文件)。同理Ingress controller的作用就是實時感知Ingress路由規則集合的變化,再與Api Server交互,獲取Service、Pod在集群中的IP等信息,然後發送給反向代理web服務器(ingress pods),刷新其路由配置信息,這就是它的服務發現機制。
2) 反向代理web服務器(ingress pods) 負責轉發外部請求到後端service,比如Nginx、Apache、traefik等等
4) NodePort類型的service 負責將外部流量引入kubernetes cluster,然後由反向代理web服務器處理。
本篇總結中的示例大致如下:首先部署ingress controller及 ingress pods;然後利用NodePort類型的service將外部流最引入kubernetes cluster中;然後再定義後端deployment及後端service;然後定義ingress 轉發規則集合;最後訪問測試。
1) 部署ingress pods
[root@docker79 ingress]# mkdir example
[root@docker79 ingress]# cd example/
[root@docker79 example]# kubectl apply -f .
configmap/nginx-configuration created
deployment.extensions/default-http-backend created
service/default-http-backend created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
namespace/ingress-nginx configured
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
configmap/tcp-services created
configmap/udp-services created
deployment.extensions/nginx-ingress-controller created
[root@docker79 example]#
[root@docker79 example]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
default-http-backend-6586bc58b6-8rxtp 1/1 Running 0 59m
nginx-ingress-controller-6bd7c597cb-xdrcm 1/1 Running 0 59m
[root@docker79 example]#
2) 在ingress-nginx 名稱空間定義nodePort類型的service 引入外部流量,過程如下所示:
[root@docker79 ingress]# cat nginx-frontend-ingress-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-frontend-ingress-svc
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 30080
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 30443
selector:
app.kubernetes.io/name: ingress-nginx
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f nginx-frontend-ingress-svc.yaml
service/nginx-frontend-ingress-svc created
[root@docker79 ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-http-backend ClusterIP 10.102.71.36 <none> 80/TCP 1h
nginx-frontend-ingress-svc NodePort 10.103.52.44 <none> 80:30080/TCP,443:30443/TCP 11s
[root@docker79 ingress]#
3) 下例中部署兩個後端nginx-backend-deploy、tomcat-backend-deploy。
首先部署nginx-backend-deploy及nginx-backend-svc,過程如下:
[root@docker79 ingress]# vim backend-nginx.yaml
[root@docker79 ingress]# cat backend-nginx.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-backend-svc
namespace: default
spec:
selector:
app: nginx-backend
release: canary
ports:
- name: http
targetPort: 80
port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-backend-deploy
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: nginx-backend
release: canary
template:
metadata:
labels:
app: nginx-backend
release: canary
spec:
containers:
- name: nginx-container
image: nginx:1.15-alpine
ports:
- name: http
containerPort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f backend-nginx.yaml
service/nginx-backend-svc created
deployment.apps/nginx-backend-deploy created
[root@docker79 ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-backend-deploy-9c9b85bf4-6587j 1/1 Running 0 8s
nginx-backend-deploy-9c9b85bf4-k64tv 1/1 Running 0 8s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d
nginx-backend-svc ClusterIP 10.110.150.187 <none> 80/TCP 1m
[root@docker79 ingress]#
然後部署tomcat-backend-deploy及tomcat-backend-svc,過程如下:
[root@docker79 ingress]# vim backend-tomcat.yaml
[root@docker79 ingress]# cat backend-tomcat.yaml
apiVersion: v1
kind: Service
metadata:
name: tomcat-backend-svc
namespace: default
spec:
selector:
app: tomcat-backend
release: canary
ports:
- name: http
port: 8080
targetPort: 8080
- name: ajp
port: 8009
targetPort: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deploy
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: tomcat-backend
release: canary
template:
metadata:
labels:
app: tomcat-backend
release: canary
spec:
containers:
- name: tomcat
image: tomcat:8.5.32-jre8-alpine
ports:
- name: http
containerPort: 8080
- name: ajp
containerPort: 8009
[root@docker79 ingress]# kubectl apply -f backend-tomcat.yaml
service/tomcat-backend-svc created
deployment.apps/tomcat-deploy created
[root@docker79 ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-backend-deploy-9c9b85bf4-6587j 1/1 Running 0 24m
nginx-backend-deploy-9c9b85bf4-k64tv 1/1 Running 0 24m
tomcat-deploy-7b4c97ff9f-fv8rx 1/1 Running 0 2m
tomcat-deploy-7b4c97ff9f-rj94r 1/1 Running 0 2m
[root@docker79 ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d
nginx-backend-svc ClusterIP 10.110.150.187 <none> 80/TCP 24m
tomcat-backend-svc ClusterIP 10.105.189.191 <none> 8080/TCP,8009/TCP 2m
[root@docker79 ingress]#
4) 部署 ingress 轉發規則集合
首先部署nginx-backend-svc的ingress 轉發規則集合,過程如下所示:
[root@docker79 ingress]# vim ingress-rules-ngx.yaml
[root@docker79 ingress]# cat ingress-rules-ngx.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rule-ngx
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: nginx.inspiry.com
http:
paths:
- path:
backend:
serviceName: nginx-backend-svc
servicePort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f ingress-rules.yaml
ingress.extensions/ingress-rule-ngx created
[root@docker79 ingress]# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-rule-ngx nginx.inspiry.com 80 9s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress ingress-rule-ngx
Name: ingress-rule-ngx
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
nginx.inspiry.com
nginx-backend-svc:80 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-ngx","namespace":"default"},"spec":{"rules":[{"host":"nginx.inspiry.com","http":{"paths":[{"backend":{"serviceName":"nginx-backend-svc","servicePort":80},"path":null}]}}]}}
kubernetes.io/ingress.class: nginx
Events: <none>
[root@docker79 ingress]#
然後再部署tomcat-backend-svc的ingress 轉發規則集合,過程如下所示:
[root@docker79 ingress]# vim ingress-rules-tomcat.yaml
[root@docker79 ingress]# cat ingress-rules-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rule-tomcat
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: tomcat.inspiry.cn
http:
paths:
- path:
backend:
serviceName: tomcat-backend-svc
servicePort: 8080
[root@docker79 ingress]# kubectl apply -f ingress-rules-tomcat.yaml
ingress.extensions/ingress-rule-tomcat created
[root@docker79 ingress]# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-rule-ngx nginx.inspiry.com 80 11m
ingress-rule-tomcat tomcat.inspiry.cn 80 6s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress ingress-rule-tomcat
Name: ingress-rule-tomcat
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
tomcat.inspiry.cn
tomcat-backend-svc:8080 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-tomcat","namespace":"default"},"spec":{"rules":[{"host":"tomcat.inspiry.cn","http":{"paths":[{"backend":{"serviceName":"tomcat-backend-svc","servicePort":8080},"path":null}]}}]}}
kubernetes.io/ingress.class: nginx
Events: <none>
[root@docker79 ingress]#
5) 測試
在kubernetes cluster 外的任何一臺主機上使用 curl http://nginx.inspiry.com:30080/ 或 curl http://tomcat.inspiry.cn:30080/ 即可訪問相應nginx-backend 、tomcat-backend 的pods。
生產環境中還可以在k8s cluster外再創建一層 LB,將接收到的 nginx.inspiry.com、tomcat.inspiry.cn 域名分別轉發到192.168.20.79、192.168.20.78、192.168.20.77為後端的upstream上。
觀察ingress controller的內容如下:
[root@docker79 ~]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
default-http-backend-6586bc58b6-8rxtp 1/1 Running 0 1h
nginx-ingress-controller-6bd7c597cb-xdrcm 1/1 Running 0 1h
[root@docker79 ~]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-xdrcm -- /bin/sh
$ ls
fastcgi.conf geoip mime.types nginx.conf scgi_params uwsgi_params.default
fastcgi.conf.default koi-utf mime.types.default nginx.conf.default scgi_params.default win-utf
fastcgi_params koi-win modsecurity opentracing.json template
fastcgi_params.default lua modules owasp-modsecurity-crs uwsgi_params
$ cat nginx.conf
...... (可仔細觀察該文件的內容,發現相應的轉發規則是隨著 ingress rules的變化而變化的)
Kebernetes學習總結(5) Ingress