trarfik-個人雲主機擼一擼(01)
tracfik是什麼?
Traefik 是一個開源的Edge Router,它使釋出你的服務成為一種有趣和簡單的體驗。它代表你的系統接收請求,並找出負責處理這些請求的元件。
除了它的許多功能外,Traefik的與眾不同之處在於它能自動為你的服務發現正確的配置。當Traefik檢查你的基礎設施時,神奇的事情發生了,它發現了相關資訊,並發現哪個服務為哪個請求服務。
Traefik原生相容每一種主要的叢集技術,如Kubernetes、Docker、Docker Swarm、AWS、Mesos、Marathon等;並且可以同時處理許多叢集。(它甚至適用於在裸機上執行的傳統軟體)。
有了Traefik,就不需要維護和同步一個單獨的配置檔案:一切都會自動、實時地發生(沒有重新啟動,沒有連線中斷)。有了Traefik,你可以把時間花在開發和部署新功能上,而不是配置和維護其工作狀態上。
雲主機安裝
前置準備
- 準備好k8s環境
- 開放相對安全組埠
這裡使用的是單機版k8s,安全組策略已經開放了30000-40000埠
安裝
一共4個yaml檔案:
- crd.yaml
- rbac.yaml
- deployment.yaml
- dashboard.yaml
CRD物件
crd.yaml用於建立CRD物件
--- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressroutes.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRoute plural: ingressroutes singular: ingressroute scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: middlewares.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: Middleware plural: middlewares singular: middleware scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressroutetcps.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRouteTCP plural: ingressroutetcps singular: ingressroutetcp scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressrouteudps.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRouteUDP plural: ingressrouteudps singular: ingressrouteudp scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: tlsoptions.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: TLSOption plural: tlsoptions singular: tlsoption scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: tlsstores.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: TLSStore plural: tlsstores singular: tlsstore scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: traefikservices.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: TraefikService plural: traefikservices singular: traefikservice scope: Namespaced
RBAC許可權
rbac.yaml用於給traefik授權k8s叢集許可權,這裡ServiceAccount使用者位於kube-system名稱空間下
--- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses - ingressclasses verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses/status verbs: - update - apiGroups: - traefik.containo.us resources: - middlewares - ingressroutes - traefikservices - ingressroutetcps - ingressrouteudps - tlsoptions - tlsstores verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: kube-system --- kind: ServiceAccount apiVersion: v1 metadata: name: traefik-ingress-controller namespace: kube-system
Deployment控制器
deployment.yaml中的args是traefik的啟動引數可按需修改,其中前兩項配置是來定義web和websecure這兩個入口點的,--api=true開啟,就會建立一個名為api@internal的特殊 service,在 dashboard 中可以直接使用這個 service 來訪問,然後其他比較重要的就是開啟 kubernetesingress 和 這兩個 kubernetescrd provider。
這裡因為單機雲主機所以直接使用了主機網路,且埠也從80、443修改成了30080、30443
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: traefik
namespace: kube-system
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
strategy:
type: Recreate
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
tolerations:
- operator: "Exists"
containers:
- image: traefik:2.3
name: traefik
ports:
- name: web
containerPort: 30080
hostPort: 30080
- name: websecure
containerPort: 30443
hostPort: 30443
- name: mysql
containerPort: 33306
hostPort: 33306
args:
- --entryPoints.web.address=:30080
- --entryPoints.websecure.address=:30443
- --entryPoints.mysql.address=:33306
- --log.level=INFO
- --accesslog
- --api=true
- --api.dashboard=true
- --ping=true
- --providers.kubernetesingress
- --providers.kubernetescrd
resources:
requests:
cpu: "50m"
memory: "50Mi"
limits:
cpu: "200m"
memory: "100Mi"
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
readinessProbe:
httpGet:
path: /ping
port: 8080
failureThreshold: 1
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
livenessProbe:
httpGet:
path: /ping
port: 8080
failureThreshold: 3
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
# 使用主機網路
hostNetwork: true
Dashboard
dashboard.yaml為管理頁面,其中的dashboard.gitee.com修改對應自己域名,記得在/etc/hosts加上對應解析
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-dashboard
namespace: kube-system
spec:
entryPoints:
- web
routes:
- match: Host(`dashboard.gitee.com`)
kind: Rule
services:
- name: api@internal
kind: TraefikService
啟動順序
kubectl apply -f crd.yaml
kubectl apply -f rbac.yaml
kubectl apply -f deployment.yaml
kubectl apply -f dashboard.yaml
// 顯示
root@i-t66xixhz:/opt/traefik# kubectl apply -f crd.yaml
customresourcedefinition.apiextensions.k8s.io/ingressroutes.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/middlewares.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/ingressroutetcps.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/ingressrouteudps.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/tlsoptions.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/tlsstores.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/traefikservices.traefik.containo.us created
root@i-t66xixhz:/opt/traefik# kubectl apply -f rbac.yaml
clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created
clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created
serviceaccount/traefik-ingress-controller created
root@i-t66xixhz:/opt/traefik# kubectl apply -f deployment.yaml
deployment.apps/traefik created
root@i-t66xixhz:/opt/traefik# kubectl apply -f dashboard.yaml
ingressroute.traefik.containo.us/traefik-dashboard created
訪問
http://dashboard.gitee.com:30080/dashboard/
兩個版本web
準備了兩個Nginx web,通過configmap掛載了不同index內容
---
apiVersion: v1
kind: Namespace
metadata:
name: web
---
apiVersion: v1
kind: ConfigMap
metadata:
name: web-config
namespace: web
data:
index-v1.html: |
<h1>web v1</h1>
index-v2.html: |
<h1>web v2</h1>
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-v1
namespace: web
spec:
selector:
matchLabels:
app: nginx
version: v1
replicas: 1
template:
metadata:
labels:
app: nginx
version: v1
spec:
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
name: port-v1
volumeMounts:
- name: config
mountPath: "/usr/share/nginx/html/index.html"
subPath: index-v1.html
readOnly: true
volumes:
- name: config
configMap:
name: web-config
---
apiVersion: v1
kind: Service
metadata:
name: app-v1
namespace: web
spec:
selector:
version: v1
ports:
- name: http
port: 80
targetPort: port-v1
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-v2
namespace: web
spec:
selector:
matchLabels:
app: nginx
version: v2
replicas: 1
template:
metadata:
labels:
app: nginx
version: v2
spec:
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
name: port-v2
volumeMounts:
- name: config
mountPath: "/usr/share/nginx/html/index.html"
subPath: index-v2.html
readOnly: true
volumes:
- name: config
configMap:
name: web-config
---
apiVersion: v1
kind: Service
metadata:
name: app-v2
namespace: web
spec:
selector:
version: v2
ports:
- name: http
port: 80
targetPort: port-v2
通過traefik暴露服務
建立一個 IngressRoute 資源物件:(web-ingressroute.yaml),配置的 Service 是Kubernetes Service物件。
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: app-ingressroute
namespace: web
spec:
entryPoints:
- web
routes:
- match: Host(`web.gitee.com`)
kind: Rule
services:
- name: app-v1
namespace: web # 名稱空間
port: 80
這個時候我們對域名web.gitee.com做/etc/hosts解析
訪問http://web.gitee.com:30080/
HTTPS的暴露
基於上面的基礎建立證書對應的secret,已申請到的證書檔案:tls.crt和tls.key
kubectl create secret tls web-tls --cert=tls.crt --key=tls.key
IngressRoute配置tls,使用websecure
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: app-ingressroute
namespace: web
spec:
entryPoints:
- websecure
routes:
- match: Host(`web.gitee.com`)
kind: Rule
services:
- name: app-v1
namespace: web # 名稱空間
port: 80
tls:
secretName: web-tls
中介軟體
中介軟體是 Traefik2.0 中一個非常有特色的功能,我們可以根據自己的各種需求去選擇不同的中介軟體來滿足服務,Traefik 官方已經內建了許多不同功能的中介軟體,其中一些可以修改請求,頭資訊,一些負責重定向,一些新增身份驗證等等,而且中介軟體還可以通過鏈式組合(洋蔥模式)的方式來適用各種情況。
示例:http跳轉https(redirectScheme)
web-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: redirect-https
namespace: web
spec:
redirectScheme:
scheme: https
然後將這個中介軟體附加到 http 服務的IngressRoute上面去,因為 https 的不需要跳轉
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: app-ingressroute
namespace: web
spec:
entryPoints:
- web
routes:
- match: Host(`web.gitee.com`)
kind: Rule
services:
- name: app-v1
namespace: web # 名稱空間
port: 80
middlewares:
- name: redirect-https
這個時候我們再去訪問 http 服務可以發現就會自動跳轉到 https 去了。更多中介軟體的用法可以看:http中介軟體和tcp中介軟體
灰度釋出
Traefik2.0 的一個更強大的功能就是灰度釋出,灰度釋出我們有時候也會稱為金絲雀釋出(Canary),主要就是讓一部分測試的服務也參與到線上去,經過測試觀察看是否符號上線要求。
2.3版本新增了一個 TraefikService 的 CRD 資源,我們可以直接利用這個物件來配置 web app,之前的版本需要通過 File Provider,比較麻煩。
下面利用 Traefik2.0 中提供的帶權重的輪詢(WRR)功能來控制我們的流量,將3/4的流量路由到 app-v1,1/4 的流量路由到 app-v2 。新建一個描述app的資源清單:(web-traefikservice.yaml)
---
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: app
namespace: web
spec:
weighted:
services:
- name: app-v1
weight: 3 # 定義權重
port: 80
kind: Service # 可選,預設就是 Service
- name: app-v2
weight: 1
port: 80
接著建立一個 IngressRoute 資源物件:(web-ingressroute.yaml),不過需要注意的是現在我們配置的 Service 不再是直接的 Kubernetes 物件了,而是上面我們定義的 TraefikService 物件,直接建立上面的資源物件
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: app-ingressroute
namespace: web
spec:
entryPoints:
- web
routes:
- match: Host(`web.gitee.com`)
kind: Rule
services:
- name: app
namespace: web # 名稱空間
kind: TraefikService
訪問http://web.gitee.com:30080/,去瀏覽器中連續訪問 4 次,我們可以觀察到 app-v1 這應用會收到 3 次請求,而 app-v2 這個應用只收到 1 次請求,符合上面我們的權重配置。
流量複製
Traefik 2.0 還引入了流量映象服務,是一種可以將流入流量複製並同時將其傳送給其他服務的方法,映象服務可以獲得給定百分比的請求同時也會忽略這部分請求的響應。
在 2.3 版本中我們已經可以通過TraefikService資源物件中的mirroring來進行配置,下面將服務 v1 的流量複製 50% 到服務 v2
---
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: app
namespace: web
spec:
mirroring:
name: app-v1 # 傳送 100% 的請求到 K8S 的 Service "app-v1"
port: 80
mirrors:
- name: app-v2 # 然後複製 50% 的請求到 app-v2
percent: 50
port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: app-ingressroute
namespace: web
spec:
entryPoints:
- web
routes:
- match: Host(`web.gitee.com`)
kind: Rule
services:
- name: app
namespace: web # 名稱空間
kind: TraefikService
訪問2遍瀏覽器,通過檢視日誌如下:
v1: kubectl logs -f nginx-v1-64c95b8dd6-5sltw -n web
v2: kubectl logs -f nginx-v2-57588c6859-qwg5s -n web
TCP服務
首先建立一個普通的mysql服務(mysql-tcp.yaml)
---
apiVersion: v1
kind: ConfigMap
metadata:
name: mysql-config
data:
my.cnf: |
[mysqld]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
symbolic-links=0
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
spec:
selector:
matchLabels:
app: mysql
strategy:
type: Recreate # 在建立新 Pods 之前,所有現有的 Pods 會被殺死
template:
metadata:
labels:
app: mysql
spec:
containers:
- image: mysql:5.7
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: a123456
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysqlcnf
mountPath: /etc/mysql/my.cnf
subPath: my.cnf
volumes:
- name: mysqlcnf
configMap:
name: mysql-config
---
apiVersion: v1
kind: Service
metadata:
name: mysql-traefik
spec:
selector:
app: mysql
ports:
- port: 3306
建立成功後就可以來為 mysql 服務配置一個路由了。我們這裡建立一個IngressRouteTCP 型別的 CRD 物件(前面我們就已經安裝了對應的 CRD 資源),因為沒有配置證書,所以HostSNI使用萬用字元 ***** 進行配置(mysql-ingressroute-tcp.yaml)
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: mysql-traefik-tcp
spec:
entryPoints:
- mysql
routes:
- match: HostSNI(`*`)
services:
- name: mysql-traefik
port: 3306
要注意上面的** entryPoints **部分,是根據我們啟動的 Traefik 的靜態配置中的 entryPoints 來決定的,我們當然可以使用前面我們定義得 30080(web) 和 30443(websecure) 這兩個入口點,但是也可以可以自己新增一個用於 mysql 服務的專門入口點。更多EntryPoints知識
- --entryPoints.web.address=:30080
- --entryPoints.websecure.address=:30443
- --entryPoints.mysql.address=:33306
建立成功
# kubectl get IngressRouteTCP
NAME AGE
mysql-traefik-tcp 9m22s
嘗試連線
# mysql -h 1xx.1xx.1xx.172 -P 33306 -u root -pa123456
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.36 MySQL Community Server (GPL)
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.02 sec)
安全的TCP(TLS)
建立TLS的TCP需要根據證書檔案cert.pem和key.key,建立secret
kubectl create secret tls traefik-mysql-certs --cert=cert.pem --key=key.key
然後在 IngressRouteTCP 物件,增加 TLS 配置:
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: mysql-traefik-tcp
spec:
entryPoints:
- mysql
routes:
- match: HostSNI(`mysql.traefik.com`)
services:
- name: mysql-traefik
port: 3306
tls:
secretName: traefik-mysql-certs