1. 程式人生 > 其它 >trarfik-個人雲主機擼一擼(01)

trarfik-個人雲主機擼一擼(01)

tracfik是什麼?

Traefik 是一個開源的Edge Router,它使釋出你的服務成為一種有趣和簡單的體驗。它代表你的系統接收請求,並找出負責處理這些請求的元件。
除了它的許多功能外,Traefik的與眾不同之處在於它能自動為你的服務發現正確的配置。當Traefik檢查你的基礎設施時,神奇的事情發生了,它發現了相關資訊,並發現哪個服務為哪個請求服務。
Traefik原生相容每一種主要的叢集技術,如Kubernetes、Docker、Docker Swarm、AWS、Mesos、Marathon等;並且可以同時處理許多叢集。(它甚至適用於在裸機上執行的傳統軟體)。
有了Traefik,就不需要維護和同步一個單獨的配置檔案:一切都會自動、實時地發生(沒有重新啟動,沒有連線中斷)。有了Traefik,你可以把時間花在開發和部署新功能上,而不是配置和維護其工作狀態上。

雲主機安裝

前置準備

  • 準備好k8s環境
  • 開放相對安全組埠

這裡使用的是單機版k8s,安全組策略已經開放了30000-40000埠

安裝

一共4個yaml檔案:

  • crd.yaml
  • rbac.yaml
  • deployment.yaml
  • dashboard.yaml

CRD物件

crd.yaml用於建立CRD物件

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressrouteudps.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteUDP
    plural: ingressrouteudps
    singular: ingressrouteudp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us
spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced

RBAC許可權

rbac.yaml用於給traefik授權k8s叢集許可權,這裡ServiceAccount使用者位於kube-system名稱空間下

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - ingressrouteudps
      - tlsoptions
      - tlsstores
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: kube-system

---
kind: ServiceAccount
apiVersion: v1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system

Deployment控制器

deployment.yaml中的args是traefik的啟動引數可按需修改,其中前兩項配置是來定義webwebsecure這兩個入口點的,--api=true開啟,就會建立一個名為api@internal的特殊 service,在 dashboard 中可以直接使用這個 service 來訪問,然後其他比較重要的就是開啟 kubernetesingress 和 這兩個 kubernetescrd provider。
這裡因為單機雲主機所以直接使用了主機網路,且埠也從80、443修改成了30080、30443

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: traefik
  namespace: kube-system
  labels:
    app: traefik
spec:
  selector:
    matchLabels:
      app: traefik
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      tolerations:
      - operator: "Exists"
      containers:
      - image: traefik:2.3
        name: traefik
        ports:
        - name: web
          containerPort: 30080
          hostPort: 30080
        - name: websecure
          containerPort: 30443
          hostPort: 30443
        - name: mysql
          containerPort: 33306
          hostPort: 33306
        args:
        - --entryPoints.web.address=:30080
        - --entryPoints.websecure.address=:30443
        - --entryPoints.mysql.address=:33306        
        - --log.level=INFO
        - --accesslog
        - --api=true
        - --api.dashboard=true
        - --ping=true
        - --providers.kubernetesingress
        - --providers.kubernetescrd
        resources:
          requests:
            cpu: "50m"
            memory: "50Mi"
          limits:
            cpu: "200m"
            memory: "100Mi"
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        readinessProbe:
          httpGet:
            path: /ping
            port: 8080
          failureThreshold: 1
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 2
        livenessProbe:
          httpGet:
            path: /ping
            port: 8080
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 2
      # 使用主機網路
      hostNetwork: true

Dashboard

dashboard.yaml為管理頁面,其中的dashboard.gitee.com修改對應自己域名,記得在/etc/hosts加上對應解析

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: kube-system
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`dashboard.gitee.com`)
    kind: Rule
    services:
    - name: api@internal
      kind: TraefikService

啟動順序

kubectl apply -f crd.yaml
kubectl apply -f rbac.yaml 
kubectl apply -f deployment.yaml
kubectl apply -f dashboard.yaml

// 顯示
root@i-t66xixhz:/opt/traefik# kubectl apply -f crd.yaml 
customresourcedefinition.apiextensions.k8s.io/ingressroutes.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/middlewares.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/ingressroutetcps.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/ingressrouteudps.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/tlsoptions.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/tlsstores.traefik.containo.us created
customresourcedefinition.apiextensions.k8s.io/traefikservices.traefik.containo.us created
root@i-t66xixhz:/opt/traefik# kubectl apply -f rbac.yaml 
clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created
clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created
serviceaccount/traefik-ingress-controller created
root@i-t66xixhz:/opt/traefik# kubectl apply -f deployment.yaml 
deployment.apps/traefik created
root@i-t66xixhz:/opt/traefik# kubectl apply -f dashboard.yaml 
ingressroute.traefik.containo.us/traefik-dashboard created

訪問

http://dashboard.gitee.com:30080/dashboard/

兩個版本web

準備了兩個Nginx web,通過configmap掛載了不同index內容

---
apiVersion: v1
kind: Namespace
metadata:
  name: web

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: web-config
  namespace: web
data:
  index-v1.html: |
    <h1>web v1</h1>
  index-v2.html: |
    <h1>web v2</h1>

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: web
spec:
  selector:
    matchLabels:
      app: nginx
      version: v1
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
          name: port-v1
        volumeMounts:
        - name: config
          mountPath: "/usr/share/nginx/html/index.html"
          subPath: index-v1.html
          readOnly: true
      volumes:
        - name: config
          configMap:
            name: web-config

---
apiVersion: v1
kind: Service
metadata:
  name: app-v1
  namespace: web
spec:
  selector:
    version: v1
  ports:
  - name: http
    port: 80
    targetPort: port-v1

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
  namespace: web
spec:
  selector:
    matchLabels:
      app: nginx
      version: v2
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
          name: port-v2
        volumeMounts:
        - name: config
          mountPath: "/usr/share/nginx/html/index.html"
          subPath: index-v2.html
          readOnly: true
      volumes:
        - name: config
          configMap:
            name: web-config

---
apiVersion: v1
kind: Service
metadata:
  name: app-v2
  namespace: web
spec:
  selector:
    version: v2
  ports:
  - name: http
    port: 80
    targetPort: port-v2

通過traefik暴露服務

建立一個 IngressRoute 資源物件:(web-ingressroute.yaml),配置的 ServiceKubernetes Service物件。

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: app-ingressroute
  namespace: web
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`web.gitee.com`)
    kind: Rule
    services:
    - name: app-v1
      namespace: web # 名稱空間
      port: 80

這個時候我們對域名web.gitee.com/etc/hosts解析
訪問http://web.gitee.com:30080/

HTTPS的暴露

基於上面的基礎建立證書對應的secret,已申請到的證書檔案:tls.crt和tls.key

kubectl create secret tls web-tls --cert=tls.crt --key=tls.key

IngressRoute配置tls,使用websecure

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: app-ingressroute
  namespace: web
spec:
  entryPoints:
    - websecure
  routes:
  - match: Host(`web.gitee.com`)
    kind: Rule
    services:
    - name: app-v1
      namespace: web # 名稱空間
      port: 80
  tls:
    secretName: web-tls

中介軟體

中介軟體是 Traefik2.0 中一個非常有特色的功能,我們可以根據自己的各種需求去選擇不同的中介軟體來滿足服務,Traefik 官方已經內建了許多不同功能的中介軟體,其中一些可以修改請求,頭資訊,一些負責重定向,一些新增身份驗證等等,而且中介軟體還可以通過鏈式組合(洋蔥模式)的方式來適用各種情況。

示例:http跳轉https(redirectScheme)

web-middleware.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: redirect-https
  namespace: web
spec:
  redirectScheme:
    scheme: https

然後將這個中介軟體附加到 http 服務的IngressRoute上面去,因為 https 的不需要跳轉

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: app-ingressroute
  namespace: web
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`web.gitee.com`)
    kind: Rule
    services:
    - name: app-v1
      namespace: web # 名稱空間
      port: 80
    middlewares: 
    - name: redirect-https

這個時候我們再去訪問 http 服務可以發現就會自動跳轉到 https 去了。更多中介軟體的用法可以看:http中介軟體tcp中介軟體

灰度釋出

Traefik2.0 的一個更強大的功能就是灰度釋出,灰度釋出我們有時候也會稱為金絲雀釋出(Canary),主要就是讓一部分測試的服務也參與到線上去,經過測試觀察看是否符號上線要求。
2.3版本新增了一個 TraefikService 的 CRD 資源,我們可以直接利用這個物件來配置 web app,之前的版本需要通過 File Provider,比較麻煩。
下面利用 Traefik2.0 中提供的帶權重的輪詢(WRR)功能來控制我們的流量,將3/4的流量路由到 app-v1,1/4 的流量路由到 app-v2 。新建一個描述app的資源清單:(web-traefikservice.yaml)

---
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: app
  namespace: web
spec:
  weighted:
    services:
      - name: app-v1
        weight: 3  # 定義權重
        port: 80
        kind: Service  # 可選,預設就是 Service
      - name: app-v2
        weight: 1
        port: 80

接著建立一個 IngressRoute 資源物件:(web-ingressroute.yaml),不過需要注意的是現在我們配置的 Service 不再是直接的 Kubernetes 物件了,而是上面我們定義的 TraefikService 物件,直接建立上面的資源物件

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: app-ingressroute
  namespace: web
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`web.gitee.com`)
    kind: Rule
    services:
    - name: app
      namespace: web # 名稱空間
      kind: TraefikService

訪問http://web.gitee.com:30080/,去瀏覽器中連續訪問 4 次,我們可以觀察到 app-v1 這應用會收到 3 次請求,而 app-v2 這個應用只收到 1 次請求,符合上面我們的權重配置。

流量複製

Traefik 2.0 還引入了流量映象服務,是一種可以將流入流量複製並同時將其傳送給其他服務的方法,映象服務可以獲得給定百分比的請求同時也會忽略這部分請求的響應。
在 2.3 版本中我們已經可以通過TraefikService資源物件中的mirroring來進行配置,下面將服務 v1 的流量複製 50% 到服務 v2

---
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: app
  namespace: web
spec:
  mirroring:
    name: app-v1 # 傳送 100% 的請求到 K8S 的 Service "app-v1"
    port: 80
    mirrors:
    - name: app-v2 # 然後複製 50% 的請求到 app-v2 
      percent: 50
      port: 80


---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: app-ingressroute
  namespace: web
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`web.gitee.com`)
    kind: Rule
    services:
    - name: app
      namespace: web # 名稱空間
      kind: TraefikService

訪問2遍瀏覽器,通過檢視日誌如下:

v1: kubectl logs -f nginx-v1-64c95b8dd6-5sltw -n web 
v2: kubectl logs -f nginx-v2-57588c6859-qwg5s -n web

TCP服務

首先建立一個普通的mysql服務(mysql-tcp.yaml)

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-config
data:
  my.cnf: |
    [mysqld]
    pid-file        = /var/run/mysqld/mysqld.pid
    socket          = /var/run/mysqld/mysqld.sock
    datadir         = /var/lib/mysql
    symbolic-links=0


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate # 在建立新 Pods 之前,所有現有的 Pods 會被殺死
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mysql:5.7
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: a123456
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysqlcnf
          mountPath: /etc/mysql/my.cnf
          subPath: my.cnf
      volumes:
      - name: mysqlcnf
        configMap:
          name: mysql-config

---
apiVersion: v1
kind: Service
metadata:
  name: mysql-traefik
spec:
  selector:
    app: mysql
  ports:
  - port: 3306

建立成功後就可以來為 mysql 服務配置一個路由了。我們這裡建立一個IngressRouteTCP 型別的 CRD 物件(前面我們就已經安裝了對應的 CRD 資源),因為沒有配置證書,所以HostSNI使用萬用字元 ***** 進行配置(mysql-ingressroute-tcp.yaml)

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: mysql-traefik-tcp
spec:
  entryPoints:
    - mysql
  routes:
  - match: HostSNI(`*`)
    services:
    - name: mysql-traefik
      port: 3306

要注意上面的** entryPoints **部分,是根據我們啟動的 Traefik 的靜態配置中的 entryPoints 來決定的,我們當然可以使用前面我們定義得 30080(web) 和 30443(websecure) 這兩個入口點,但是也可以可以自己新增一個用於 mysql 服務的專門入口點。更多EntryPoints知識

- --entryPoints.web.address=:30080
- --entryPoints.websecure.address=:30443
- --entryPoints.mysql.address=:33306  

建立成功

# kubectl get IngressRouteTCP 
NAME                AGE
mysql-traefik-tcp   9m22s

嘗試連線

# mysql -h 1xx.1xx.1xx.172 -P 33306 -u root -pa123456
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.36 MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.02 sec)

安全的TCP(TLS)

建立TLS的TCP需要根據證書檔案cert.pem和key.key,建立secret

kubectl create secret tls traefik-mysql-certs --cert=cert.pem --key=key.key

然後在 IngressRouteTCP 物件,增加 TLS 配置:

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: mysql-traefik-tcp
spec:
  entryPoints:
    - mysql
  routes:
  - match: HostSNI(`mysql.traefik.com`)
    services:
    - name: mysql-traefik
      port: 3306
  tls: 
    secretName: traefik-mysql-certs