本文記錄使用Velero Restic快速完成雲原生應用遷移至ACK叢集的實踐過程。

0. 實踐步驟概覽

（1）建立GKE叢集（或自建Kubernetes叢集）
（2）在GKE叢集上部署示例應用Jenkins Application並執行一個構建任務
（3）建立ACK叢集
（4）在ACK叢集中部署Minio Service用於應用遷移時資料中轉服務
（5）在GKE上部署Velero並備份整個Jenkins Application
（6）把Jenkins Application使用的容器映象同步到阿里雲容器映象倉庫
（7）在ACK上建立Jenkins Application所使用的StorageClass
（8）在ACK上部署Velero並恢復整個Jenkins Application
（9）在ACK上替換Jenkins Application所使用的容器映象
（10）驗證Jenkins Application能否正常提供服務及其構建任務是否存在

1. 環境物料清單

GKE叢集（或自建Kubernetes叢集）
ACK叢集

2. 建立GKE叢集並部署示例應用Jenkins Application

(1)在MarketPlace中找到Jenkins應用並配置部署到GKE叢集的jenkins名稱空間下：

(2)應用正在完成部署：

(3)檢視Ingress並訪問jenkins服務：

(4)Jenkins應用的初始化配置：

(5)Jenkin應用上建立一個名為gke-to-ack的任務並執行構建：

（6）Jenkins應用的k8s資源列表如下

$ kubectl -n jenkins get deploy
NAME                         DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
jenkins-jenkins-deployment   1         1         1            1           177m
$ kubectl -n jenkins get svc
NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
jenkins-jenkins-agents-connector   ClusterIP   10.39.241.75    <none>        50000/TCP        177m
jenkins-jenkins-ui                 NodePort    10.39.253.141   <none>        8080:31759/TCP   177m
$ kubectl -n jenkins get ing
NAME                 HOSTS   ADDRESS        PORTS     AGE
jenkins-jenkins-ui   *       xx.xx.xx.xx   80, 443   177m
$ kubectl -n jenkins get pvc
NAME                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
jenkins-jenkins-pvc   Bound    pvc-e64eacc4-a482-11e9-bfa9-42010a8000da   8Gi        RWO            standard       177m
$ kubectl -n jenkins get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                       STORAGECLASS   REASON   AGE
pvc-e64eacc4-a482-11e9-bfa9-42010a8000da   8Gi        RWO            Delete           Bound    jenkins/jenkins-jenkins-pvc                                 standard                177m
 

3. 建立ACK叢集並部署Minio物件儲存應用

(1) 自定義替換minio-deploy.yaml中的 MINIO_ACCESS_KEY 和 MINIO_SECRET_KEY值並部署：

kubectl apply -f minio-deploy.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: minio

---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  namespace: minio
  name: minio
  labels:
    component: minio
spec:
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        component: minio
    spec:
      volumes:
      - name: storage
        emptyDir: {}
      - name: config
        emptyDir: {}
      containers:
      - name: minio
        image: minio/minio:latest
        imagePullPolicy: IfNotPresent
        args:
        - server
        - /storage
        - --config-dir=/config
        env:
        - name: MINIO_ACCESS_KEY
          value: "<your MINIO_ACCESS_KEY>"
        - name: MINIO_SECRET_KEY
          value: "your MINIO_SECRET_KEY"
        ports:
        - containerPort: 9000
        volumeMounts:
        - name: storage
          mountPath: "/storage"
        - name: config
          mountPath: "/config"

---
apiVersion: v1
kind: Service
metadata:
  namespace: minio
  name: minio
  labels:
    component: minio
spec:
  # ClusterIP is recommended for production environments.
  # Change to NodePort if needed per documentation,
  # but only if you run Minio in a test/trial environment, for example with Minikube.
  type: LoadBalancer
  ports:
    - port: 9000
      targetPort: 9000
      protocol: TCP
  selector:
    component: minio
 

(2) 檢視minio LoadBalancer資訊並登陸應用：

$ kubectl -n minio get svc
NAME    TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)          AGE
minio   LoadBalancer   172.21.2.216   xxx.xxx.xxx.xx   9000:30912/TCP   2d4h

(3) 替換minio-job.yaml中的minio_server_url minino_access_key minio_secret_key並執行job在minio中建立名為velero的bucket：

kubectl apply -f minio-job.yaml

apiVersion: batch/v1
kind: Job
metadata:
  namespace: minio1
  name: minio1-setup
  labels:
    component: minio1
spec:
  template:
    metadata:
      name: minio1-setup
    spec:
      restartPolicy: OnFailure
      volumes:
      - name: config
        emptyDir: {}
      containers:
      - name: mc
        image: minio/mc:latest
        imagePullPolicy: IfNotPresent
        command:
        - /bin/sh
        - -c
        - "mc --config-dir=/config config host add velero <your minio_server_url> <your minino_access_key> <your minio_secret_key> && mc --config-dir=/config mb -p velero/velero"
        volumeMounts:
        - name: config
          mountPath: "/config"

(4) 檢視bucket是否建立成功：

4. 在GKE和ACK上部署velero

(1) 安裝velero client：
請從official release下載最新版本的velero客戶端：

(2) 安裝velero server：
建立credentials-velero檔案並設定aws_access_key_id 與 aws_secret_access_key的值

[default]
aws_access_key_id = <your minio_access_key>
aws_secret_access_key = <your minio_secret_key>

ACK叢集中請替換minio_server_url的值並指定image引數部署velero server

velero install     --provider aws    --image registry.cn-hangzhou.aliyuncs.com/acs/velero:latest  --bucket velero     --secret-file ./credentials-velero     --use-volume-snapshots=false     --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=<your minio_server_url> --use-restic     --wait

GKE中請替換minio_server_url的值並部署velero server

velero install     --provider aws    --bucket velero     --secret-file ./credentials-velero     --use-volume-snapshots=false     --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=<your minio_server_url> --use-restic     --wait

5. 在GKE叢集中備份Jenkins Application

(1)在備份帶volume資訊的pod之前，我們要通過給pod加annotation來告訴velero哪些pod需要包含volume資料， 檢視jenkins應用的Deployment資源：

$ kubectl -n jenkins get deploy jenkins-jenkins-deployment -oyaml

(2)為pod加annotation:

$ kubectl -n jenkins get po
NAME                                          READY   STATUS      RESTARTS   AGE
jenkins-deployer-zh5p4                        0/1     Completed   0          109m
jenkins-jenkins-deployment-7df86c64d4-tqqlr   1/1     Running     0          109m
$ kubectl -n jenkins annotate pod/jenkins-jenkins-deployment-7df86c64d4-tqqlr backup.velero.io/backup-volumes=jenkins-jenkins-pvc
pod/jenkins-jenkins-deployment-7df86c64d4-tqqlr annotated

(3)建立備份

$ velero backup create gcloud-jenkins-backup-restic --include-namespaces jenkins --wait
Backup request "gcloud-jenkins-backup-restic" submitted successfully.
Waiting for backup to complete. You may safely press ctrl-c to stop waiting - your backup will continue in the background.
.......................
Backup completed with status: Completed. You may check for more information using the commands `velero backup describe gcloud-jenkins-backup-restic` and `velero backup logs gcloud-jenkins-backup-restic`.

(4) 檢視備份

$ ./velero backup get
NAME                           STATUS      CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
gcloud-jenkins-backup-restic   Completed   2019-07-12 18:48:48 +0800 +08   29d       default            <none>

（5）登入Minio Server檢視

5. 同步（或批量）遷移容器映象

把Jenkins應用使用的容器映象匯入到阿里雲容器映象倉庫，匯入到的地址為：

registry.cn-hangzhou.aliyuncs.com/haoshuwei/jenkins:2.150.3

批量遷移可參考：https://github.com/AliyunContainerService/sync-repo.git

6. 在ACK叢集中建立Jenkins應用所使用的StorageClass standard

$ kubectl apply -f storageclass.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: standard
provisioner: alicloud/disk
parameters:
  type: cloud
reclaimPolicy: Delete

7. 在ACK叢集中恢復Jenkins Application

檢視備份資訊：

$ velero  backup get
NAME                           STATUS      CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
gcloud-jenkins-backup-restic   Completed   2019-07-12 18:48:48 +0800 CST   29d       default            <none>

恢復應用：

$ velero restore create --from-backup gcloud-jenkins-backup-restic

此處需要編輯deployment修改image為registry.cn-hangzhou.aliyuncs.com/haoshuwei/jenkins:2.150.3

檢視restore的狀態：

$ velero restore describe gcloud-jenkins-backup-restic-20190712190536
$ velero restore logs gcloud-jenkins-backup-restic-20190712190536

restore完成後的狀態為：

$ velero restore get
NAME                                          BACKUP                         STATUS      WARNINGS   ERRORS   CREATED                         SELECTOR
gcloud-jenkins-backup-restic-20190712190536   gcloud-jenkins-backup-restic   Completed   0          0        2019-07-12 19:05:36 +0800 CST   <none>

8. 檢視ACK叢集上jenkins應用的ingress並訪問服務進行驗證

$ kubectl -n jenkins get ing
NAME                 HOSTS   ADDRESS          PORTS     AGE
jenkins-jenkins-ui   *       xx.xx.xx.xx   80, 443   56m

至此，一個帶pv儲存的jenkins應用被完整地從GKE遷移到ACK上。

原文連結
本文為雲棲社群原創內容，未經