019.Kubernetes二進位制叢集儲存longhorn

阿新 • • 發佈：2020-06-29

一 Longhorn儲存部署

1.1 Longhorn概述

Longhorn是用於Kubernetes的開源分散式塊儲存系統。

提示：更多介紹參考：https://github.com/longhorn/longhorn。

1.2 Longhorn部署

  1 [[email protected] ~]# cd /opt/k8s/work/

  2 [[email protected] work]# source /root/environment.sh

  3 [[email protected] work]# for all_ip in ${ALL_IPS[@]}

  4   do

  5     echo ">>> ${all_ip}"
 

  6     ssh [email protected]${all_ip} "yum -y install iscsi-initiator-utils &"

  7   done

提示：本步驟操作僅需要在master01節點操作。

  1 [[email protected] work]# mkdir longhorn

  2 [[email protected] work]# cd longhorn/

  3 [[email protected] longhorn]# wget \

  4 https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
 

  5 [[email protected] longhorn]# vi longhorn.yaml

  1 #……

  2 ---

  3 kind: Service

  4 apiVersion: v1

  5 metadata:

  6   labels:

  7     app: longhorn-ui

  8   name: longhorn-frontend

  9   namespace: longhorn-system

 10 spec:

 11   type: NodePort			#修改為nodeport

 12   selector:

 13     app: longhorn-ui
 

 14   ports:

 15   - port: 80

 16     targetPort: 8000

 17     nodePort: 30002

 18 ---

 19 ……

 20 kind: DaemonSet

 21 ……

 22         imagePullPolicy: IfNotPresent

 23 ……

 24 #……

  1 [[email protected] longhorn]# kubectl apply -f longhorn.yaml

  2 [[email protected] longhorn]# kubectl -n longhorn-system get pods -o wide

提示：若部署異常可刪除重建，若出現無法刪除namespace，可通過如下操作進行刪除：

  1 wget https://github.com/longhorn/longhorn/blob/master/uninstall/uninstall.yaml

  2 rm -rf /var/lib/longhorn/

  3 kubectl apply -f uninstall.yaml

  4 kubectl delete -f longhorn.yaml

1.5 動態sc建立

提示：預設longhorn部署完成已建立一個sc，也可通過如下手動編寫yaml建立。

  1 [[email protected] longhorn]# kubectl get sc

  2 NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE

  3 ……

  4 longhorn               driver.longhorn.io      Delete          Immediate              true                   15m

  5 [[email protected] longhorn]# vi longhornsc.yaml

  1 kind: StorageClass

  2 apiVersion: storage.k8s.io/v1

  3 metadata:

  4   name: longhornsc

  5 provisioner: rancher.io/longhorn

  6 parameters:

  7   numberOfReplicas: "3"

  8   staleReplicaTimeout: "30"

  9   fromBackup: ""

  1 [[email protected] longhorn]# kubectl create -f longhornsc.yaml

1.6 測試PV及PVC

  1 [[email protected] longhorn]# vi longhornpod.yaml

  1 apiVersion: v1

  2 kind: PersistentVolumeClaim

  3 metadata:

  4   name: longhorn-pvc

  5 spec:

  6   accessModes:

  7     - ReadWriteOnce

  8   storageClassName: longhorn

  9   resources:

 10     requests:

 11       storage: 2Gi

 12 ---

 13 apiVersion: v1

 14 kind: Pod

 15 metadata:

 16   name: longhorn-pod

 17   namespace: default

 18 spec:

 19   containers:

 20   - name: volume-test

 21     image: nginx:stable-alpine

 22     imagePullPolicy: IfNotPresent

 23     volumeMounts:

 24     - name: volv

 25       mountPath: /data

 26     ports:

 27     - containerPort: 80

 28   volumes:

 29   - name: volv

 30     persistentVolumeClaim:

 31       claimName: longhorn-pvc

  1 [[email protected] longhorn]# kubectl apply -f longhornpod.yaml

  2 [[email protected] longhorn]# kubectl get pods -o wide

  3 [[email protected] longhorn]# kubectl get pvc

  4 [[email protected] longhorn]# kubectl get pv

提示：本步驟操作僅需要在master01節點操作。

1.7 Ingress暴露Longhorn

  1 [[email protected] longhorn]# yum -y install httpd-tools

  2 [[email protected] longhorn]# htpasswd -c auth xhy			#建立使用者名稱和密碼

提示：也可通過如下命令建立：

  1 [[email protected] longhorn]# kubectl -n longhorn-system create secret generic longhorn-basic-auth --from-file=auth

  2

  3 [[email protected] longhorn]# vi longhorn-ingress.yaml		#建立ingress規則

  1 apiVersion: networking.k8s.io/v1beta1

  2 kind: Ingress

  3 metadata:

  4   name: longhorn-ingress

  5   namespace: longhorn-system

  6   annotations:

  7     kubernetes.io/ingress.class: "nginx"

  8     nginx.ingress.kubernetes.io/auth-type: basic

  9     nginx.ingress.kubernetes.io/auth-secret: longhorn-basic-auth

 10     nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '

 11 spec:

 12   rules:

 13   - host: longhorn.odocker.com

 14     http:

 15       paths:

 16       - path: /

 17         backend:

 18           serviceName: longhorn-frontend

 19           servicePort: 80

  1 [[email protected] longhorn]# kubectl apply -f longhorn-ingress.yaml

提示：本步驟操作僅需要在master01節點操作。

1.8 確認驗證

瀏覽器訪問：longhorn.odocker.com，並輸入賬號和密碼。

登入檢視。

附加 Helm部署

附0.1 helm安裝

Helm 是 Kubernetes 的軟體包管理工具，此處作為建議項部署，更多helm參考《053.叢集管理-Helm工具》。

  1 [[email protected] ~]# cd /opt/k8s/work/

  2 [[email protected] work]# mkdir /opt/k8s/work/helm

  3 [[email protected] work]# source /root/environment.sh

  4 [[email protected] work]# wget http://down.linuxsb.com:8888/helm-v3.2.3-linux-amd64.tar.gz

  5 [[email protected] work]# tar -zxvf helm-v3.2.3-linux-amd64.tar.gz -C /opt/k8s/work/helm

  6 [[email protected] work]# for master_ip in ${MASTER_IPS[@]}

  7   do

  8     echo ">>> ${master_ip}"

  9     scp -rp /opt/k8s/work/helm/linux-amd64/helm [email protected]${master_ip}:/opt/k8s/bin/

 10     ssh [email protected]${master_ip} "chmod +x /opt/k8s/bin/*"

 11     ssh [email protected]${master_ip} "helm version"

 12     ssh [email protected]${master_ip} "echo 'source <(helm completion bash)' >> $HOME/.bashrc"

 13   done

附0.2 helm安裝

  1 [[email protected] work]# helm repo add brigade https://brigadecore.github.io/charts

  2 [[email protected] work]# helm repo add stable https://kubernetes-charts.storage.googleapis.com/				#新增官方repo

  3 [[email protected] work]# helm repo add bitnami https://charts.bitnami.com/bitnami

  4 [[email protected] work]# helm repo list			#檢視repo