29, k8s 架構之master-slave
阿新 • • 發佈：2020-08-01
1,對專案的理解 單體架構？微服務？ 怎麼部署？ 啟動是否有依賴？ 部署到k8s平臺怎麼個流程？ 1. 製作映象 2. 容器放到Pod 3. 控制器管理Pod 4. 暴露應用 5. 對外發布應用 6. 日誌管理/監控 不同環境區分配置檔案 configmap entrypoint.sh 統一配置中心，例如 Apollo，Disconf 2,安裝docker-ce yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum-config-manager --enable docker-ce-edge yum install docker-ce Docker配置加速： curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io systemctl start docker 安裝docker-compose: curl -L https://get.daocloud.io/docker/compose/releases/download/1.12.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose docker-compose version # 檢視版本號，測試是否安裝成功 3,Harbor映象倉庫 1、下載離線安裝包 https://github.com/goharbor/harbor/releases 2、解壓並配置訪問地址 tar zxvf harbor-offline-installer-v1.6.1.tgz cd harbor vi harbor.cfg hostname = 192.168.0.12 harbor_admin_password = 123456 3、準備配置 ./prepare 4、匯入映象並啟動 ./install.sh 5、檢視容器狀態 docker-compose ps 4,Git程式碼版本倉庫 1、安裝Git yum install git 2、建立Git使用者並設定密碼 useradd git passwd git 3、建立倉庫 su - git mkdir app.git cd app.git git --bare init 4、配置客戶端與Git伺服器SSH免互動認證 5、測試 git clone [email protected]:/home/git/app.git git add . git commit –m “test” git push origin master 5， 注意： 拉取官方提供的jenkins docker容器時，預設會將jenkins的資料檔案儲存在這個目錄，/var/Jenkins_home, 所以要對這個目錄做持久化。 yum install -y nfs-utils vim /etc/exports /ifs/kubernetes *(rw,no_root_squash) systemctl start nfs 測試： mount -t nfs 192.168.0.12:/ifs/kubernetes /mnt/ 取消掛載： umount /mnt/ 安裝nfs-client，實現自動PV供給。 先修改deployment.yaml 的NFS的IP地址和NFS共享路徑 [root@centos7 ~]# cd nfs-client/ [root@centos7 nfs-client]# [root@centos7 nfs-client]# [root@centos7 nfs-client]# [root@centos7 nfs-client]# ll total 12 -rw-r--r-- 1 root root 225 Mar 31 2019 class.yaml -rw-r--r-- 1 root root 977 Aug 1 19:19 deployment.yaml -rw-r--r-- 1 root root 1526 Mar 31 2019 rbac.yaml [root@centos7 nfs-client]# [root@centos7 nfs-client]# [root@centos7 nfs-client]# cat class.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: managed-nfs-storage provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME' parameters: archiveOnDelete: "true" [root@centos7 nfs-client]# [root@centos7 nfs-client]# cat rbac.yaml kind: ServiceAccount apiVersion: v1 metadata: name: nfs-client-provisioner --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: nfs-client-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["create", "update", "patch"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner namespace: default roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner rules: - apiGroups: [""] resources: ["endpoints"] verbs: ["get", "list", "watch", "create", "update", "patch"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default roleRef: kind: Role name: leader-locking-nfs-client-provisioner apiGroup: rbac.authorization.k8s.io [root@centos7 nfs-client]# [root@centos7 nfs-client]# [root@centos7 nfs-client]# [root@centos7 nfs-client]# cat deployment.yaml apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner --- kind: Deployment apiVersion: apps/v1 metadata: name: nfs-client-provisioner spec: selector: matchLabels: app: nfs-client-provisioner replicas: 1 strategy: type: Recreate template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image: lizhenliang/nfs-client-provisioner:latest volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER value: 192.168.0.12 - name: NFS_PATH value: /ifs/kubernetes volumes: - name: nfs-client-root nfs: server: 192.168.0.12 path: /ifs/kubernetes [root@centos7 nfs-client]# 6，安裝Jenkins 參考官網：https://github.com/jenkinsci/kubernetes-plugin/blob/fc40c869edfd9e3904a9a56b0f80c5a25e988fa1/src/main/kubernetes/jenkins.yml [root@centos7 jenkins]# cat rbac.yml --- # 建立名為jenkins的ServiceAccount apiVersion: v1 kind: ServiceAccount metadata: name: jenkins --- # 建立名為jenkins的Role，授予允許管理API組的資源Pod kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: jenkins rules: - apiGroups: [""] resources: ["pods"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/exec"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/log"] verbs: ["get","list","watch"] - apiGroups: [""] resources: ["secrets"] verbs: ["get"] --- # 將名為jenkins的Role繫結到名為jenkins的ServiceAccount apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: jenkins roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: jenkins subjects: - kind: ServiceAccount name: jenkins [root@centos7 jenkins]# [root@centos7 jenkins]# [root@centos7 jenkins]# [root@centos7 jenkins]# cat service-account.yml # In GKE need to get RBAC permissions first with # kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin [--user=<user-name>|--group=<group-name>] --- apiVersion: v1 kind: ServiceAccount metadata: name: jenkins --- kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: jenkins rules: - apiGroups: [""] resources: ["pods"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/exec"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/log"] verbs: ["get","list","watch"] - apiGroups: [""] resources: ["secrets"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: jenkins roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: jenkins subjects: - kind: ServiceAccount name: jenkins [root@centos7 jenkins]# [root@centos7 jenkins]# cat service.yml apiVersion: v1 kind: Service metadata: name: jenkins spec: selector: app: jenkins type: NodePort ports: - name: http port: 80 targetPort: 8080 protocol: TCP nodePort: 30006 - name: agent port: 50000 protocol: TCP [root@centos7 jenkins]# [root@centos7 jenkins]# cat statefulset.yml apiVersion: apps/v1 kind: StatefulSet metadata: name: jenkins labels: name: jenkins spec: selector: matchLabels: app: jenkins serviceName: jenkins replicas: 1 updateStrategy: type: RollingUpdate template: metadata: labels: app: jenkins spec: terminationGracePeriodSeconds: 10 serviceAccountName: jenkins containers: - name: jenkins image: jenkins/jenkins:lts-alpine imagePullPolicy: Always ports: - containerPort: 8080 - containerPort: 50000 resources: limits: cpu: 1 memory: 1Gi requests: cpu: 0.5 memory: 500Mi env: - name: LIMITS_MEMORY valueFrom: resourceFieldRef: resource: limits.memory divisor: 1Mi - name: JAVA_OPTS value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home volumeClaimTemplates: - metadata: name: jenkins-home spec: storageClassName: "managed-nfs-storage" accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi [root@centos7 jenkins]# 7，Jenkins Master/Slave架構 安裝外掛： git kubernetes Pipeline 構建Jenkins Slave映象: [root@centos7 jenkins-slave]# ll total 776 -rw-r--r--. 1 root root 407 May 20 2019 Dockerfile -rwxr-xr-x. 1 root root 1980 Apr 6 2018 jenkins-slave -rw-r--r--. 1 root root 10409 Dec 29 2018 settings.xml -rw-r--r--. 1 root root 770802 Jun 11 2018 slave.jar [root@centos7 jenkins-slave]# [root@centos7 jenkins-slave]# cat Dockerfile FROM centos:7 LABEL maintainer lizhenliang RUN yum install -y java-1.8.0-openjdk maven curl git libtool-ltdl-devel && \ yum clean all && \ rm -rf /var/cache/yum/* && \ mkdir -p /usr/share/jenkins COPY slave.jar /usr/share/jenkins/slave.jar COPY jenkins-slave /usr/bin/jenkins-slave COPY settings.xml /etc/maven/settings.xml RUN chmod +x /usr/bin/jenkins-slave ENTRYPOINT ["jenkins-slave"] [root@centos7 jenkins-slave]# docker build -t 192.168.0.12/libary/jenkins-slave-jdk:1.8 .