1. 程式人生 > 其它 >第五章 部署master主控節點

第五章 部署master主控節點

一、部署etcd叢集

1、叢集規劃

10.4.7.12		leader
10.4.7.21		follower
10.4.7.22		follower

本例部署以10.4.7.12為例,另外兩臺安裝類似

2、簽發證書

在10.4.7.200上操作
建立基於根證書的config配置檔案

[root@hdss7-200 ~]# vim /opt/certs/ca-config.json
{
    "signing": {
        "default": {
            "expiry": "175200h"
        },
        "profiles": {
            "server": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth"
                ]
            },
            "client": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "client auth"
                ]
            },
            "peer": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}
註釋:
Server certificate:服務端使用,客戶端以此驗證服務端身份,例如docker服務端,kube-apiserver服務端;
Client certificate:客戶端使用,用於服務端驗證客戶端身份,etcdctl、etcd-proxy、fleetctl、docker客戶端;
Peer certificate:雙向證書,用於etcd叢集之間相互通訊。

3、建立生成自簽證書籤名請求(csr)的JSON配置檔案

[root@hdss7-200 ~]# vim /opt/certs/etcd-peer-csr.json
{
    "CN": "k8s-etcd",
    "hosts": [
        "10.4.7.11",
        "10.4.7.12",
        "10.4.7.21",
        "10.4.7.22"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "od",
            "OU": "ops"
        }
    ]
}
注意:重點在hosts上,將所有可能的etcd伺服器新增到host列表,不能使用網段,新增etcd伺服器需要重新簽發證書

4、簽發證書

[root@hdss7-200 harbor]# cd /opt/certs/
[root@hdss7-200 certs]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssl-json -bare etcd-peer
[root@hdss7-200 certs]# ls etcd*
etcd-peer.csr  etcd-peer-csr.json  etcd-peer-key.pem  etcd-peer.pem

5、安裝etcd服務

在10.4.7.12/21/22上部署,以12為例

[root@hdss7-12 ~]# mkdir /opt/src
[root@hdss7-12 ~]# cd /opt/src/
[root@hdss7-12 src]# curl -L https://github.com/coreos/etcd/releases/download/v3.3.1/etcd-v3.3.1-linux-amd64.tar.gz -o etcd-v3.3.1-linux-amd64.tar.gz
[root@hdss7-12 src]# tar -zxvf etcd-v3.3.1-linux-amd64.tar.gz -C /opt/etcd-v3.3.1
[root@hdss7-12 src]# ln -s /opt/etcd-v3.3.1 /opt/etcd
[root@hdss7-12 src]# ll /opt/
總用量 8
lrwxrwxrwx 1 root root   16 6月   8 20:36 etcd -> /opt/etcd-v3.3.1
drwxr-xr-x 5 etcd etcd 4096 7月   9 20:40 etcd-v3.3.1
drwxr-xr-x 2 root root 4096 7月   9 20:36 src
建立目錄
[root@hdss7-12 src]# mkdir -p /opt/etcd/certs /data/etcd /data/logs/etcd-server

6、下發證書

將10.4.7.200上生成的證書分配到etcd叢集伺服器上

[root@hdss7-200 certs]# for i in 12 21 22;do scp ca.pem etcd-peer.pem etcd-peer-key.pem hdss7-${i}:/opt/etcd/certs/ ;done

7、建立etcd啟動指令碼

[root@hdss7-12 src]# vim /opt/etcd/etcd-server-startup.sh
#!/bin/sh
# listen-peer-urls etcd節點之間通訊埠
# listen-client-urls 客戶端與etcd通訊埠
# quota-backend-bytes 配額大小
# 需要修改的引數:name,listen-peer-urls,listen-client-urls,initial-advertise-peer-urls,advertise-client-urls
WORK_DIR=$(dirname $(readlink -f $0))
[ $? -eq 0 ] && cd $WORK_DIR || exit

/opt/etcd/etcd --name etcd-server-7-12 \
    --data-dir /data/etcd/etcd-server \
    --listen-peer-urls https://10.4.7.12:2380 \
    --listen-client-urls https://10.4.7.12:2379,http://127.0.0.1:2379 \
    --quota-backend-bytes 8000000000 \
    --initial-advertise-peer-urls https://10.4.7.12:2380 \
    --advertise-client-urls https://10.4.7.12:2379,http://127.0.0.1:2379 \
    --initial-cluster  etcd-server-7-12=https://10.4.7.12:2380,etcd-server-7-21=https://10.4.7.21:2380,etcd-server-7-22=https://10.4.7.22:2380 \
    --ca-file ./certs/ca.pem \
    --cert-file ./certs/etcd-peer.pem \
    --key-file ./certs/etcd-peer-key.pem \
    --client-cert-auth  \
    --trusted-ca-file ./certs/ca.pem \
    --peer-ca-file ./certs/ca.pem \
    --peer-cert-file ./certs/etcd-peer.pem \
    --peer-key-file ./certs/etcd-peer-key.pem \
    --peer-client-cert-auth \
    --peer-trusted-ca-file ./certs/ca.pem \
--log-output stdout
[root@hdss7-12 src]# chmod +x /opt/etcd/etcd-server-startup.sh
[root@hdss7-12 src]# chown -R etcd.etcd /opt/etcd/ /data/etcd /data/logs/etcd-server

8、安裝後臺管理工具supervisor

[root@hdss7-12 src]# yum -y install supervisor
[root@hdss7-12 src]# systemctl start supervisord ; systemctl enable supervisord
[root@hdss7-12 src]# vim /etc/supervisord.d/etcd-server.ini
[program:etcd-server-7-12]
command=/opt/etcd/etcd-server-startup.sh              ; the program (relative uses PATH, can take args)
numprocs=1                                            ; number of processes copies to start (def 1)
directory=/opt/etcd                                   ; directory to cwd to before exec (def no cwd)
autostart=true                                        ; start at supervisord start (default: true)
autorestart=true                                      ; retstart at unexpected quit (default: true)
startsecs=30                                          ; number of secs prog must stay running (def. 1)
startretries=3                                        ; max # of serial start failures (default 3)
exitcodes=0,2                                         ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                       ; signal used to kill process (default TERM)
stopwaitsecs=10                                       ; max num secs to wait b4 SIGKILL (default 10)
user=etcd                                             ; setuid to this UNIX account to run the program
redirect_stderr=true                                  ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/etcd-server/etcd.stdout.log ; stdout log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                          ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=5                              ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                           ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                           ; emit events on stdout writes (default false)
[root@hdss7-12 src]# supervisorctl update
etcd-server-7-12: added process group/
[root@hdss7-12 src]# supervisorctl start etcd-server-7-12
etcd-server-7-12: started

9、檢視叢集服務狀態

[root@hdss7-12 src]#  supervisorctl status
etcd-server-7-12                 RUNNING   pid 2270, uptime 0:00:38
檢視etcd後臺埠2379與2380
檢視器群狀態,如下是叢集未全部啟動
[root@hdss7-12 src] /opt/etcd/etcdctl member list
client: etcd cluster is unavailable or misconfigured; error #0: client: endpoint http://127.0.0.1:2379 exceeded header timeout
; error #1: dial tcp 127.0.0.1:4001: getsockopt: connection refused
如下是起來兩個節點的時候
[root@hdss7-12 src] /opt/etcd/etcdctl member list
988139385f78284:	 name=etcd-server-7-22 peerURLs=https://10.4.7.22:2380 clientURLs= isLeader=false
5a0ef2a004fc4349: name=etcd-server-7-21 peerURLs=https://10.4.7.21:2380 clientURLs=http://127.0.0.1:2379,https://10.4.7.12:2379 isLeader=false
f4a0cb0a765574a8: name=etcd-server-7-12 peerURLs=https://10.4.7.12:2380 clientURLs=http://127.0.0.1:2379,https://10.4.7.12:2379 isLeader=true
全部起來的情況
[root@hdss7-12 src] /opt/etcd/etcdctl member list
988139385f78284: name=etcd-server-7-22 peerURLs=https://10.4.7.22:2380 clientURLs=http://127.0.0.1:2379,https://10.4.7.12:2379 isLeader=false
5a0ef2a004fc4349: name=etcd-server-7-21 peerURLs=https://10.4.7.21:2380 clientURLs=http://127.0.0.1:2379,https://10.4.7.12:2379 isLeader=false
f4a0cb0a765574a8: name=etcd-server-7-12 peerURLs=https://10.4.7.12:2380 clientURLs=http://127.0.0.1:2379,https://10.4.7.12:2379 isLeader=true
隨著etcd服務的重啟會發生變化
檢視健康狀態
[root@hdss7-12 src] /opt/etcd/etcdctl cluster-health
member 988139385f78284 is healthy: got healthy result from http://127.0.0.1:2379
member 5a0ef2a004fc4349 is healthy: got healthy result from http://127.0.0.1:2379
member f4a0cb0a765574a8 is healthy: got healthy result from http://127.0.0.1:2379
cluster is healthy
啟停方式
src]# supervisorctl stop etcd-server-7-12
src]# supervisorctl start etcd-server-7-12
src]# supervisorctl restart etcd-server-7-12
src]# supervisorctl status etcd-server-7-12

二、部署kube-apiserver叢集

三、部署controller-manager

四、部署kube-scheduler

五、檢查主控節點