K8S-Demo叢集實踐：部署etcd三節點高可用叢集
阿新 • • 發佈：2021-01-20
K8S-Demo叢集實踐：部署etcd三節點高可用叢集
- ETCD 是基於Raft演算法的分散式，一致性的KV儲存系統，由CoreOS公司發起的一個開源專案，授權協議為Apache。
- 通過前面環境變數你已經知道了叢集節點名稱和IP
- master1：192.168.66.10
- master2：192.168.66.11
- master3：192.168.66.12
- 如果沒有特殊說明，本文件的所有操作均在master1節點上執行
- k8s-demo採用 Etcd v3.4.x版本
- 如果跨主機通訊方案選擇flanneld，則需要將Etcd降級到v3.3.x版本
一、下載和分發etcd二進位制檔案
[[email protected] ~]# cd /opt/install/soft
[[email protected] soft]# wget https://github.com/coreos/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-amd64.tar.gz
[ 二、配置etcd服務
1、準備服務模板
[[email protected] ~]# cd /opt/install/service
[[email protected] service]# cat > etcd.service.template <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
Type=notify
WorkingDirectory=${ETCD_DATA_DIR}
ExecStart=/opt/k8s/bin/etcd \\
--data-dir=${ETCD_DATA_DIR} \\
--wal-dir=${ETCD_WAL_DIR} \\
--name=##NODE_NAME## \\
--cert-file=/opt/k8s/etc/etcd/cert/etcd.pem \\
--key-file=/opt/k8s/etc/etcd/cert/etcd-key.pem \\
--trusted-ca-file=/opt/k8s/etc/cert/ca.pem \\
--peer-cert-file=/opt/k8s/etc/etcd/cert/etcd.pem \\
--peer-key-file=/opt/k8s/etc/etcd/cert/etcd-key.pem \\
--peer-trusted-ca-file=/opt/k8s/etc/cert/ca.pem \\
--peer-client-cert-auth \\
--client-cert-auth \\
--listen-peer-urls=https://##NODE_IP##:2380 \\
--initial-advertise-peer-urls=https://##NODE_IP##:2380 \\
--listen-client-urls=https://##NODE_IP##:2379,http://127.0.0.1:2379 \\
--advertise-client-urls=https://##NODE_IP##:2379 \\
--initial-cluster-token=etcd-cluster-0 \\
--initial-cluster=${ETCD_NODES} \\
--initial-cluster-state=new \\
--auto-compaction-mode=periodic \\
--auto-compaction-retention=1 \\
--max-request-bytes=33554432 \\
--quota-backend-bytes=6442450944 \\
--heartbeat-interval=250 \\
--election-timeout=2000
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
2、為每個節點生產服務配置檔案
[[email protected] ~]# cd /opt/install/service
[[email protected] service]# for (( i=0; i < 3; i++ ))
do
sed -e "s/##NODE_NAME##/${MASTER_NAMES[i]}/" -e "s/##NODE_IP##/${MASTER_IPS[i]}/" etcd.service.template > etcd-${MASTER_IPS[i]}.service
done
3、分發服務配置檔案到3個Master節點
[[email protected] ~]# cd /opt/install/service
[[email protected] service]# for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
scp etcd-${node_ip}.service [email protected]${node_ip}:/etc/systemd/system/etcd.service
done
4、啟動Etcd服務
[[email protected] ~]# for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh [email protected]${node_ip} "mkdir -p ${ETCD_DATA_DIR} ${ETCD_WAL_DIR}"
ssh [email protected]${node_ip} "systemctl daemon-reload && systemctl enable etcd && systemctl restart etcd "
done
- 如果遇到錯誤，請檢查etcd資料目錄和工作目錄是否建立成功
三、檢查etcd服務狀態
1、檢查etcd服務是否成功啟動
[[email protected] ~]# for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
ssh [email protected]${node_ip} "systemctl status etcd|grep Active"
done
- 確保狀態為active (running)，否則檢視日誌:
[[email protected] ~]# journalctl -u etcd
2、檢查etcd服務健康狀況
[[email protected] ~]# for node_ip in ${MASTER_IPS[@]}
do
echo ">>> ${node_ip}"
/opt/k8s/bin/etcdctl \
--endpoints=https://${node_ip}:2379 \
--cacert=/opt/k8s/etc/cert/ca.pem \
--cert=/opt/k8s/etc/etcd/cert/etcd.pem \
--key=/opt/k8s/etc/etcd/cert/etcd-key.pem endpoint health
done
- 預期輸出
>>> 192.168.66.10
https://192.168.66.10:2379 is healthy: successfully committed proposal: took = 6.196779ms
>>> 192.168.66.11
https://192.168.66.11:2379 is healthy: successfully committed proposal: took = 7.343025ms
>>> 192.168.66.12
https://192.168.66.12:2379 is healthy: successfully committed proposal: took = 7.327491ms
3、檢視當前Leader節點
[[email protected] ~]# /opt/k8s/bin/etcdctl \
-w table --cacert=/etc/kubernetes/cert/ca.pem \
--cert=/opt/k8s/etc/etcd/cert/etcd.pem \
--key=/opt/k8s/etc/etcd/cert/etcd-key.pem \
--endpoints=${ETCD_ENDPOINTS} endpoint status
預期輸出
±---------------------------±-----------------±--------±--------±----------±-----------±----------±-----------±-------------------±-------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
±---------------------------±-----------------±--------±--------±----------±-----------±----------±-----------±-------------------±-------+
| https://192.168.66.10:2379 | b2f5e996ff142369 | 3.4.3 | 20 kB | true | false | 112 | 15 | 15 | |
| https://192.168.66.11:2379 | 8aac12c9432579ff | 3.4.3 | 20 kB | false | false | 112 | 15 | 15 | |
| https://192.168.66.12:2379 | eee70ab8f420a137 | 3.4.3 | 20 kB | false | false | 112 | 15 | 15 | |
±---------------------------±-----------------±--------±--------±----------±-----------±----------±-----------±-------------------±-------+
- 當前的leader為192.168.66.10
四、etcdctl命令介紹
# 存入
etcdctl put key value
# 讀取
etcdctl get key
# 區間查詢
etcdctl get key1 key2
# 只讀取key的值
etcdctl get --print-value-only key
# 讀取key開頭的資料
etcdctl get --prefix key
# 讀取所有key
etcdctl get / --prefix --keys-only
# 從key開始讀取後面的資料
etcdctl get --from-key key
# 查詢所有的key-value
etcdctl get --from-key ""
# 刪除
etcdctl del key
etcdctl del --prev-kv key
etcdctl del --prev-kv --from-key key
etcdctl del --prev-kv --prefix key
# 刪除所有資料
etcdctl del --prefix ""
- 參考 https://www.cnblogs.com/doscho/p/6252556.html
五、etcd備份和恢復
# 備份快照：可以定期執行
etcdctl snapshot save /data/backup/xxxx.db
# 恢復步驟
1、關閉所有Master節點的 Etcd服務 systemctl stop etcd
2、備份 ETCD 儲存目錄下資料
3、拷貝 ETCD 備份快照到每個Etcd節點
4、在每個節點執行恢復命令
ETCDCTL_API=3 etcdctl snapshot restore /data/backup/etcd-snapshot-xxxxx.db \
--name etcd-0 \
--initial-cluster "master1=https://192.168.66.10:2380,master2=https://192.168.66.11:2380,master3=https://192.168.66.12:2380" \
--initial-cluster-token etcd-cluster \
--initial-advertise-peer-urls https://192.168.66.10:2380 \
--data-dir=/var/lib/etcd/default.etcd
- 參考 https://zhuanlan.zhihu.com/p/101523337
- 叢集的狀態資料都儲存在Etcd中，及時備份，完善的恢復流程和指令碼很重要
- 先用起來，通過操作實踐認識k8s，積累多了自然就理解了
- 把理解的知識分享出來，自造福田，自得福緣
- 追求簡單，容易使人理解，知識的上下文也是知識的一部分，例如版本，時間等
- 歡迎留言交流，也可以提出問題，一般在週末回覆和完善文件
- [email protected] 2021-1-19。