Redis Cluster 部署、線上擴容、縮容、遷移、叢集管理實踐
概述
Redis 分片叢集是為了應對業務增長帶來的資料增長, 實現線上動態橫向擴縮容，叢集中的每個分片主節點處理 16384 個雜湊槽的一個子集; 為了穩定也可以進行高可用部署, 即將各分片節點啟用一個或多個副本節點; 當分片主節點掛掉,其對應的副本節點自動提升為新主節點,當原主節點重新上線後,自動成為新主節點的副本節點; 當一對分片主從節點都故障時,整個叢集不能正常工作，需要人工介入恢復啟動故障節點分片叢集
1、建立一個有 3 個分片節點的叢集# 建立具有 3 個分片節點的叢集、每個分片有 1 個副本 # redis-cli --cluster create 192.168.31.17:6384 192.168.31.18:6384 192.168.31.19:6384 --cluster-replicas 1建立具有 3 個分片節點的叢集, 每個分片預設無副本
redis-cli --cluster create 192.168.31.17:6384 192.168.31.18:6384 192.168.31.19:6384
> >>> Performing hash slots allocation on 3 nodes...
> Master[0] -> Slots 0 - 5460
> Master[1] -> Slots 5461 - 10922
> Master[2] -> Slots 10923 - 16383
> M: 1db08d1048b20b22dfbf5c5d724d95f46a843d29 192.168.31.17:6384
> slots:[0-5460] (5461 slots) master
> M: 3f9d82efdf04bb5fcf3741de4ea9314925bbeefd 192.168.31.18:6384
> slots:[5461-10922] (5462 slots) master
> M: 16a0c6947fcb7d498b7c96aa5b5a70491298e966 192.168.31.19:6384
> slots:[10923-16383] (5461 slots) master
> Can I set the above configuration? (type 'yes' to accept): yes
> >>> Nodes configuration updated
> >>> Assign a different config epoch to each node
> >>> Sending CLUSTER MEET messages to join the cluster
> Waiting for the cluster to join
> ..
> >>> Performing Cluster Check (using node 192.168.31.17:6384)
> M: 1db08d1048b20b22dfbf5c5d724d95f46a843d29 192.168.31.17:6384
> slots:[0-5460] (5461 slots) master
> M: 16a0c6947fcb7d498b7c96aa5b5a70491298e966 192.168.31.19:6384
> slots:[10923-16383] (5461 slots) master
> M: 3f9d82efdf04bb5fcf3741de4ea9314925bbeefd 192.168.31.18:6384
> slots:[5461-10922] (5462 slots) master
> [OK] All nodes agree about slots configuration.
> >>> Check for open slots...
> >>> Check slots coverage...
> [OK] All 16384 slots covered.
驗證
1、檢視指定 redis 節點狀態(包括鍵數、記憶體使用量、請求數、連線數等)2、檢視各主節點雜湊槽、鍵的分佈情況, 以及副本數、鍵總數redis-cli --stat -p 6384 -h 192.168.31.17 > ------- data ------ --------------------- load -------------------- - child - > keys mem clients blocked requests connections > 30022 9.25M 2 0 30518 (+2) 59 > 30022 9.25M 2 0 30520 (+2) 59 > 30022 9.25M 2 0 30522 (+2) 59 > 30022 9.25M 2 0 30524 (+2) 59 > 30022 9.25M 2 0 30526 (+2) 59 > 30022 9.25M 2 0 30528 (+2) 59 > 30022 9.25M 2 0 30530 (+2) 59
redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6384 (79c19006...) -> 49888 keys | 5462 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 50115 keys | 5461 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 49995 keys | 5461 slots | 1 slaves.
> [OK] 149998 keys in 3 masters.
> 9.16 keys per slot on average.
叢集管理
1、redis-cli 互動式命令列redis-cli -p 6384 -h 192.168.31.17 # 檢視叢集節點資訊 cluster nodesmeet 新增節點, 任意節點執行即可
cluster meet 192.168.31.17 6385
forget 根據節點 ID 刪除節點, 需要在所有節點執行該命令
cluster forget 82d14741e161d07012783ae7c9bf1ef20ddc28a0
將當前 cluster 節點配置為指定主節點(1db08d1048b20b22dfbf5c5d724d95f46a843d29)的副本節點
cluster replicate 1db08d1048b20b22dfbf5c5d724d95f46a843d29
檢視指定 master 節點(1db08d1048b20b22dfbf5c5d724d95f46a843d29)的副本節點
cluster replicas 1db08d1048b20b22dfbf5c5d724d95f46a843d29
cluster reset # 重置當前節點叢集狀態檔案 nodes.conf
2、redis-cli --cluster 子命令
# 在叢集中所有節點上執行命令 client list
redis-cli --cluster call 192.168.31.17:6384 client list
# 將節點 192.168.31.17:6385 新增為新的主節點, 需要從其它主節點遷移部分槽位到該新主節點才能正常工作
redis-cli --cluster add-node 192.168.31.17:6385 192.168.31.17:6384
# 將節點 192.168.31.17:6385 新增為主節點(1db08d1048b20b22dfbf5c5d724d95f46a843d29)的從節點
redis-cli --cluster add-node 192.168.31.17:6385 192.168.31.17:6384 --cluster-slave --cluster-master-id 1db08d1048b20b22dfbf5c5d724d95f46a843d29
# 刪除節點 82d14741e161d07012783ae7c9bf1ef20ddc28a0 且關閉該節點
redis-cli --cluster del-node 192.168.31.17:6384 82d14741e161d07012783ae7c9bf1ef20ddc28a0
# 從指定節點遷移 1024 個槽位到目標節點 073ad3a6dfe64289be66c6d8e302b29afc4b0ec7
redis-cli --cluster reshard 192.168.31.17:6385 --cluster-from 1db08d1048b20b22dfbf5c5d724d95f46a843d29 --cluster-to 073ad3a6dfe64289be66c6d8e302b29afc4b0ec7 --cluster-slots 1024
# 使用如下命令, 進入互動式輸入對應引數值進行節點間的槽位遷移
redis-cli --cluster reshard 192.168.31.17:6384
# 當各節點槽位不均衡時,執行該命令、自動均衡各節點槽位 slots
redis-cli --cluster rebalance 192.168.31.17:6384
擴容
1、叢集主節點擴容, 由 3 個主節點擴充套件到 5 個主節點
// 新增新節點 192.168.31.{18,19}:6386
redis-cli --cluster add-node 192.168.31.18:6386 192.168.31.17:6384
redis-cli --cluster add-node 192.168.31.19:6386 192.168.31.17:6384
// 自動平衡叢集各主節點雜湊槽的分佈
redis-cli --cluster rebalance 192.168.31.17:6384 --cluster-use-empty-masters
redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6386 (c64357f2...) -> 29894 keys | 3278 slots | 0 slaves.
> 192.168.31.18:6384 (79c19006...) -> 29927 keys | 3276 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 29835 keys | 3276 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 30083 keys | 3276 slots | 1 slaves.
> 192.168.31.19:6386 (a0f71de7...) -> 30259 keys | 3278 slots | 0 slaves.
> [OK] 149998 keys in 5 masters.
> 9.16 keys per slot on average.
縮容
1、叢集主節點擴容, 由 5 個主節點擴充套件到 3 個主節點// 遷移節點 192.168.31.18:6386 的雜湊槽 redis-cli --cluster reshard 192.168.31.17:6384 \ --cluster-from c64357f288902d1209fdf91f4804bf13b2a59fcf \ --cluster-to 79c19006ae3dcd12522090ddcc2b5b917b0feb2a \ --cluster-slots 3278 \ --cluster-yes// 遷移節點 192.168.31.19:6386 的雜湊槽
redis-cli --cluster reshard 192.168.31.17:6384
--cluster-from a0f71de78c2355f99de2b4c19796afb02e23c623
--cluster-to df7d9f8e2d7dbad37bdd1094c2770d1feabbd228
--cluster-slots 3278
--cluster-yes// 檢視各主節點雜湊槽、鍵的分佈情況
redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6386 (c64357f2...) -> 0 keys | 0 slots | 0 slaves.
> 192.168.31.18:6384 (79c19006...) -> 59821 keys | 6554 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 29835 keys | 3276 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 60342 keys | 6554 slots | 1 slaves.
> 192.168.31.19:6386 (a0f71de7...) -> 0 keys | 0 slots | 0 slaves.
> [OK] 149998 keys in 5 masters.
> 9.16 keys per slot on average.// 自動平衡叢集各主節點雜湊槽的分佈
redis-cli --cluster rebalance 192.168.31.17:6384// 刪除節點 192.168.31.{18,19}:6386
redis-cli --cluster del-node 192.168.31.17:6384 c64357f288902d1209fdf91f4804bf13b2a59fcf
redis-cli --cluster del-node 192.168.31.17:6384 a0f71de78c2355f99de2b4c19796afb02e23c623
// 檢視各主節點雜湊槽、鍵的分佈情況
redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6384 (79c19006...) -> 49762 keys | 5461 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 50060 keys | 5462 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 50176 keys | 5461 slots | 1 slaves.
> [OK] 149998 keys in 3 masters.
> 9.16 keys per slot on average.
遷移
Redis-shake 是一個用於在兩個 Redis 之間同步資料的工具,支援 Redis Cluster 的線上遷移與離線遷移(備份檔案匯入) 支援解析、恢復、備份、同步四個功能1、線上遷移
首次全量同步,然後進行增量的拉取(通過 psync 命令)
# 建立遷移目標 redis 叢集
redis-cli --cluster create 192.168.31.17:6386 192.168.31.18:6386 192.168.31.19:6386
# cat redis-shake.conf 源及目標端主要配置項, 其它配置項預設即可
source.type: cluster
# master 可改為 slave, 表示從副本同步資料到目標端, 192.168.31.19 為任意源端 redis 節點
source.address: [email protected]:6384
target.type: cluster
target.address: [email protected]:6386
./redis-shake -type sync -conf redis-shake.conf
2、離線遷移
適用於源例項與目標例項的網路無法連通的場景,或源端例項部署在其他雲廠商 Redis 服務中,無法實現線上遷移
# cat redis-shake.conf 源及目標端主要配置項, 其它配置項預設即可
source.address: [email protected]:6384
# restore 匯入資料時指定的備份檔案
source.rdb.input: local_dump.0;local_dump.1;local_dump.2
target.type: cluster
target.address: [email protected]:6386
備份: ./redis-shake -type dump -conf redis-shake.conf
恢復: ./redis-shake -type restore -conf redis-shake.conf
來源：https://www.iwmyx.cn/redisclusterbszxa.html
來源：運維技術幫