Redis Cluster叢集架構實現(四)--技術流ken
阿新 • • 發佈：2018-11-16
Redis叢集簡介
通過前面三篇部落格的介紹《Redis基礎認識及常用命令使用(一)--技術流ken》，《Redis基礎知識補充及持久化、備份介紹(二)--技術流ken》，《Redis主從複製、多例項、高可用(三)--技術流ken》，現在已經對redis的基礎知識，常用命令，持久化，備份，主從複製，多例項的安裝以及redis的高可用熟練掌握了。本篇部落格將介紹redis cluster叢集，也是一個比較複雜的內容，本篇部落格將採用較為簡潔的方式來呈現redis叢集。
有關redis叢集的介紹可以參考下面我摘自redis官網的簡介。
Redis 叢集是一個可以在多個 Redis 節點之間進行資料共享
Redis 叢集不支援那些需要同時處理多個鍵的 Redis 命令， 因為執行這些命令需要在多個 Redis 節點之間移動資料， 並且在高負載的情況下， 這些命令將降低 Redis 叢集的效能， 並導致不可預測的行為。
Redis 叢集通過分割槽（partition）來提供一定程度的可用性（availability）： 即使叢集中有一部分節點失效或者無法進行通訊， 叢集也可以繼續處理命令請求。
Redis 叢集提供了以下兩個好處：
- 將資料自動切分（split）到多個節點的能力。
- 當叢集中的一部分節點失效或者無法進行通訊時， 仍然可以繼續處理命令請求的能力。 --摘自redis官網
總而言之，redis叢集實現了資料的共享以及去中心化。
Redis叢集實現
實現redis叢集，現在採用一臺伺服器安裝6個redis例項，有關如果安裝多例項的詳細講解請參考我上篇部落格《Redis主從複製、多例項、高可用(三)--技術流ken》
6個多例項，三個為主節點，三個為從節點
環境：
CentOS Linux release 7.5.1804 (Core)
redis-4.0.11
建立多例項
第一步：建立多例項資料目錄
[email protected] ~]# mkdir /ken [[email protected]~]# cd /ken [[email protected] ken]# mkdir 6379 6380 6381 6382 6383 6384 [[email protected] ken]# ls 6379 6380 6381 6382 6383 6384
第二步：上傳redis安裝包
這裡使用的是redis-4.0.11的版本，可以在redis官方網站進行下載https://redis.io/
[[email protected] ~]# rz
第三步：解壓安裝包
[[email protected] ~]# tar xf redis-4.0.11.tar.gz
第四步：複製解壓安裝包的配置檔案到6379目錄下
[[email protected] ~]# cp redis-4.0.11/redis.conf /ken/6379/
第五步：移動redis解壓包至/usr/local/redis下
[[email protected] ~]# mv redis-4.0.11 /usr/local/redis
第六步：編譯安裝
[[email protected] ~]# cd /usr/local/redis
[[email protected] redis]# make && make install
[[email protected] ~]# ln /usr/local/redis/src/ /bin -s
第七步：修改配置檔案
[[email protected] ~]# grep -E -v "^#|^$" /ken/6379/redis.conf bind 10.220.5.137 #繫結本機ip地址 protected-mode yes port 6379 #監聽埠 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize yes #後臺執行 supervised no pidfile /ken/6379/redis_6379.pid #pid檔案儲存位置 loglevel notice logfile "" databases 16 always-show-logo yes save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb dir /ken/6379/ #rdb檔案儲存位置 slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no slave-lazy-flush no appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble no lua-time-limit 5000 cluster-enabled yes #開啟叢集 cluster-config-file nodes-6379.conf #叢集檔名稱 cluster-node-timeout 15000 #叢集超時時間 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes
第八步：把上面這個修改過的檔案複製到其他的例項資料目錄之下
[[email protected] ~]# cp /ken/6379/redis.conf /ken/6380/ [[email protected] ~]# cp /ken/6379/redis.conf /ken/6381/ [[email protected] ~]# cp /ken/6379/redis.conf /ken/6382/ [[email protected] ~]# cp /ken/6379/redis.conf /ken/6383/ [[email protected] ~]# cp /ken/6379/redis.conf /ken/6384/
第九步:修改配置檔案
只要使用sed即可進行修改
[[email protected] ~]# sed -i 's/6379/6380/g' /ken/6380/redis.conf [[email protected] ~]# sed -i 's/6379/6381/g' /ken/6381/redis.conf [[email protected] ~]# sed -i 's/6379/6382/g' /ken/6382/redis.conf [[email protected] ~]# sed -i 's/6379/6383/g' /ken/6383/redis.conf [[email protected] ~]# sed -i 's/6379/6384/g' /ken/6384/redis.conf
第十步：啟動各個例項
[[email protected] ~]# redis-server /ken/6380/redis.conf [[email protected] ~]# redis-server /ken/6381/redis.conf [[email protected] ~]# redis-server /ken/6382/redis.conf [[email protected] ~]# redis-server /ken/6383/redis.conf [[email protected] ~]# redis-server /ken/6384/redis.conf [[email protected] ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 10.220.5.137:16380 *:* LISTEN 0 128 10.220.5.137:16381 *:* LISTEN 0 128 10.220.5.137:16382 *:* LISTEN 0 128 10.220.5.137:16383 *:* LISTEN 0 128 10.220.5.137:16384 *:* LISTEN 0 128 *:10050 *:* LISTEN 0 128 *:10051 *:* LISTEN 0 128 10.220.5.137:6379 *:* LISTEN 0 128 10.220.5.137:6380 *:* LISTEN 0 128 10.220.5.137:6381 *:* LISTEN 0 128 10.220.5.137:6382 *:* LISTEN 0 128 10.220.5.137:6383 *:* LISTEN 0 128 *:111 *:* LISTEN 0 128 10.220.5.137:6384 *:* LISTEN 0 128 *:22 *:* LISTEN 0 128 10.220.5.137:16379 *:* LISTEN 0 128 :::10050 :::* LISTEN 0 128 :::10051 :::* LISTEN 0 70 :::3306 :::* LISTEN 0 128 :::111 :::* LISTEN 0 128 :::80 :::* LISTEN 0 128 :::22 :::*
安裝ruby2.3
實現redis cluster功能，依賴redis-trib.rb，而這個工具是依賴一個ruby開發工具包的，所以需要安裝ruby環境，並安裝依賴包
第一步:上傳解壓安裝包
[[email protected] ~]# rz [[email protected] ~]# tar xf ruby-2.3.5.tar.gz
第二步：編譯安裝
建議虛擬機器記憶體至少1個G以上
[[email protected] ~]# cd ruby-2.3.5/ [[email protected] ruby-2.3.5]# ./configure --prefix=/usr/local/ruby && make && make install
第三步：安裝redis-trib.rb的依賴
[[email protected] ~]# ln -s /usr/local/ruby/bin/gem /bin [[email protected] ~]# gem install -l redis-3.3.0.gem Successfully installed redis-3.3.0 Parsing documentation for redis-3.3.0 Installing ri documentation for redis-3.3.0 Done installing documentation for redis after 0 seconds 1 gem installed
[[email protected] ~]# ln -s /usr/local/ruby/bin/ruby /bin
獲取叢集幫助
只需要輸入redis-trib.rb回車即可
[[email protected] ~]# redis-trib.rb Usage: redis-trib <command> <options> <arguments ...> create host1:port1 ... hostN:portN --replicas <arg> check host:port info host:port fix host:port --timeout <arg> reshard host:port --from <arg> --to <arg> --slots <arg> --yes --timeout <arg> --pipeline <arg> rebalance host:port --weight <arg> --auto-weights --use-empty-masters --timeout <arg> --simulate --pipeline <arg> --threshold <arg> add-node new_host:new_port existing_host:existing_port --slave --master-id <arg> del-node host:port node_id set-timeout host:port milliseconds call host:port command arg arg .. arg import host:port --from <arg> --copy --replace help (show this help) For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
建立redis叢集
命令的意義如下：
- 給定
redis-trib.rb程式的命令是create， 這表示我們希望建立一個新的叢集。 - 選項
--replicas 1表示我們希望為叢集中的每個主節點建立一個從節點。 - 之後跟著的其他引數則是例項的地址列表， 我們希望程式使用這些地址所指示的例項來建立新叢集。
簡單來說， 以上命令的意思就是讓 redis-trib 程式建立一個包含三個主節點和三個從節點的叢集。
接著， redis-trib 會打印出一份預想中的配置給你看， 如果你覺得沒問題的話， 就可以輸入 yes ， redis-trib 就會將這份配置應用到叢集當中：
[[email protected] ~]# redis-trib.rb create --replicas 1 10.220.5.137:6379 10.220.5.137:6380 10.220.5.137:6381 10.220.5.137:6382 10.220.5.137:6383 10.220.5.137:6384 >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 10.220.5.137:6379 10.220.5.137:6380 10.220.5.137:6381 Adding replica 10.220.5.137:6383 to 10.220.5.137:6379 Adding replica 10.220.5.137:6384 to 10.220.5.137:6380 Adding replica 10.220.5.137:6382 to 10.220.5.137:6381 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-10922 (5462 slots) master M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:10923-16383 (5461 slots) master S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 Can I set the above configuration? (type 'yes' to accept): yes #輸入yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join..... >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
Redis叢集測試
測試 Redis 叢集比較簡單的辦法就是使用 redis-rb-cluster 或者 redis-cli ， 接下來將使用 redis-cli 為例來進行演示：
第一步：登入叢集中
一定要加個-c，後面輸入哪個埠都可以，這就是redis的區中心化思想
[[email protected] ~]# redis-cli -c -h 10.220.5.137 -p 6379
第二步：建立key
可以發現建立的key被分配到了不同的節點
10.220.5.137:6379> keys * (empty list or set) 10.220.5.137:6379> set name ken -> Redirected to slot [5798] located at 10.220.5.137:6380 OK 10.220.5.137:6380> set addr jiangsu -> Redirected to slot [12790] located at 10.220.5.137:6381 OK 10.220.5.137:6381> set tel 123445 -> Redirected to slot [7485] located at 10.220.5.137:6380 OK 10.220.5.137:6380> set ege 25 OK 10.220.5.137:6380> set gender male -> Redirected to slot [15355] located at 10.220.5.137:6381 OK 10.220.5.137:6381> keys * 1) "addr" 2) "gender"
第三步：獲取key
在埠6381的節點之上沒有tel這個key,但是我們仍然可以使用get tel獲取到值，這就實現了redis叢集的資料共享
10.220.5.137:6381> keys * 1) "addr" 2) "gender" 10.220.5.137:6381> get tel -> Redirected to slot [7485] located at 10.220.5.137:6380 "123445"
在叢集中新增新的節點
第一步：建立一個新的例項
[[email protected] ~]# mkdir /ken/6385 [[email protected] ~]# cp /ken/6379/redis.conf /ken/6385/ [[email protected] ~]# sed -i 's/6379/6385/g' /ken/6385/redis.conf [[email protected] ~]# redis-server /ken/6385/redis.conf 10461:C 15 Nov 21:21:11.646 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 10461:C 15 Nov 21:21:11.646 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=10461, just started 10461:C 15 Nov 21:21:11.646 # Configuration loaded
第二步：新增該節點到叢集中
命令中的 add-node 表示我們要讓 redis-trib 將一個節點新增到叢集裡面， add-node 之後跟著的是新節點的 IP 地址和埠號， 再之後跟著的是叢集中任意一個已存在節點的 IP 地址和埠號， 這裡我們使用的是 10.220.5.137:6379 。
[[email protected] ~]# redis-trib.rb add-node 10.220.5.137:6385 10.220.5.137:6379 >>> Adding node 10.220.5.137:6385 to cluster 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:10923-16383 (5461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. >>> Send CLUSTER MEET to node 10.220.5.137:6385 to make it join the cluster. [OK] New node added correctly.
第三步:檢視
通過 cluster nodes 命令， 我們可以確認新節點10.220.5.137:6385 已經被新增到叢集裡面了
10.220.5.137:6379> CLUSTER nodes 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379@16379 myself,master - 0 1542288244000 1 connected 0-5460 daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385@16385 master - 0 1542288242824 0 connected ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381@16381 master - 0 1542288244000 3 connected 10923-16383 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380@16380 master - 0 1542288242000 2 connected 5461-10922 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383@16383 slave 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 0 1542288242000 5 connected b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384@16384 slave ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 0 1542288244852 6 connected 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382@16382 slave 57753754f4c89054ab14e8ec517604c4fc4c8ed5 0 1542288243844 4 connected
第四步：使新增的新節點為從節點
如果我們打算讓新節點成為 10.220.5.137:6379 的從節點， 那麼我們只要用客戶端連線上新節點， 然後執行以下命令就可以了
[[email protected] ~]# redis-cli -c -h 10.220.5.137 -p 6385 10.220.5.137:6385> CLUSTER REPLICATE 57753754f4c89054ab14e8ec517604c4fc4c8ed5 OK
其中57753754f4c89054ab14e8ec517604c4fc4c8ed5是主節點10.220.5.137:6379的id
對redis中的資料做重新分片
[[email protected] ~]# redis-trib.rb reshard 10.220.5.137:6379
你只需要指定叢集中其中一個節點的地址， redis-trib 就會自動找到叢集中的其他節點。
How many slots do you want to move (from 1 to 16384)? 1000
我們將打算移動的槽數量設定為 1000 個。
What is the receiving node ID? 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d
定目標需要使用節點的 ID ， 而不是 IP 地址和埠。 比如說， 我們打算使用叢集的第一個主節點來作為目標， 它的 IP 地址和埠是 10.220.5.137:6380 ， 而節點 ID 則是3a9aa9592afc594c7e4206cc82ffb37d46a5b23d ， 那麼我們應該向 redis-trib 提供節點的 ID
Source node #1:ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8
接著， redis-trib 會向你詢問重新分片的源節點（source node）， 也即是， 要從哪個節點中取出 1000 個雜湊槽， 並將這些槽移動到目標節點上面。
Source node #2:done
Do you want to proceed with the proposed reshard plan (yes/no)? yes
輸入 yes 並使用按下回車之後， redis-trib 就會正式開始執行重新分片操作， 將指定的雜湊槽從源節點一個個地移動到目標節點上面
Moving slot 12905 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12906 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12907 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12908 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12909 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12910 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12911 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12912 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12913 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12914 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12915 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12916 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12917 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12918 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12919 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12920 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12921 from 10.220.5.137:6381 to 10.220.5.137:6380: Moving slot 12922 from 10.220.5.137:6381 to 10.220.5.137:6380:
在重新分片操作執行完畢之後， 可以使用以下命令來檢查叢集是否正常
[[email protected] ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 2 additional replica(s) S: daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-12922 (7462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
刪除節點
如果節點中有slot，那麼需要先將slot 執行reshard給其他節點，然後才能執行刪除操作
1. 刪除空slot
刪除10.220.5.137:6385
[[email protected] ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 2 additional replica(s) S: daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d 10.220.5.137:6385 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) ...
執行如下命令
[[email protected] ~]# redis-trib.rb del-node 10.220.5.137:6379 daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d >>> Removing node daf9464ef45d0c73e1ee18f7cc7ddbc7d9719f2d from cluster 10.220.5.137:6379 >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node.
檢視發現6385節點已經被移除
[[email protected] ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-12922 (7462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
2.刪除帶有slot的節點
提示有資料報錯
[[email protected] ~]# redis-trib.rb del-node 10.220.5.137:6379 ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 >>> Removing node ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 from cluster 10.220.5.137:6379 [ERR] Node 10.220.5.137:6381 is not empty! Reshard data away and try again.
需要重新分片
[[email protected] ~]# redis-trib.rb reshard 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 slots:12923-16383 (3461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-12922 (7462 slots) master 1 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. How many slots do you want to move (from 1 to 16384)? 3461 What is the receiving node ID? 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node #1:ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 Source node #2:done
再檢查slot已經為空
>>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 10.220.5.137:6381 #已經為空 slots: (0 slots) master 0 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-16383 (10923 slots) master 2 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
再執行刪除操作
[[email protected] ~]# redis-trib.rb del-node 10.220.5.137:6379 ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 >>> Removing node ffadc66b51b6c87ad3c5a33dc9ec39f984bb80e8 from cluster 10.220.5.137:6379 >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node.
再次檢視。6381節點已經被刪除
[[email protected] ~]# redis-trib.rb check 10.220.5.137:6379 >>> Performing Cluster Check (using node 10.220.5.137:6379) M: 57753754f4c89054ab14e8ec517604c4fc4c8ed5 10.220.5.137:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) M: 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d 10.220.5.137:6380 slots:5461-16383 (10923 slots) master 2 additional replica(s) S: 640bb3a62565a0bb74980852fee7e8636cf238bc 10.220.5.137:6383 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: b8214c9e4b617c7992e71a2fa07a4a64cfeddf58 10.220.5.137:6384 slots: (0 slots) slave replicates 3a9aa9592afc594c7e4206cc82ffb37d46a5b23d S: 1bc40ffc4c54099f8cf47efdeb82a9f42e717bf6 10.220.5.137:6382 slots: (0 slots) slave replicates 57753754f4c89054ab14e8ec517604c4fc4c8ed5 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.