本篇主要講Redis Cluster的安裝，讓我們先用起來，感受一下到底是怎麽回事，後面再繼續學習有關如何Cluster如何failover,添加節點，刪除節點，遷移slots等功能。

Redis Cluster特點

(1)Redis Cluster 共有16384(0-16383)個hash slots,數據寫入時，根據CRC16('key')%16384 hash slots分配到不同的節點上；

(2)當整個集群部分節點crash不影響繼續使用，如有A,B,C三個master和A1,B1,C1三個slave組成的cluster,如果B宕機，則B1會自動提成為master,當B恢復之後，也會自動成為B1的slave,保證集群的高可用性；

(3)當集群負載比較高，不足以支撐現在的業務，可以添加空節點，然後rebalance slot或者reshard slot；

安裝Redis Cluster

(1)環境準備

Redis版本都是3.2.9,有6個redis實例分別在6臺不同的主機上運行，其中3個作為Master,另外3個分別作為它們的Slave存在，端口統一使用默認的端口6379；

Master1: sht-sgmhadoopcm-01(172.16.101.54)

Slave1: sht-sgmhadoopdn-01(172.16.101.58)

Master2: sht-sgmhadoopnn-01(172.16.101.55)

Slave2: sht-sgmhadoopdn-02

Master3: sht-sgmhadoopnn-02(172.16.101.56)

Slave3: sht-sgmhadoopdn-03(172.16.101.60)

#修改配置文件，每個節點都需要執行，修改其他節點對應的IP地址

[root@sht-sgmhadoopcm-01 redis]# vim redis.conf

port 6379

bind 172.16.101.54

appendonly yes

cluster-enabled yes

cluster-config-file nodes-6379.conf

cluster-node-timeout 15000

(2)在需要ruby的節點上安裝ruby

[root@sht-sgmhadoopcm-01 redis]# yum install ruby -y

#下面這一步可能報錯，因為CentOS7.3默認安裝的ruby2.0.0，版本太低

[root@sht-sgmhadoopcm-01 redis]# gem install redis

Fetching: redis-4.0.1.gem (100%)

Successfully installed redis-4.0.1

Parsing documentation for redis-4.0.1

Installing ri documentation for redis-4.0.1

Done installing documentation for redis after 2 seconds

1 gem installed

(3)啟動所有節點

[root@sht-sgmhadoopcm-01 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopnn-02 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopnn-01 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopdn-01 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopdn-02 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopdn-03 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopnn-02 redis]# ps -ef|grep redis

root 1716 1 0 12:40 ? 00:00:00 src/redis-server 172.16.101.56:6379 [cluster]

root 1720 1504 0 12:41 pts/2 00:00:00 grep --color=auto redis

(4)使用redis-trib.rb腳本創建cluster

Create的過程中已經給三個master分配好了hash slots,並且制定好Slave;

選項--replicas 1 表示為每個創建的主服務器節點創建一個從服務器節點;

[root@sht-sgmhadoopcm-01 redis]# ruby src/redis-trib.rb create --replicas 1 172.16.101.54:6379 172.16.101.55:6379 172.16.101.56:6379 172.16.101.58:6379 172.16.101.59:6379 172.16.101.60:6379

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

172.16.101.54:6379

172.16.101.55:6379

172.16.101.56:6379

Adding replica 172.16.101.58:6379 to 172.16.101.54:6379

Adding replica 172.16.101.59:6379 to 172.16.101.55:6379

Adding replica 172.16.101.60:6379 to 172.16.101.56:6379

M: 610aa83831404be545b25cc7f7322e987da1dd33 172.16.101.54:6379

slots:0-5460 (5461 slots) master

M: 4c7348ac575a1a3bd4357e937a8f94f1654193cf 172.16.101.55:6379

slots:5461-10922 (5462 slots) master

M: ece3a80f6989fa7f8316d73ee30ceea84340f24e 172.16.101.56:6379

slots:10923-16383 (5461 slots) master

S: 28e8d5c8fd12533622d8110f8d262cb50120ca02 172.16.101.58:6379

replicates 610aa83831404be545b25cc7f7322e987da1dd33

S: b50ba506525ee547823b3d9a7e5a095933e3ff42 172.16.101.59:6379

replicates 4c7348ac575a1a3bd4357e937a8f94f1654193cf

S: c79de1092011ed395ee772969274c0567c2b5dd1 172.16.101.60:6379

replicates ece3a80f6989fa7f8316d73ee30ceea84340f24e

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join.....

>>> Performing Cluster Check (using node 172.16.101.54:6379)

M: 610aa83831404be545b25cc7f7322e987da1dd33 172.16.101.54:6379

slots:0-5460 (5461 slots) master

1 additional replica(s)

S: b50ba506525ee547823b3d9a7e5a095933e3ff42 172.16.101.59:6379

slots: (0 slots) slave

replicates 4c7348ac575a1a3bd4357e937a8f94f1654193cf

S: 28e8d5c8fd12533622d8110f8d262cb50120ca02 172.16.101.58:6379

slots: (0 slots) slave

replicates 610aa83831404be545b25cc7f7322e987da1dd33

M: 4c7348ac575a1a3bd4357e937a8f94f1654193cf 172.16.101.55:6379

slots:5461-10922 (5462 slots) master

1 additional replica(s)

M: ece3a80f6989fa7f8316d73ee30ceea84340f24e 172.16.101.56:6379

slots:10923-16383 (5461 slots) master

1 additional replica(s)

S: c79de1092011ed395ee772969274c0567c2b5dd1 172.16.101.60:6379

slots: (0 slots) slave

replicates ece3a80f6989fa7f8316d73ee30ceea84340f24e

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

(5)查看Cluster基本信息

[root@sht-sgmhadoopcm-01 redis]# src/redis-cli -c -h 172.16.101.54 -p 6379

172.16.101.54:6379> info replication

# Replication

role:master

connected_slaves:1

slave0:ip=172.16.101.58,port=6379,state=online,offset=2591,lag=0

master_repl_offset:2591

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:2590

172.16.101.54:6379> cluster info

cluster_state:ok

cluster_slots_assigned:16384

cluster_slots_ok:16384

cluster_slots_pfail:0

cluster_slots_fail:0

cluster_known_nodes:6

cluster_size:3

cluster_current_epoch:6

cluster_my_epoch:1

cluster_stats_messages_sent:4462

cluster_stats_messages_received:4462

172.16.101.54:6379> cluster nodes

b50ba506525ee547823b3d9a7e5a095933e3ff42 172.16.101.59:6379 slave 4c7348ac575a1a3bd4357e937a8f94f1654193cf 0 1532497136876 5 connected

610aa83831404be545b25cc7f7322e987da1dd33 172.16.101.54:6379 myself,master - 0 0 1 connected 0-5460

28e8d5c8fd12533622d8110f8d262cb50120ca02 172.16.101.58:6379 slave 610aa83831404be545b25cc7f7322e987da1dd33 0 1532497135872 4 connected

4c7348ac575a1a3bd4357e937a8f94f1654193cf 172.16.101.55:6379 master - 0 1532497133867 2 connected 5461-10922

ece3a80f6989fa7f8316d73ee30ceea84340f24e 172.16.101.56:6379 master - 0 1532497134870 3 connected 10923-16383

c79de1092011ed395ee772969274c0567c2b5dd1 172.16.101.60:6379 slave ece3a80f6989fa7f8316d73ee30ceea84340f24e 0 1532497137878 6 connected

[root@sht-sgmhadoopcm-01 redis]# cat nodes-6379.conf

b50ba506525ee547823b3d9a7e5a095933e3ff42 172.16.101.59:6379 slave 4c7348ac575a1a3bd4357e937a8f94f1654193cf 0 1532493960094 5 connected

610aa83831404be545b25cc7f7322e987da1dd33 172.16.101.54:6379 myself,master - 0 0 1 connected 0-5460

28e8d5c8fd12533622d8110f8d262cb50120ca02 172.16.101.58:6379 slave 610aa83831404be545b25cc7f7322e987da1dd33 0 1532493955077 4 connected

4c7348ac575a1a3bd4357e937a8f94f1654193cf 172.16.101.55:6379 master - 0 1532493958087 2 connected 5461-10922

ece3a80f6989fa7f8316d73ee30ceea84340f24e 172.16.101.56:6379 master - 0 1532493956080 3 connected 10923-16383

c79de1092011ed395ee772969274c0567c2b5dd1 172.16.101.60:6379 slave ece3a80f6989fa7f8316d73ee30ceea84340f24e 0 1532493959091 6 connected

vars currentEpoch 6 lastVoteEpoch 0

FAQ

Error1: ruby2.0.0版本太低

[root@sht-sgmhadoopcm-01 redis]# gem install redis

Fetching: redis-4.0.1.gem (100%)

ERROR: Error installing redis:

redis requires Ruby version >= 2.2.2.

解決方法：

[root@sht-sgmhadoopcm-01 redis]# ruby --version

ruby 2.0.0p648 (2015-12-16) [x86_64-linux]

[root@sht-sgmhadoopcm-01 redis]# gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

[root@sht-sgmhadoopcm-01 redis]# curl -sSLhttps://get.rvm.io | bash -s stable

[root@sht-sgmhadoopcm-01 redis]# find / -name rvm -print

/usr/local/rvm

/usr/local/rvm/src/rvm

/usr/local/rvm/src/rvm/bin/rvm

/usr/local/rvm/src/rvm/lib/rvm

/usr/local/rvm/src/rvm/scripts/rvm

/usr/local/rvm/bin/rvm

/usr/local/rvm/lib/rvm

/usr/local/rvm/scripts/rvm

[root@sht-sgmhadoopcm-01 redis]# source /usr/local/rvm/scripts/rvm

[root@sht-sgmhadoopcm-01 redis]# rvm list known

[root@sht-sgmhadoopcm-01 redis]# rvm install 2.4.1

[root@sht-sgmhadoopcm-01 redis]# rvm use 2.4.1

Using /usr/local/rvm/gems/ruby-2.4.1

[root@sht-sgmhadoopcm-01 redis]# rvm use 2.4.1 --default

Using /usr/local/rvm/gems/ruby-2.4.1

[root@sht-sgmhadoopcm-01 redis]# rvm remove 2.3.4

ruby-2.3.4 - #already gone

Using /usr/local/rvm/gems/ruby-2.4.1

[root@sht-sgmhadoopcm-01 redis]# ruby --version

ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

Error2：添加節點數據不為空

[root@sht-sgmhadoopcm-01 redis]# src/redis-trib.rb add-node 172.16.101.66:6379 172.16.101.54:6379

......

[ERR] Node 172.16.101.66:6379 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

解決方法：

(1)刪除aof和rdb文件

(2)刪除nodes-6379.conf配置文件

(3)執行flushdb and flushall

參考鏈接

https://redis.io/topics/cluster-tutorial

Redis Cluster--安裝配置