1. 程式人生 > 實用技巧 >Redis叢集搭建

Redis叢集搭建

目錄

一、Redis Cluster 分散式叢集

1.什麼是Redis Cluster

1.Redis叢集是一個可以在多個Redis節點之間進行資料共享的設施(installation)
2.Redis叢集不支援那些需要同時處理多個鍵的Redis命令,因為執行這些命令需要在多個Redis節點之間移動資料,並且在高負載的情況下,這些命令將降低Redis叢集的效能,並導致不可預測的行為。(使用ack協議)
3.Redis叢集通過分割槽(partition)來提供一定程度的可用性(availability):即使叢集中有一部分節點失效或者無法進行通訊,叢集也可以繼續處理命令請求。
4.Redis叢集有將資料自動切分(split)到多個節點的能力。

2.Redis Cluster的特點

#高效能:
1.在多分片節點中,將16384個槽位,均勻分佈到多個分片節點中
2.存資料時,將key做crc16(key),然後和16384進行取模,得出槽位值(0-16384之間)
3.根據計算得出的槽位值,找到相對應的分片節點的主節點,儲存到相應槽位上
4.如果客戶端當時連線的節點不是將來要儲存的分片節點,分片叢集會將客戶端連線切換至真正儲存節點進行資料儲存
5.客戶端與redis節點直連,不需要中間proxy層.客戶端不需要連線叢集所有節點,連線叢集中任何一個可用節點即可
6.Redis Cluster解決了redis資源利用率的問題

#高可用
7.在搭建叢集時,會為每一個分片的主節點,對應一個從節點,實現slaveof功能,同時當主節點down,實現類似於sentinel的自動failover的功能。

3.槽的概念

1.在叢集中,會把所有節點分為16384個槽位
2.槽位的序號是 0 - 16383,序號不重要,數量才重要
3.每一個槽位分配到資料的概率是一樣

4.redis故障轉移

1.在叢集裡面,節點會對其他節點進行下線檢測。
2.當一個主節點下線時,叢集裡面的其他主節點負責對下線主節點進行故障移。
3.換句話說,叢集的節點集成了下線檢測和故障轉移等類似 Sentinel 的功能。

二、redis叢集搭建(方法一)

須知

# 如下只是叢集關聯的一種方式,還有一種方式需要安裝相應工具,但那種方式比較簡單,但會自動進行主從,缺點也存在,會自動進行主從,如果不是想要的主從狀態,將要自己手動修改。(詳情見方法二)

1.環境準備

節點 IP
節點1 172.16.1.51 6379,6380
節點2 172.16.1.52 6379,6380
節點3 172.16.1.53 6379,6380

2.搭建redis

#刪除以前的redis資料
[root@db01 ~]# rm -rf /service/redis/*

#建立多例項目錄
[root@db01 ~]# mkdir /service/redis/{6379,6380}
[root@db02 ~]# mkdir /service/redis/{6379,6380}
[root@db03 ~]# mkdir /service/redis/{6379,6380}

#配置所有redis
[root@db01 ~]# vim /service/redis/6379/redis.conf
bind 172.16.1.51 127.0.0.1
port 6379
daemonize yes
pidfile /service/redis/6379/redis.pid
loglevel notice
logfile /service/redis/6379/redis.log
dbfilename dump.rdb
dir /service/redis/6379
cluster-enabled yes						# 開啟redis叢集
cluster-config-file nodes.conf			                # 叢集關聯檔案地址
cluster-node-timeout 5000				        # 節點互聯超時閾值

[root@db01 ~]# vim /service/redis/6380/redis.conf
bind 172.16.1.51 127.0.0.1
port 6380
daemonize yes
pidfile /service/redis/6380/redis.pid
loglevel notice
logfile /service/redis/6380/redis.log
dbfilename dump.rdb
dir /service/redis/6380
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000

3.啟動所有redis

[root@db01 ~]# redis-server /service/redis/6379/redis.conf 
[root@db01 ~]# redis-server /service/redis/6380/redis.conf
[root@db02 ~]# redis-server /service/redis/6379/redis.conf 
[root@db02 ~]# redis-server /service/redis/6380/redis.conf
[root@db03 ~]# redis-server /service/redis/6379/redis.conf 
[root@db03 ~]# redis-server /service/redis/6380/redis.conf

4.關聯所有redis節點

1)登入所有節點

[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6379
[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6380
[root@db02 ~]# redis-cli -h 172.16.1.52 -p 6379
[root@db02 ~]# redis-cli -h 172.16.1.52 -p 6380
[root@db03 ~]# redis-cli -h 172.16.1.53 -p 6379
[root@db03 ~]# redis-cli -h 172.16.1.53 -p 6380

2)檢視叢集節點

#檢視叢集節 79 myself,master - 0 0 0 connected

3)關聯所有節點

# 將需要加入叢集中的主從伺服器全部進行關聯
172.16.1.51:6379> CLUSTER MEET 172.16.1.51 6380
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.52 6379
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.52 6380
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.53 6379
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.53 6380
OK

#檢視叢集狀態,所有節點
172.16.1.51:6379> CLUSTER NODES
aee9f4e6e09a452fd44bca7639be442b5138f141 172.16.1.52:6380 master - 0 1596687131655 4 connected
777412c8d6554e3390e1083bf1f55002be08cf62 172.16.1.51:6380 master - 0 1596687131352 2 connected
ef18ab5bab6d8bc06917a0cf2dc9bffa8b431087 172.16.1.52:6379 master - 0 1596687132362 3 connected
f2747c92813ea06b25c3e9c8d5232b46b3cf9d3d 172.16.1.53:6379 master - 0 1596687131856 0 connected
25f735f08ac62b2f758c1e2c21e178cc46279087 172.16.1.53:6380 master - 0 1596687131251 5 connected
28faba09f4c0ec8cdb90d92e09636796427b7143 172.16.1.51:6379 myself,master - 0 0 1 connected

5.分配槽位

#檢視叢集狀態
172.16.1.51:6379> CLUSTER INFO
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:0
cluster_current_epoch:5
cluster_my_epoch:1
cluster_stats_messages_sent:1168
cluster_stats_messages_received:1168

#槽位規劃
db01:     5462 個槽位  (0-5461)
db02:     5461 個槽位  (5462-10922)
db03:	  5461 個槽位  (10923-16383)

#分配槽位
[root@db01 ~]# redis-cli -p 6379 -h 172.16.1.51 CLUSTER ADDSLOTS {0..5461}
OK
[root@db02 ~]# redis-cli -p 6379 -h 172.16.1.52 CLUSTER ADDSLOTS {5462..10922}
OK
[root@db02 ~]# redis-cli -p 6379 -h 172.16.1.53 CLUSTER ADDSLOTS {10923..16383}

6.插入資料測試叢集

#插入一條資料
172.16.1.51:6379> set k1 v1
(error) MOVED 12706 172.16.1.53:6379
#報錯,該key的值只能插入到12706這個槽位

[root@db03 ~]# redis-cli -h 172.16.1.53
172.16.1.53:6379> set k1 v1
OK

#ASK協議,自動切換將資料新增到指定槽位(加一個-c引數即可)
[root@db03 ~]# redis-cli -h 172.16.1.53
172.16.1.53:6379> set k2 v2
(error) MOVED 449 172.16.1.51:6379
172.16.1.53:6379> quit
[root@db03 ~]# redis-cli -c -h 172.16.1.53
172.16.1.53:6379> set k2 v2
-> Redirected to slot [449] located at 172.16.1.51:6379
OK

#指令碼插入資料測試
[root@db03 ~]# vim data.sh 
#!/bin/bash
for i in {1..1000};do
    redis-cli -c -p 6379 -h 172.16.1.51 set k${i} v${i}
done

#檢視資料分配
172.16.1.51:6379> DBSIZE
(integer) 341
172.16.1.52:6379> DBSIZE
(integer) 332
172.16.1.53:6379> DBSIZE
(integer) 327

7.新增副本節點

1)檢視節點

172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 master - 0 1596763193422 4 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763192412 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 master - 0 1596763192512 3 connected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master - 0 1596763191908 0 connected 10923-16383
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596763193925 2 connected

2)配置主從

#db01的6380做db02的6379的從庫
172.16.1.51:6380> CLUSTER REPLICATE 5eb9e5356534ff4acda736d13f0dc9fc3d40049b
OK

#db02的6380做db03的6379的從庫
172.16.1.52:6380> CLUSTER REPLICATE acc3a4d0e6e43fc74630c1f0714865fdcbdaf677
OK

#db03的6380做db01的6379的從庫
172.16.1.53:6380> CLUSTER REPLICATE 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a
OK

3)再次檢視節點資訊

172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596763362696 5 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763363202 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 0 1596763362192 3 connected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master - 0 1596763363203 0 connected 10923-16383
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 slave acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 0 1596763364211 2 connected

8.故障演示

#停掉一臺節點
[root@db03 ~]# reboot

#到另一臺機器檢視叢集狀態,發現叢集是正常的
172.16.1.51:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_sent:327031
cluster_stats_messages_received:326973

#檢視節點資訊,副本被提升為主庫
172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596763771309 5 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763771310 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave,fail 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 1596763736458 1596763734245 3 disconnected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master,fail - 1596763736458 1596763735246 0 disconnected
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596763772319 6 connected 10923-16383

9.節點恢復

#修復機器
[root@db03 ~]# redis-server /service/redis/6379/redis.conf 
[root@db03 ~]# redis-server /service/redis/6380/redis.conf

#再次檢視節點資訊
172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596764061287 5 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596764060781 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 0 1596764059770 3 connected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 slave 381b54584572e8013becdae2eeaff48bf6eb5450 0 1596764062094 6 connected
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596764061789 6 connected 10923-16383

#原主節點修復後變為從節點

三、使用工具搭建redis叢集(方法二)

1.環境準備

節點 IP
節點1 172.16.1.51 6379,6380
節點2 172.16.1.52 6379,6380
節點3 172.16.1.53 6379,6380

2.搭建redis

3.啟動所有redis

4.安裝叢集外掛

#EPEL源安裝ruby支援
[root@db01 ~]# yum install ruby rubygems -y

#檢視gem源
[root@db01 ~]# gem sources -l
*** CURRENT SOURCES ***

http://rubygems.org/

#新增阿里雲的gem源
[root@db01 ~]# gem sources -a http://mirrors.aliyun.com/rubygems/
http://mirrors.aliyun.com/rubygems/ added to sources 

#刪除國外gem源
[root@db01 ~]# gem sources  --remove https://rubygems.org/
http://rubygems.org/ removed from sources

#再次檢視gem源
[root@db01 ~]# gem sources -l

#使用gem安裝redis的ruby外掛
[root@db01 ~]# gem install redis -v 3.3.3
Successfully installed redis-3.3.3
1 gem installed
Installing ri documentation for redis-3.3.3...
Installing RDoc documentation for redis-3.3.3...
  1. redis-trib.rb命令
[root@db01 ~]# redis-trib.rb 
create			#建立一個叢集
check			#檢查叢集
info			#叢集狀態
fix		        #修復叢集
reshard			#重新分配槽位
rebalance		#平衡槽位數量
add-node		#新增節點
del-node		#刪除節點
set-timeout		#設定超時時間
call			#向叢集所有機器輸入命令
import			#匯入資料
help                    #幫助

6.關聯所有節點

[root@db01 ~]# redis-trib.rb create --replicas 1 172.16.1.51:6379 172.16.1.52:6379 172.16.1.53:6379 172.16.1.52:6380 172.16.1.53:6380 172.16.1.51:6380 
# --replicas引數指定叢集中每個主節點配備幾個從節點,這裡設定為1。
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.16.1.51:6379
172.16.1.52:6379
172.16.1.53:6379
Adding replica 172.16.1.52:6380 to 172.16.1.51:6379
Adding replica 172.16.1.51:6380 to 172.16.1.52:6379
Adding replica 172.16.1.53:6380 to 172.16.1.53:6379
M: 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379
   slots:0-5460 (5461 slots) master
M: 7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379
   slots:5461-10922 (5462 slots) master
M: d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379
   slots:10923-16383 (5461 slots) master
S: fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380
   replicates 5ad7bd957133eac9c3a692b35f8ae72258cf0ece
S: e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380
   replicates d27553035a3e91c78d375208c72b756e9b2523d4
S: 1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380
   replicates 7c79559b280db9d9c182f3a25c718efe9e934fc7
Can I set the above configuration? (type 'yes' to accept): yes   
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 172.16.1.51:6379)
M: 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380
   slots: (0 slots) slave
   replicates d27553035a3e91c78d375208c72b756e9b2523d4
M: d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380
   slots: (0 slots) slave
   replicates 5ad7bd957133eac9c3a692b35f8ae72258cf0ece
S: 1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380
   slots: (0 slots) slave
   replicates 7c79559b280db9d9c182f3a25c718efe9e934fc7
M: 7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

7.檢視叢集狀態

[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6379 CLUSTER NODES
e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380 slave d27553035a3e91c78d375208c72b756e9b2523d4 0 1596767315453 5 connected
d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379 master - 0 1596767315453 3 connected 10923-16383
5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5460
fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380 slave 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 0 1596767313429 4 connected
1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380 slave 7c79559b280db9d9c182f3a25c718efe9e934fc7 0 1596767313935 6 connected
7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379 master - 0 1596767314949 2 connected 5461-10922

8.重新做主從

#由於使用工具,始終有一臺機器為主庫本機的從庫,所以要重新分配主從
172.16.1.52:6380> CLUSTER REPLICATE d27553035a3e91c78d375208c72b756e9b2523d4
OK
172.16.1.53:6380> CLUSTER REPLICATE 5ad7bd957133eac9c3a692b35f8ae72258cf0ece
OK

9.插入資料測試

[root@db01 ~]# redis-trib.rb info 172.16.1.52:6379
172.16.1.52:6379 (7c79559b...) -> 332 keys | 5462 slots | 1 slaves.
172.16.1.51:6379 (5ad7bd95...) -> 341 keys | 5461 slots | 1 slaves.
172.16.1.53:6379 (d2755303...) -> 327 keys | 5461 slots | 1 slaves.
[OK] 1000 keys in 3 masters.
0.06 keys per slot on average.

[root@db01 ~]# redis-trib.rb info 172.16.1.52:6379
172.16.1.52:6379 (7c79559b...) -> 661 keys | 5462 slots | 1 slaves.
172.16.1.51:6379 (5ad7bd95...) -> 674 keys | 5461 slots | 1 slaves.
172.16.1.53:6379 (d2755303...) -> 665 keys | 5461 slots | 1 slaves.
[OK] 2000 keys in 3 masters.
0.12 keys per slot on average.