Redis叢集搭建
阿新 • • 發佈:2020-08-07
目錄
一、Redis Cluster 分散式叢集
1.什麼是Redis Cluster
1.Redis叢集是一個可以在多個Redis節點之間進行資料共享的設施(installation) 2.Redis叢集不支援那些需要同時處理多個鍵的Redis命令,因為執行這些命令需要在多個Redis節點之間移動資料,並且在高負載的情況下,這些命令將降低Redis叢集的效能,並導致不可預測的行為。(使用ack協議) 3.Redis叢集通過分割槽(partition)來提供一定程度的可用性(availability):即使叢集中有一部分節點失效或者無法進行通訊,叢集也可以繼續處理命令請求。 4.Redis叢集有將資料自動切分(split)到多個節點的能力。
2.Redis Cluster的特點
#高效能: 1.在多分片節點中,將16384個槽位,均勻分佈到多個分片節點中 2.存資料時,將key做crc16(key),然後和16384進行取模,得出槽位值(0-16384之間) 3.根據計算得出的槽位值,找到相對應的分片節點的主節點,儲存到相應槽位上 4.如果客戶端當時連線的節點不是將來要儲存的分片節點,分片叢集會將客戶端連線切換至真正儲存節點進行資料儲存 5.客戶端與redis節點直連,不需要中間proxy層.客戶端不需要連線叢集所有節點,連線叢集中任何一個可用節點即可 6.Redis Cluster解決了redis資源利用率的問題 #高可用 7.在搭建叢集時,會為每一個分片的主節點,對應一個從節點,實現slaveof功能,同時當主節點down,實現類似於sentinel的自動failover的功能。
3.槽的概念
1.在叢集中,會把所有節點分為16384個槽位
2.槽位的序號是 0 - 16383,序號不重要,數量才重要
3.每一個槽位分配到資料的概率是一樣
4.redis故障轉移
1.在叢集裡面,節點會對其他節點進行下線檢測。
2.當一個主節點下線時,叢集裡面的其他主節點負責對下線主節點進行故障移。
3.換句話說,叢集的節點集成了下線檢測和故障轉移等類似 Sentinel 的功能。
二、redis叢集搭建(方法一)
須知
# 如下只是叢集關聯的一種方式,還有一種方式需要安裝相應工具,但那種方式比較簡單,但會自動進行主從,缺點也存在,會自動進行主從,如果不是想要的主從狀態,將要自己手動修改。(詳情見方法二)
1.環境準備
節點 | IP | 埠 |
---|---|---|
節點1 | 172.16.1.51 | 6379,6380 |
節點2 | 172.16.1.52 | 6379,6380 |
節點3 | 172.16.1.53 | 6379,6380 |
2.搭建redis
#刪除以前的redis資料
[root@db01 ~]# rm -rf /service/redis/*
#建立多例項目錄
[root@db01 ~]# mkdir /service/redis/{6379,6380}
[root@db02 ~]# mkdir /service/redis/{6379,6380}
[root@db03 ~]# mkdir /service/redis/{6379,6380}
#配置所有redis
[root@db01 ~]# vim /service/redis/6379/redis.conf
bind 172.16.1.51 127.0.0.1
port 6379
daemonize yes
pidfile /service/redis/6379/redis.pid
loglevel notice
logfile /service/redis/6379/redis.log
dbfilename dump.rdb
dir /service/redis/6379
cluster-enabled yes # 開啟redis叢集
cluster-config-file nodes.conf # 叢集關聯檔案地址
cluster-node-timeout 5000 # 節點互聯超時閾值
[root@db01 ~]# vim /service/redis/6380/redis.conf
bind 172.16.1.51 127.0.0.1
port 6380
daemonize yes
pidfile /service/redis/6380/redis.pid
loglevel notice
logfile /service/redis/6380/redis.log
dbfilename dump.rdb
dir /service/redis/6380
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
3.啟動所有redis
[root@db01 ~]# redis-server /service/redis/6379/redis.conf
[root@db01 ~]# redis-server /service/redis/6380/redis.conf
[root@db02 ~]# redis-server /service/redis/6379/redis.conf
[root@db02 ~]# redis-server /service/redis/6380/redis.conf
[root@db03 ~]# redis-server /service/redis/6379/redis.conf
[root@db03 ~]# redis-server /service/redis/6380/redis.conf
4.關聯所有redis節點
1)登入所有節點
[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6379
[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6380
[root@db02 ~]# redis-cli -h 172.16.1.52 -p 6379
[root@db02 ~]# redis-cli -h 172.16.1.52 -p 6380
[root@db03 ~]# redis-cli -h 172.16.1.53 -p 6379
[root@db03 ~]# redis-cli -h 172.16.1.53 -p 6380
2)檢視叢集節點
#檢視叢集節 79 myself,master - 0 0 0 connected
3)關聯所有節點
# 將需要加入叢集中的主從伺服器全部進行關聯
172.16.1.51:6379> CLUSTER MEET 172.16.1.51 6380
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.52 6379
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.52 6380
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.53 6379
OK
172.16.1.51:6379> CLUSTER MEET 172.16.1.53 6380
OK
#檢視叢集狀態,所有節點
172.16.1.51:6379> CLUSTER NODES
aee9f4e6e09a452fd44bca7639be442b5138f141 172.16.1.52:6380 master - 0 1596687131655 4 connected
777412c8d6554e3390e1083bf1f55002be08cf62 172.16.1.51:6380 master - 0 1596687131352 2 connected
ef18ab5bab6d8bc06917a0cf2dc9bffa8b431087 172.16.1.52:6379 master - 0 1596687132362 3 connected
f2747c92813ea06b25c3e9c8d5232b46b3cf9d3d 172.16.1.53:6379 master - 0 1596687131856 0 connected
25f735f08ac62b2f758c1e2c21e178cc46279087 172.16.1.53:6380 master - 0 1596687131251 5 connected
28faba09f4c0ec8cdb90d92e09636796427b7143 172.16.1.51:6379 myself,master - 0 0 1 connected
5.分配槽位
#檢視叢集狀態
172.16.1.51:6379> CLUSTER INFO
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:0
cluster_current_epoch:5
cluster_my_epoch:1
cluster_stats_messages_sent:1168
cluster_stats_messages_received:1168
#槽位規劃
db01: 5462 個槽位 (0-5461)
db02: 5461 個槽位 (5462-10922)
db03: 5461 個槽位 (10923-16383)
#分配槽位
[root@db01 ~]# redis-cli -p 6379 -h 172.16.1.51 CLUSTER ADDSLOTS {0..5461}
OK
[root@db02 ~]# redis-cli -p 6379 -h 172.16.1.52 CLUSTER ADDSLOTS {5462..10922}
OK
[root@db02 ~]# redis-cli -p 6379 -h 172.16.1.53 CLUSTER ADDSLOTS {10923..16383}
6.插入資料測試叢集
#插入一條資料
172.16.1.51:6379> set k1 v1
(error) MOVED 12706 172.16.1.53:6379
#報錯,該key的值只能插入到12706這個槽位
[root@db03 ~]# redis-cli -h 172.16.1.53
172.16.1.53:6379> set k1 v1
OK
#ASK協議,自動切換將資料新增到指定槽位(加一個-c引數即可)
[root@db03 ~]# redis-cli -h 172.16.1.53
172.16.1.53:6379> set k2 v2
(error) MOVED 449 172.16.1.51:6379
172.16.1.53:6379> quit
[root@db03 ~]# redis-cli -c -h 172.16.1.53
172.16.1.53:6379> set k2 v2
-> Redirected to slot [449] located at 172.16.1.51:6379
OK
#指令碼插入資料測試
[root@db03 ~]# vim data.sh
#!/bin/bash
for i in {1..1000};do
redis-cli -c -p 6379 -h 172.16.1.51 set k${i} v${i}
done
#檢視資料分配
172.16.1.51:6379> DBSIZE
(integer) 341
172.16.1.52:6379> DBSIZE
(integer) 332
172.16.1.53:6379> DBSIZE
(integer) 327
7.新增副本節點
1)檢視節點
172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 master - 0 1596763193422 4 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763192412 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 master - 0 1596763192512 3 connected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master - 0 1596763191908 0 connected 10923-16383
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596763193925 2 connected
2)配置主從
#db01的6380做db02的6379的從庫
172.16.1.51:6380> CLUSTER REPLICATE 5eb9e5356534ff4acda736d13f0dc9fc3d40049b
OK
#db02的6380做db03的6379的從庫
172.16.1.52:6380> CLUSTER REPLICATE acc3a4d0e6e43fc74630c1f0714865fdcbdaf677
OK
#db03的6380做db01的6379的從庫
172.16.1.53:6380> CLUSTER REPLICATE 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a
OK
3)再次檢視節點資訊
172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596763362696 5 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763363202 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 0 1596763362192 3 connected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master - 0 1596763363203 0 connected 10923-16383
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 slave acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 0 1596763364211 2 connected
8.故障演示
#停掉一臺節點
[root@db03 ~]# reboot
#到另一臺機器檢視叢集狀態,發現叢集是正常的
172.16.1.51:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_sent:327031
cluster_stats_messages_received:326973
#檢視節點資訊,副本被提升為主庫
172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596763771309 5 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763771310 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave,fail 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 1596763736458 1596763734245 3 disconnected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master,fail - 1596763736458 1596763735246 0 disconnected
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596763772319 6 connected 10923-16383
9.節點恢復
#修復機器
[root@db03 ~]# redis-server /service/redis/6379/redis.conf
[root@db03 ~]# redis-server /service/redis/6380/redis.conf
#再次檢視節點資訊
172.16.1.51:6379> CLUSTER NODES
5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596764061287 5 connected
5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596764060781 5 connected 5462-10922
50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 0 1596764059770 3 connected
acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 slave 381b54584572e8013becdae2eeaff48bf6eb5450 0 1596764062094 6 connected
2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461
381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596764061789 6 connected 10923-16383
#原主節點修復後變為從節點
三、使用工具搭建redis叢集(方法二)
1.環境準備
節點 | IP | 埠 |
---|---|---|
節點1 | 172.16.1.51 | 6379,6380 |
節點2 | 172.16.1.52 | 6379,6380 |
節點3 | 172.16.1.53 | 6379,6380 |
2.搭建redis
3.啟動所有redis
4.安裝叢集外掛
#EPEL源安裝ruby支援
[root@db01 ~]# yum install ruby rubygems -y
#檢視gem源
[root@db01 ~]# gem sources -l
*** CURRENT SOURCES ***
http://rubygems.org/
#新增阿里雲的gem源
[root@db01 ~]# gem sources -a http://mirrors.aliyun.com/rubygems/
http://mirrors.aliyun.com/rubygems/ added to sources
#刪除國外gem源
[root@db01 ~]# gem sources --remove https://rubygems.org/
http://rubygems.org/ removed from sources
#再次檢視gem源
[root@db01 ~]# gem sources -l
#使用gem安裝redis的ruby外掛
[root@db01 ~]# gem install redis -v 3.3.3
Successfully installed redis-3.3.3
1 gem installed
Installing ri documentation for redis-3.3.3...
Installing RDoc documentation for redis-3.3.3...
- redis-trib.rb命令
[root@db01 ~]# redis-trib.rb
create #建立一個叢集
check #檢查叢集
info #叢集狀態
fix #修復叢集
reshard #重新分配槽位
rebalance #平衡槽位數量
add-node #新增節點
del-node #刪除節點
set-timeout #設定超時時間
call #向叢集所有機器輸入命令
import #匯入資料
help #幫助
6.關聯所有節點
[root@db01 ~]# redis-trib.rb create --replicas 1 172.16.1.51:6379 172.16.1.52:6379 172.16.1.53:6379 172.16.1.52:6380 172.16.1.53:6380 172.16.1.51:6380
# --replicas引數指定叢集中每個主節點配備幾個從節點,這裡設定為1。
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.16.1.51:6379
172.16.1.52:6379
172.16.1.53:6379
Adding replica 172.16.1.52:6380 to 172.16.1.51:6379
Adding replica 172.16.1.51:6380 to 172.16.1.52:6379
Adding replica 172.16.1.53:6380 to 172.16.1.53:6379
M: 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379
slots:0-5460 (5461 slots) master
M: 7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379
slots:5461-10922 (5462 slots) master
M: d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379
slots:10923-16383 (5461 slots) master
S: fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380
replicates 5ad7bd957133eac9c3a692b35f8ae72258cf0ece
S: e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380
replicates d27553035a3e91c78d375208c72b756e9b2523d4
S: 1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380
replicates 7c79559b280db9d9c182f3a25c718efe9e934fc7
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 172.16.1.51:6379)
M: 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380
slots: (0 slots) slave
replicates d27553035a3e91c78d375208c72b756e9b2523d4
M: d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380
slots: (0 slots) slave
replicates 5ad7bd957133eac9c3a692b35f8ae72258cf0ece
S: 1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380
slots: (0 slots) slave
replicates 7c79559b280db9d9c182f3a25c718efe9e934fc7
M: 7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
7.檢視叢集狀態
[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6379 CLUSTER NODES
e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380 slave d27553035a3e91c78d375208c72b756e9b2523d4 0 1596767315453 5 connected
d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379 master - 0 1596767315453 3 connected 10923-16383
5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5460
fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380 slave 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 0 1596767313429 4 connected
1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380 slave 7c79559b280db9d9c182f3a25c718efe9e934fc7 0 1596767313935 6 connected
7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379 master - 0 1596767314949 2 connected 5461-10922
8.重新做主從
#由於使用工具,始終有一臺機器為主庫本機的從庫,所以要重新分配主從
172.16.1.52:6380> CLUSTER REPLICATE d27553035a3e91c78d375208c72b756e9b2523d4
OK
172.16.1.53:6380> CLUSTER REPLICATE 5ad7bd957133eac9c3a692b35f8ae72258cf0ece
OK
9.插入資料測試
[root@db01 ~]# redis-trib.rb info 172.16.1.52:6379
172.16.1.52:6379 (7c79559b...) -> 332 keys | 5462 slots | 1 slaves.
172.16.1.51:6379 (5ad7bd95...) -> 341 keys | 5461 slots | 1 slaves.
172.16.1.53:6379 (d2755303...) -> 327 keys | 5461 slots | 1 slaves.
[OK] 1000 keys in 3 masters.
0.06 keys per slot on average.
[root@db01 ~]# redis-trib.rb info 172.16.1.52:6379
172.16.1.52:6379 (7c79559b...) -> 661 keys | 5462 slots | 1 slaves.
172.16.1.51:6379 (5ad7bd95...) -> 674 keys | 5461 slots | 1 slaves.
172.16.1.53:6379 (d2755303...) -> 665 keys | 5461 slots | 1 slaves.
[OK] 2000 keys in 3 masters.
0.12 keys per slot on average.