1. 程式人生 > >redis群集節點刪除和新增

redis群集節點刪除和新增

redis叢集介紹

多個redis節點實現網路互連,資料共享
所有節點都是一主一從(或一主多從),其中從不提供服務,僅作為備份備用,保證redis資料的安全性
不支援同時處理多個鍵,redis叢集寫入資料是向叢集裡隨機寫入的(比如raid5磁碟陣列格式),因為redis叢集需要把鍵平均分佈在各個節點上,併發量很高的情況下同時建立鍵值會降低效能並會導致不可預估的行為
redis叢集支援線上增加、刪除節點,在擴充redis的情況下,是非常容易擴容叢集服務數量的
客戶端可以連線任意一個主節點進行讀寫

redis群集節點刪除和新增

redis叢集配置

redis群集場景設定
使用兩臺機器,分別開啟三個redis服務(基於不同埠)
伺服器ip為192.168.1.223和192.168.1.234
192.168.1.234上開啟7000、7002、7002三個埠,設定角色全部為主
192.168.1.223上開啟7001、7003、7005三個埠,設定角色全部為從
兩臺機器都要編譯安裝redis,編輯三個不同的redis.conf,分別設定不同埠號、dir等引數,還需要增加cluster相關引數,啟動為六個redis服務
首先在192.168.1.234機器上拷貝三個不同的redis配置檔案,修改配置檔案的埠、儲存資料檔案的路徑等,並建立配置檔案中指定儲存資料檔案的目錄
在192.168.1.234機器上建立三個基於不同埠的配置檔案,如:

[[email protected] /]# cat /etc/redis_7000.conf 
port 7000
bind 192.168.1.234
daemonize yes
pidfile /var/run/redis_7000.pid
dir /data/redis_data/7000
cluster-enabled yes
cluster-config-file nodes_7000.conf
cluster-node-timeout 10100
appendonly yes
[[email protected] /]# cat /etc/redis_7002.conf 
port 7002
bind 192.168.1.234
daemonize yes
pidfile /var/run/redis_7002.pid
dir /data/redis_data/7002
cluster-enabled yes
cluster-config-file nodes_7002.conf
cluster-node-timeout 10100
appendonly yes
[
[email protected]
/]# cat /etc/redis_7004.conf port 7004 bind 192.168.1.234 daemonize yes pidfile /var/run/redis_7004.pid dir /data/redis_data/7004 cluster-enabled yes cluster-config-file nodes_7004.conf cluster-node-timeout 10100 appendonly yes

啟動這三個基於不同埠的redis服務,並檢視其啟動埠資訊

[[email protected] /]# /usr/local/bin/redis-server /etc/redis_7000.conf 
91981:C 14 Nov 2018 18:31:38.224 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
91981:C 14 Nov 2018 18:31:38.224 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=91981, just started
91981:C 14 Nov 2018 18:31:38.224 # Configuration loaded
[
[email protected]
/]# /usr/local/bin/redis-server /etc/redis_7002.conf 91986:C 14 Nov 2018 18:31:46.752 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 91986:C 14 Nov 2018 18:31:46.752 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=91986, just started 91986:C 14 Nov 2018 18:31:46.752 # Configuration loaded [[email protected] /]# /usr/local/bin/redis-server /etc/redis_7004.conf 91991:C 14 Nov 2018 18:31:50.830 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 91991:C 14 Nov 2018 18:31:50.830 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=91991, just started 91991:C 14 Nov 2018 18:31:50.830 # Configuration loaded [[email protected] /]# netstat -ntlp |grep redis tcp        0      0 192.168.1.234:7000      0.0.0.0:*               LISTEN      91982/redis-server   tcp        0      0 192.168.1.234:7002      0.0.0.0:*               LISTEN      91987/redis-server   tcp        0      0 192.168.1.234:7004      0.0.0.0:*               LISTEN      91992/redis-server   tcp        0      0 192.168.1.234:17000     0.0.0.0:*               LISTEN      91982/redis-server   tcp        0      0 192.168.1.234:17002     0.0.0.0:*               LISTEN      91987/redis-server   tcp        0      0 192.168.1.234:17004     0.0.0.0:*               LISTEN      91992/redis-server 192.168.1.223的redis也是按照這種方式啟動多個redis服務的 [[email protected] /]# /usr/local/bin/redis-server /etc/redis_7001.conf 26608:C 15 Nov 2018 00:00:10.297 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 26608:C 15 Nov 2018 00:00:10.297 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=26608, just started 26608:C 15 Nov 2018 00:00:10.297 # Configuration loaded [[email protected] /]# /usr/local/bin/redis-server /etc/redis_7003.conf 26622:C 15 Nov 2018 00:00:22.984 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 26622:C 15 Nov 2018 00:00:22.985 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=26622, just started 26622:C 15 Nov 2018 00:00:22.985 # Configuration loaded [[email protected] /]# /usr/local/bin/redis-server /etc/redis_7005.conf 26627:C 15 Nov 2018 00:00:27.128 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 26627:C 15 Nov 2018 00:00:27.128 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=26627, just started 26627:C 15 Nov 2018 00:00:27.128 # Configuration loaded [[email protected] /]# netstat -antp |grep redis tcp        0      0 192.168.1.200:7003          0.0.0.0:*                   LISTEN      26623/redis-server   tcp        0      0 192.168.1.200:7005          0.0.0.0:*                   LISTEN      26628/redis-server   tcp        0      0 192.168.1.200:17001         0.0.0.0:*                   LISTEN      26609/redis-server   tcp        0      0 192.168.1.200:17003         0.0.0.0:*                   LISTEN      26623/redis-server   tcp        0      0 192.168.1.200:17005         0.0.0.0:*                   LISTEN      26628/redis-server   tcp        0      0 192.168.1.200:7001          0.0.0.0:*                   LISTEN      26609/redis-server  

啟動多個redis服務後接下來安裝ruby中介軟體實現群集功能

redis群集安裝ruby

安裝ruby只需要在一臺伺服器上進行安裝執行
安裝Development的軟體組,安裝過程省略

[[email protected] redis_data]# yum -y groupinstall "Development Tools"

安裝需要支援的庫,安裝過程省略

[[email protected] redis_data]# yum install gdbm-devel linbdb4-devel libffi-devel libyaml libyaml-devel ncurses-devel openssl_devel readline-devel tcl-deve

建立編譯rpm包的目錄,下載ruby的原始碼包檔案,將原始碼包下載只剛剛建立的目錄下
如果下載有rpm編譯器,則執行編譯器來生成rpm包進行安裝,如果沒有就使用./configure編譯安裝

[[email protected] ~]# mkdir -p /root/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
[[email protected] ~]# rpmbuild -bb /root/rpmbuild/SPECS/ruby22x.spec
[[email protected] ~]# yum -y localinstall rpmbuild/RPMS/x86_64/ruby-2.4.5-1.e17.centos.x86_64.rpm

由於這裡我無法找到ruby的rpm編譯器,所以無法編譯成rpm包。這裡依舊使用解壓原始碼包,使用./configure來編譯安裝

[[email protected] ruby-2.4.5]# ./configure
[[email protected] ruby-2.4.5]# make 
-----------------------------編譯內容
  Files: 890
  Classes: 1312 ( 549 undocumented)
  Modules: 275 ( 110 undocumented)
  Constants: 2143 ( 538 undocumented)
  Attributes: 1053 ( 247 undocumented)
  Methods: 9837 (2106 undocumented)
  Total: 14620 (3550 undocumented)
   75.72% documented
  Elapsed: 86.3s
[[email protected] ruby-2.4.5]# make install
-----------------------------編譯內容
installing bundle gems:       /usr/local/lib/ruby/gems/2.4.0 (build_info, cache, doc, extensions, gems, specifications)
                              minitest 5.10.1
                              xmlrpc 0.2.1
                              test-unit 3.2.3
                              net-telnet 0.1.1
                              rake 12.0.0
                              did_you_mean 1.1.0
                              power_assert 0.4.1
installing rdoc:              /usr/local/share/ri/2.4.0/system
installing capi-docs:         /usr/local/share/doc/ruby

編譯安裝完成後可以檢視下ruby的版本號,驗證是否編譯安裝正確

[[email protected] ruby-2.4.5]# ruby -v
ruby 2.4.5p335 (2018-10-18 revision 65137) [x86_64-linux]
安裝redis的ruby叢集的相關命令
[[email protected] ruby-2.4.5]# gem install redis
Fetching: redis-4.0.3.gem (100%)
Successfully installed redis-4.0.3
Parsing documentation for redis-4.0.3
Installing ri documentation for redis-4.0.3
Done installing documentation for redis after 1 seconds
1 gem installed

拷貝redis的ruby命令到/usr/local/bin目錄下,新版本捨棄了redis-trib.rb的這個命令,如果安裝新版本,可以不執行此步操作

[[email protected] ruby-2.4.5]# cp /usr/local/src/redis-5.0.0/src/redis-trib.rb /usr/local/bin/

在新版本的redis中,使用redis-trib.rb來建立redis的群集會提示版本命令不可用,因為redis-trib.rb命令被捨棄不再使用。具體資訊如下:

WARNING: redis-trib.rb is not longer available!
You should use redis-cli instead.
All commands and features belonging to redis-trib.rb have been moved
to redis-cli.
In order to use them you should call redis-cli with the --cluster
option followed by the subcommand name, arguments and options.
Use the following syntax:
redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS]
Example:
redis-cli --cluster create 192.168.1.234:700 192.168.1.234:7002 192.168.1.234:7004 192.168.1.200:7001 192.168.1.200:7003 192.168.1.200:7005 --cluster-replicas 1
To get help about all subcommands, type:
redis-cli --cluster help

從上面提示可以瞭解到redis-trub.rb命令已經被redis-cli代替使用
那麼我們現在使用redis-cli來建立群集
格式為redis-cli --cluster create  ip:prot  ip:port ...... --cluster-replicas 1
這裡我建立的群集如下,在確認好群集資訊後,會提示你是否按照這個主從指定的方式儲存

[[email protected] ruby-2.4.5]# redis-cli --cluster create 192.168.1.234:7000 192.168.1.234:7002 192.168.1.234:7004 192.168.1.200:7001 192.168.1.200:7003 192.168.1.200:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.1.200:7003 to 192.168.1.234:7000                 這裡指定了主從關係:從redis-->主redis
Adding replica 192.168.1.234:7004 to 192.168.1.200:7001
Adding replica 192.168.1.200:7005 to 192.168.1.234:7002
M: 3af55accd31b6a057945c4b9e9378c7932d5f1f8 192.168.1.234:7000          輸出資訊中的M為master   S則是slave
   slots:[0-5460] (5461 slots) master
M: d8418ab100f8f32539fc60a7c8ead5927591f52a 192.168.1.234:7002         
   slots:[10923-16383] (5461 slots) master
S: 2a642a2e14d15ae8cb77b3ef04f58d32cfb09933 192.168.1.234:7004
   replicates 3e8c289a64638da55b0e812f924aaa242efbc315
M: 3e8c289a64638da55b0e812f924aaa242efbc315 192.168.1.200:7001
   slots:[5461-10922] (5462 slots) master
S: 9882ec7d971ec8b159ff126ac8e1ce480a6ba9e0 192.168.1.200:7003
   replicates 3af55accd31b6a057945c4b9e9378c7932d5f1f8
S: a3c7b8a393f2b4d3b9838753fb6496c3861bbb69 192.168.1.200:7005
   replicates d8418ab100f8f32539fc60a7c8ead5927591f52a
Can I set the above configuration? (type 'yes' to accept): yes

在上面確認互動的地方確認儲存後會輸出兩個ok資訊,可以判斷redis群集建立成功

Can I set the above configuration? (type 'yes' to accept): yes
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

redis建立叢集

redis叢集建立好後,登入redis時需要指定登入ip和埠,建立幾個key值檢視redis儲存的redis資訊
-c  登入redis叢集模式 
-h 指定叢集中的一個redis

[[email protected]1 ruby-2.4.5]# redis-cli -c -h 192.168.1.234 -p 7000
192.168.1.234:7000> set key1 123
-> Redirected to slot [9189] located at 192.168.1.200:7001
OK
192.168.1.200:7001> set key3 abc
-> Redirected to slot [935] located at 192.168.1.234:7000
OK
192.168.1.234:7000> set key2 ABC
OK
192.168.1.234:7000> set key4 ABC
-> Redirected to slot [13120] located at 192.168.1.234:7002
OK
192.168.1.234:7002> set key5 ABC
-> Redirected to slot [9057] located at 192.168.1.200:7001
OK
192.168.1.200:7001> set key6 ABC
-> Redirected to slot [4866] located at 192.168.1.234:7000
OK

檢視群集狀態資訊,redis-cli代替了redis-trib.rb,所以這裡使用新的命令格式。可以看出群集的redis資訊狀態

[[email protected] ruby-2.4.5]# redis-cli --cluster check 192.168.1.234:7000
192.168.1.234:7000 (3af55acc...) -> 4 keys | 5461 slots | 1 slaves.
192.168.1.234:7002 (d8418ab1...) -> 1 keys | 5461 slots | 1 slaves.
192.168.1.200:7001 (3e8c289a...) -> 2 keys | 5462 slots | 1 slaves.
[OK] 7 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.1.234:7000)
M: 3af55accd31b6a057945c4b9e9378c7932d5f1f8 192.168.1.234:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: a3c7b8a393f2b4d3b9838753fb6496c3861bbb69 192.168.1.200:7005
   slots: (0 slots) slave
   replicates d8418ab100f8f32539fc60a7c8ead5927591f52a
M: d8418ab100f8f32539fc60a7c8ead5927591f52a 192.168.1.234:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 3e8c289a64638da55b0e812f924aaa242efbc315 192.168.1.200:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 2a642a2e14d15ae8cb77b3ef04f58d32cfb09933 192.168.1.234:7004
   slots: (0 slots) slave
   replicates 3e8c289a64638da55b0e812f924aaa242efbc315
S: 9882ec7d971ec8b159ff126ac8e1ce480a6ba9e0 192.168.1.200:7003
   slots: (0 slots) slave
   replicates 3af55accd31b6a057945c4b9e9378c7932d5f1f8
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

redis群集的管理命令

cluster nodes   列出叢集節點

[[email protected] ruby-2.4.5]# redis-cli -c -h 192.168.1.234 -p 7000         登入任意一個redis中
192.168.1.234:7000> CLUSTER NODES
a3c7b8a393f2b4d3b9838753fb6496c3861bbb69 192.168.1.200:[email protected] slave d8418ab100f8f32539fc60a7c8ead5927591f52a 0 1542270025000 6 connected
d8418ab100f8f32539fc60a7c8ead5927591f52a 192.168.1.234:[email protected] master - 0 1542270024896 2 connected 10923-16383
3e8c289a64638da55b0e812f924aaa242efbc315 192.168.1.200:[email protected] master - 0 1542270027918 4 connected 5461-10922
2a642a2e14d15ae8cb77b3ef04f58d32cfb09933 192.168.1.234:[email protected] slave 3e8c289a64638da55b0e812f924aaa242efbc315 0 1542270026906 4 connected
3af55accd31b6a057945c4b9e9378c7932d5f1f8 192.168.1.234:[email protected] myself,master - 0 1542270024000 1 connected 0-5460
9882ec7d971ec8b159ff126ac8e1ce480a6ba9e0 192.168.1.200:[email protected] slave 3af55accd31b6a057945c4b9e9378c7932d5f1f8 0 1542270028928 5 connected

cluster info   檢視叢集資訊

192.168.1.234:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:1594
cluster_stats_messages_pong_sent:1603
cluster_stats_messages_sent:3197
cluster_stats_messages_ping_received:1598
cluster_stats_messages_pong_received:1594
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:3197

所有的新新增的redis節點,都是為master角色
cluster meet   ip  port
在192.168.1.200上新啟動一個redis,監聽埠為7007。登入群集中執行redis節點新增命令

[[email protected] redis_data]# redis-cli -c -h 192.168.1.234 -p 7000
192.168.1.234:7000> cluster meet 192.168.1.200 7007
OK
192.168.1.234:7000> cluster nodes                  //第一條就是新新增的節點資訊
cff62edb40a4df11d8390bc28d254abca16188ba 192.168.1.200:[email protected] master - 0 1542270675527 0 connected
a3c7b8a393f2b4d3b9838753fb6496c3861bbb69 192.168.1.200:[email protected] slave d8418ab100f8f32539fc60a7c8ead5927591f52a 0 1542270677550 6 connected
d8418ab100f8f32539fc60a7c8ead5927591f52a 192.168.1.234:[email protected] master - 0 1542270678560 2 connected 10923-16383
----------------省略

再新增一個新的redis節點,並將新節點指定為從節點
cluster reolicate  master node info            //登入需要設定為從節點的redis,執行命令指定主節點
指定從節點需要登入到那個redis節點當中操作,而不是在叢集中指定,在登入節點中指定本節點的master
再次新增一個redis節點

192.168.1.234:7000> cluster meet 192.168.1.234 7006
OK
cff62edb40a4df11d8390bc28d254abca16188ba 192.168.1.200:[email protected] master - 0 1542270675527 0 connected
24f1c355bc92a0ab15a6077b1bbf4bd510dfa132 192.168.1.234:[email protected] master - 0 1542270869918 7 connected

退出節點登入埠為7006的節點,

192.168.1.234:7000> exit
[[email protected] redis_data]# redis-cli -c -h 192.168.1.234 -p 7006
192.168.1.234:7006> CLUSTER REPLICATE cff62edb40a4df11d8390bc28d254abca16188ba
OK

再次檢視群集節點狀態資訊,可以看到埠7006的redis已經成為7007redis的slave節點了

192.168.1.234:7006> cluster nodes
cff62edb40a4df11d8390bc28d254abca16188ba 192.168.1.200:[email protected] master - 0 1542271290173 0 connected
24f1c355bc92a0ab15a6077b1bbf4bd510dfa132 192.168.1.234:[email protected] myself,slave cff62edb40a4df11d8390bc28d254abca16188ba 0 1542271292000 7 connected

移除一個redis節點
cluster forget    node_id
在移除某個redis節點之前,首先不能在登入該節點當中,否則不能正常移除該節點,當前處於7006的redis當中,移除節點提示資訊:

24f1c355bc92a0ab15a6077b1bbf4bd510dfa132 192.168.1.234:[email protected] myself,slave cff62edb40a4df11d8390bc28d254abca16188ba 0 1542271292000 7 connected
192.168.1.234:7006> cluster forget 24f1c355bc92a0ab15a6077b1bbf4bd510dfa132
(error) ERR I tried hard but I can't forget myself...

必須登入其他節點移除埠為7006的redis節點

[[email protected] redis_data]# redis-cli -c -h 192.168.1.234 -p 7000

移除7006節點之前檢視所有節點資訊

192.168.1.234:7000> CLUSTER NODES

redis群集節點刪除和新增

拷貝7006節點的node_id,移除7006節點

192.168.1.234:7000> cluster forget 24f1c355bc92a0ab15a6077b1bbf4bd510dfa132
OK

移除7006節點後

192.168.1.234:7000> CLUSTER NODES

redis群集節點刪除和新增

儲存redis叢集的配置檔案
cluster saveconfig  

儲存配置檔案,會在當前登入的redis資料儲存路徑下增加redis配置內容

192.168.1.234:7000> cluster saveconfig
OK
192.168.1.234:7000> exit

redis群集節點刪除和新增