部署redis集群
1 案例1:部署redis集群
1.1 問題
- 具體要求如下:
- 準備集群環境
- 安裝redis並創建集群
- 查看集群信息
1.2 方案
搭建redis集群,拓撲規劃如圖-1所示:
圖-1
IP,端口規劃如表-1所示:
表-1
1.3 步驟
實現此案例需要按照如下步驟進行。
步驟一:準備集群環境
1)按照表-1配置主機名,ip地址,配置yum源(系統源)這裏不再操作
2)把redis的軟件包傳到6臺數據庫服務器上面,安裝redis服務器,六臺服務器同樣操作(以51為例)
- [[email protected] ~]# yum -y install gcc gcc-c++ make
- [[email protected] ~]# cd redis
- redis/ redis-cluster/
- [[email protected] ~]# cd redis/
- [[email protected] redis]# ls
- lnmp redis-4.0.8.tar.gz
- [[email protected] redis]# tar -xf redis-4.0.8.tar.gz
- [[email protected] redis]# cd redis-4.0.8/
- [[email protected] redis-4.0.8]# make && make install
- [[email protected] redis-4.0.8]# ./utils/install_server.sh
- Welcome to the redis service installer
- This script will help you easily set up a running redis server
- Please select the redis port for this instance: [6379]
- Selecting default: 6379
- Please select the redis config file name [/etc/redis/6379.conf]
- Selected default - /etc/redis/6379.conf
- Please select the redis log file name [/var/log/redis_6379.log]
- Selected default - /var/log/redis_6379.log
- Please select the data directory for this instance [/var/lib/redis/6379]
- Selected default - /var/lib/redis/6379
- Please select the redis executable path [/usr/local/bin/redis-server]
- Selected config:
- Port : 6379
- Config file : /etc/redis/6379.conf
- Log file : /var/log/redis_6379.log
- Data dir : /var/lib/redis/6379
- Executable : /usr/local/bin/redis-server
- Cli Executable : /usr/local/bin/redis-cli
- Is this ok? Then press ENTER to go on or Ctrl-C to abort.
- Copied /tmp/6379.conf => /etc/init.d/redis_6379
- Installing service...
- Successfully added to chkconfig!
- Successfully added to runlevels 345!
- Starting Redis server...
- Installation successful! //安裝成功
- [[email protected] redis-4.0.8]# ss -antlp | grep 6379 //查看時有端口
- LISTEN 0 128 127.0.0.1:6379 *:* users:(("redis-server",pid=10788,fd=6))
2)修改配置文件,6臺redis服務器都要修改(以51為例子)
- [[email protected] redis-4.0.8]# /etc/init.d/redis_6379 stop
- //停止已經開啟的redis服務
- Stopping ...
- Waiting for Redis to shutdown ...
- Redis stopped
- [[email protected] redis-4.0.8]# vim /etc/redis/6379.conf
- ...
- bind 192.168.4.51 //修改ip
- port 6351 //不允許相同,只指定物理接口的ip
- daemonize yes //以守護進程方式運行
- pidfile /var/run/redis_6351.pid
- cluster-enabled yes //是否啟用集群,前提是以守護進程方式運行
- cluster-config-file nodes-6351.conf
- //存儲集群信息的配置文件,自動生成,不允許相同
- cluster-node-timeout 5000 //集群節點通信超時時間
- ...
- [[email protected] redis-4.0.8]# /etc/init.d/redis_6379 start //啟動服務
- Starting Redis server...
- [[email protected] redis-4.0.8]# ss -antlp | grep 6351 //查看有端口
- LISTEN 0 128 192.168.4.51:6351 *:* users:(("redis-server",pid=11092,fd=6))
- LISTEN 0 128 192.168.4.51:16351 *:* users:(("redis-server",pid=11092,fd=8)) //16051:集群中的主機通信時用的端口
- [[email protected] redis-4.0.8]# ps -C redis
- PID TTY TIME CMD
註意:其他幾臺主機在修改時請註意ip,端口等的修改,不要和51主機的一樣
3)關閉防火墻51-56主機(以51為例子)
- [[email protected] redis-4.0.8]# getenforce
- Permissive
- [[email protected] redis-4.0.8]# systemctl disable firewalld
- //關閉防火墻不自啟
4)查看集群信息
- [[email protected] redis-4.0.8]# redis-cli -h 192.168.4.51 -p 6351
- 192.168.4.51:6351> ping
- PONG
- 192.168.4.51:6351> cluster info
- cluster_state:fail
- cluster_slots_assigned:0
- cluster_slots_ok:0
- cluster_slots_pfail:0
- cluster_slots_fail:0
- cluster_known_nodes:1
- cluster_size:0
- ...
- 192.168.4.51:6351> cluster nodes
- f81f997d5ed988ec1587558e78d5f7dbc96abcbf :[email protected]16351 myself,master - 0 0 0 connected
步驟二:創建集群(在任意一臺上執行創建集群的腳本都可以)這裏在51上面執行
1)部署ruby腳本運行環境(在51上面執行)
- [[email protected] redis-4.0.8]# cd /root/redis-cluster/
- [[email protected] redis-cluster]# ls
- redis-3.2.1.gem ruby-devel-2.0.0.648-30.el7.x86_64.rpm
- [[email protected] redis-cluster]# yum -y install ruby rubygems
- [[email protected] redis-cluster]# rpm -ivh –nodeps \
- ruby-devel-2.0.0.648-30.el7.x86_64.rpm
- warning: ruby-devel-2.0.0.648-30.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
- Preparing... ################################# [100%]
- Updating / installing...
- 1:ruby-devel-2.0.0.648-30.el7 ################################# [100%]
- [[email protected] redis-cluster]# which gem
- /usr/bin/gem
- [[email protected] redis-cluster]# gem install redis
- Successfully installed redis-3.2.1
- Parsing documentation for redis-3.2.1
- Installing ri documentation for redis-3.2.1
- 1 gem installed
2)生成創建集群的腳本
- [[email protected] redis-cluster]# cd /root/redis/redis-4.0.8/src/
- [[email protected] src]# cp redis-trib.rb /usr/local/bin/
- [[email protected] src]# ll /usr/local/bin/redis-trib.rb
- -rwxr-xr-x. 1 root root 65991 Sep 27 16:12 /usr/local/bin/redis-trib.rb
3)創建集群
- [[email protected] src]# redis-trib.rb create --replicas 1 \
- 192.168.4.51:6351 192.168.4.52:6352 \
- 192.168.4.53:6353 192.168.4.54:6354 \
- 192.168.4.55:6355 192.168.4.56:6356
- //--replicas 1 給每一個主配置一個從庫
- [[email protected] log]# redis-trib.rb create --replicas 1 192.168.4.51:6351 192.168.4.52:6352 192.168.4.53:6353 192.168.4.54:6354 192.168.4.55:6355 192.168.4.56:6356
- >>> Creating cluster
- >>> Performing hash slots allocation on 6 nodes...
- Using 3 masters:
- 192.168.4.51:6351
- 192.168.4.52:6352
- 192.168.4.53:6353
- Adding replica 192.168.4.55:6355 to 192.168.4.51:6351
- Adding replica 192.168.4.56:6356 to 192.168.4.52:6352
- Adding replica 192.168.4.54:6354 to 192.168.4.53:6353
- ...
- ...
- ...
- [OK] All nodes agree about slots configuration.
- >>> Check for open slots...
- >>> Check slots coverage...
- [OK] All 16384 slots covered.
4)查看集群信息,任意一臺主機訪問本機的redis服務查看即可
cluster info 查看集群信息
cluster nodes 查看集群節點信息
- [[email protected] log]# redis-cli -h 192.168.4.52 -p 6352
- 192.168.4.52:6352> CLUSTER INFO
- cluster_state:ok //狀態
- cluster_slots_assigned:16384
- cluster_slots_ok:16384
- cluster_slots_pfail:0
- cluster_slots_fail:0
- cluster_known_nodes:6
- cluster_size:3
- cluster_current_epoch:6
- cluster_my_epoch:2
- cluster_stats_messages_ping_sent:367
- cluster_stats_messages_pong_sent:327
- cluster_stats_messages_meet_sent:5
- cluster_stats_messages_sent:699
- cluster_stats_messages_ping_received:327
- cluster_stats_messages_pong_received:372
- cluster_stats_messages_received:699
- 192.168.4.52:6352> CLUSTER NODES //查看集群節點信息
- 63afbb5e7d63b1f142358634578a3488e3c9e634 192.168.4.54:[email protected]16354 slave bc5c4e082a5a3391b634cf433a6486c867cfc44b 0 1538039278871 4 connected
- bc5c4e082a5a3391b634cf433a6486c867cfc44b 192.168.4.53:[email protected]16353 master - 0 1538039278571 3 connected 10923-16383
- 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:[email protected]16356 slave 8568fbd73cb296cad6915d524e34761b2114af47 0 1538039278069 6 connected
- 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:[email protected]16355 slave a3083123bc5c87a76aab2655171634d4ee84f418 0 1538039278000 5 connected
- 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 myself,master - 0 1538039277000 2 connected 5461-10922
- a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:[email protected]16351 master - 0 1538039277869 1 connected 0-5460
- 192.168.4.52:6352>
5)測試集群
命令:
redis-cli -c -h ip地址 -p 端口
- [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
- 192.168.4.51:6351> set name jim
- -> Redirected to slot [5798] located at 192.168.4.52:6352
- OK
- 192.168.4.52:6352> get name
- "jim"
- 192.168.4.52:6352> set class linux
- OK
- 192.168.4.52:6352> get class
- "linux"
- 192.168.4.52:6352> set pay 26800
- -> Redirected to slot [4013] located at 192.168.4.51:6351
- OK
- 192.168.4.51:6351> get pay
- "26800"
集群不能用的情況:
有半數或者半數以上的主庫機器掛掉,集群就不能用了
把一個從庫升級成主,沒有從庫,集群不能用(前提是:有半數或者半數以上的主庫機器掛掉)
一個主庫掛掉,它的從庫自動頂替為主庫,正常使用(前提是:有半數或者半數以上的主庫機器能用),掛掉的主庫修復好後,會成為從庫,不會搶占為主
6)集群節點選舉策略(三主,三從)
停止某個主庫的redis服務,對應的從庫會自動升級為主庫
先查看節點信息的主從情況
- [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
- 192.168.4.51:6351> CLUSTER nodes
- ...
- 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 master - 0 1538040400840 2 connected 5461-10922
- 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:[email protected]16356 slave 8568fbd73cb296cad6915d524e34761b2114af47 0 1538040400000 6 connected
- ...
- 192.168.4.51:6351>
看誰是誰的從庫,如:
看節點前後的編號id是否有相同的
如:8568fbd73cb296cad6915d524e34761b2114af47
發現52的從庫為56
停止主庫52
- [[email protected] log]# redis-cli -h 192.168.4.52 -p 6352 shutdown
- [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
- 192.168.4.51:6351> CLUSTER NODES
- ...
- 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 master,fail - 1538041052349 1538041051000 2 disconnected //52的主庫壞掉
- 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:[email protected]16356 master - 0 1538041066000 7 connected 5461-10922 //56成為主庫
- ...
開啟52,發現52成為從庫
- [[email protected] redis-4.0.8]# /etc/init.d/redis_6352 start
- Starting Redis server...
- [[email protected] log]# redis-cli -c -h 192.168.4.51 -p 6351
- 192.168.4.51:6351> CLUSTER NODES
- 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:[email protected]16352 slave 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 0 1538041254000 7 connected
2 案例2:管理redis集群
2.1 問題
- 具體要求如下:
- 練習添加主機
- 練習刪除主機
2.2 步驟
實現此案例需要按照如下步驟進行。
步驟一:添加主機
1)部署一臺新redis服務器,ip為192.168.4.58,裝包,初始化,啟用集群配置,重啟服務(這裏之前已經操作過,不會的可以參考案例1)
2) 添加集群4.58(添加master節點)
格式:redis-trib.rb 選項 參數
選項: add-nade 添加主機(不指定角色為主)
由於之前是在51上面創建ruby腳本,所以只有51上面有redis-trib.rb命令,在51上面執行
- [[email protected] ~]# redis-trib.rb add-node 192.168.4.58:6358 192.168.4.51:6351
- >>> Adding node 192.168.4.58:6358 to cluster 192.168.4.51:6351
- >>> Performing Cluster Check (using node 192.168.4.51:6351)
- S: a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:6351
- slots: (0 slots) slave
- replicates 7e8d9121f44d8331ff55b45c218b87df9bda1b70
- M: 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:6355
- slots:0-5460 (5461 slots) master
- 1 additional replica(s)
- S: 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:6352
- slots: (0 slots) slave
- replicates 28e06c5f24a2b6c6412f81369e09bc9653cc51ff
- M: bc5c4e082a5a3391b634cf433a6486c867cfc44b 192.168.4.53:6353
- slots:10923-16383 (5461 slots) master
- 1 additional replica(s)
- S: 63afbb5e7d63b1f142358634578a3488e3c9e634 192.168.4.54:6354
- slots: (0 slots) slave
- replicates bc5c4e082a5a3391b634cf433a6486c867cfc44b
- M: 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:6356
- slots:5461-10922 (5462 slots) master
- 1 additional replica(s)
- [OK] All nodes agree about slots configuration.
- >>> Check for open slots...
- >>> Check slots coverage...
- [OK] All 16384 slots covered.
- >>> Send CLUSTER MEET to node 192.168.4.58:6358 to make it join the cluster.
- [OK] New node added correctly.
3)檢查集群主機的狀態信息
選項:check 檢查權限
- [[email protected] ~]# redis-trib.rb check 192.168.4.58:6358 //查看狀態
- >>> Performing Cluster Check (using node 192.168.4.58:6358)
- M: c5e0da48f335c46a2ec199faa99b830f537dd8a0 192.168.4.58:6358
- slots: (0 slots) master //發現沒有hash槽
- 0 additional replica(s)
- M: 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:6355
- slots:0-5460 (5461 slots) master
- 1 additional replica(s)
- ...
- S: a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:6351
- slots: (0 slots) slave
- replicates 7e8d9121f44d8331ff55b45c218b87df9bda1b70
- [OK] All nodes agree about slots configuration.
- >>> Check for open slots...
- >>> Check slots coverage...
- [OK] All 16384 slots covered.
4)手動對集群進行分片遷移
選項:reshard 重新分配hash槽
- [[email protected] ~]# redis-trib.rb reshard 192.168.4.58:6358
- How many slots do you want to move (from 1 to 16384)?4096
- //拿出多少個hash 槽給主機192.168.4.58
- What is the receiving node ID? c5e0da48f335c46a2ec199faa99b830f537dd8a0
- //主機192.168.4.58的id值
- Source node #1:all //從當前所有的主裏面獲取hash槽
- Do you want to proceed with the proposed reshard plan (yes/no)?yes
- ...
- Moving slot 12283 from 192.168.4.53:6353 to 192.168.4.58:6358:
- Moving slot 12284 from 192.168.4.53:6353 to 192.168.4.58:6358:
- Moving slot 12285 from 192.168.4.53:6353 to 192.168.4.58:6358:
- Moving slot 12286 from 192.168.4.53:6353 to 192.168.4.58:6358:
- Moving slot 12287 from 192.168.4.53:6353 to 192.168.4.58:6358:
再次查看發現4.58有4096個hash slot
- [[email protected] ~]# redis-trib.rb check 192.168.4.58:6358
- >>> Performing Cluster Check (using node 192.168.4.58:6358)
- M: c5e0da48f335c46a2ec199faa99b830f537dd8a0 192.168.4.58:6358
- slots:0-1364,5461-6826,10923-12287 (4096 slots) master
- 0 additional replica(s)
5)刪除master角色的主機
先刪除主機占用的hash槽
- [[email protected] ~]# redis-trib.rb reshard 192.168.4.58:6358
- How many slots do you want to move (from 1 to 16384)?4096
- //移除hash 槽的個數
- What is the receiving node ID? bc5c4e082a5a3391b634cf433a6486c867cfc44b
- //要移動給誰的id即目標主機(這裏可以隨機寫一個master的ID)
- Source node #1: c5e0da48f335c46a2ec199faa99b830f537dd8a0
- //從誰那移動即源主機(這裏寫4.58的ID)
- Source node #2:done //設置完畢
- ...
- Moving slot 12282 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
- Moving slot 12283 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
- Moving slot 12284 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
- Moving slot 12285 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
- Moving slot 12286 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
- Moving slot 12287 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
- Do you want to proceed with the proposed reshard plan (yes/no)?yes //提交
- ...
- Moving slot 12282 from 192.168.4.58:6358 to 192.168.4.53:6353:
- Moving slot 12283 from 192.168.4.58:6358 to 192.168.4.53:6353:
- Moving slot 12284 from 192.168.4.58:6358 to 192.168.4.53:6353:
- Moving slot 12285 from 192.168.4.58:6358 to 192.168.4.53:6353:
- Moving slot 12286 from 192.168.4.58:6358 to 192.168.4.53:6353:
- Moving slot 12287 from 192.168.4.58:6358 to 192.168.4.53:6353:
刪除集群主機4.58(刪除之後redis服務自動關閉)
- [[email protected] ~]# redis-trib.rb del-node 192.168.4.58:6358 \
- c5e0da48f335c46a2ec199faa99b830f537dd8a0 //刪除誰+刪除的id
- >>> Removing node c5e0da48f335c46a2ec199faa99b830f537dd8a0 from cluster 192.168.4.58:6358
- >>> Sending CLUSTER FORGET messages to the cluster...
- >>> SHUTDOWN the node.
6)添加從節點主機,隨機添加
- [[email protected] ~]# redis-trib.rb add-node --slave \
- 192.168.4.57:6357 192.168.4.51:6351
- >>> Adding node 192.168.4.57:6357 to cluster 192.168.4.51:6351
- >>> Performing Cluster Check (using node 192.168.4.51:6351)
- ……
- ……
- [OK] All 16384 slots covered.
- Automatically selected master 192.168.4.51:6351
- >>> Send CLUSTER MEET to node 192.168.4.57:6357 to make it join the cluster.
- Waiting for the cluster to join.
- >>> Configure node as replica of 192.168.4.51:6351.
- [OK] New node added correctly.
7)移除從節點,從節點主機沒有槽位範圍,直接移除即可
命令格式:
redis-trib.rb del-node 192.168.4.57:6357 主機id值
- [[email protected] ~]# redis-trib.rb del-node 192.168.4.57:6357 \
- f6649ea99b2f01faca26217691222c17a3854381
- >>> Removing node f6649ea99b2f01faca26217691222c17a3854381
- from cluster 192.168.4.57:6351
- >>> Sending CLUSTER FORGET messages to the cluster...
- >>> SHUTDOWN the node.
部署redis集群