redis主從 + 哨兵(sentinel)+ VIP漂移實現高可用
1. 環境及ip角色說明:
這裡使用三臺伺服器,每臺伺服器上開啟一個redis-server和redis-sentinel服務,
redis-server埠為8000,redis-sentinel的埠為6800,修改預設埠是安全的第一步
redis-server:
10.10.10.63:8000(主)
10.10.10.64:8000(從)
10.10.10.65:8000(從)
redis-sentinel:
10.10.10.63:6800
10.10.10.64:6800
10.10.10.65:6800
VIP:10.10.10.100
2. 配置redis-server
yum -y install redis #每臺伺服器都要安裝
cp /etc/redis.conf{,.bak} && cp /etc/redis-sentinel.conf{,.bak} #先備份配置檔案
修改主redis-server配置檔案內容如下:
vim /etc/redis.conf
port 8000 #埠
daemonize yes #開啟後臺守護程序
bind 0.0.0.0 #繫結地址,預設是127.0.0.1,0.0.0.0表示繫結所有地址
pidfile "/var/run/redis-8000.pid"
logfile "/var/log/redis/redis-8000.log"
protected-mode no #關閉保護模式
修改從redis-server配置檔案內容如下:
vim /etc/redis.conf
port 8000
daemonize yes
bind 0.0.0.0
protected-mode no
pidfile "/var/run/redis-8000.pid"
logfile "/var/log/redis/redis-8000.log"
slaveof 10.10.10.63 8000#指定主伺服器ip和埠
啟動redis-server,指定配置檔案位置(主從伺服器都要啟動)
redis-server /etc/redis.conf
檢視主複製情況
[[email protected] ~]# redis-cli -h 10.10.10.63 -p 8000 info replication
# Replication
role:master
connected_slaves:2
slave0:ip=10.10.10.65,port=8000,state=online,offset=29,lag=0
slave1:ip=10.10.10.64,port=8000,state=online,offset=29,lag=0
master_repl_offset:29
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:28
檢視從複製情況
[[email protected] ~]# redis-cli -h 10.10.10.64 -p 8000 info replication
# Replication
role:slave
master_host:10.10.10.63
master_port:8000
master_link_status:up
master_last_io_seconds_ago:10
master_sync_in_progress:0
slave_repl_offset:211
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
3. 配置redis-sentinel(主從伺服器配置都一樣)
[[email protected] ~]# vim /etc/redis-sentinel.conf
daemonize yes #開啟後臺守護程序
port 6800 #埠
bind 0.0.0.0 #繫結所有地址
logfile /var/log/redis/sentinel.log
pidfile /var/run/sentinel.pid
#master8000為例項名稱,監聽主伺服器(10.10.10.63)8000埠,
#這裡的2表示,如果2臺sentinel認為主redis掛了,才算掛
sentinel monitor master8000 10.10.10.63 8000 2
#如果5秒內master8000沒有響應,就認為SDOWN
sentinel down-after-milliseconds master8000 5000
sentinel failover-timeout master8000 15000
#自動切換後,執行指令碼,實現vip自動漂移
sentinel client-reconfig-script master8000 /opt/notify_master6800.sh
新增/opt/notify_master6800.sh,實現vip自動漂移(主從伺服器都要配置)
[[email protected] ~]# cat /opt/notify_master6800.sh
#!/bin/bash
MASTER_IP=$6 #第六個引數是新主redis的ip地址
LOCAL_IP='10.10.10.63' #這裡記得要改,每臺伺服器寫自己的本地ip即可
VIP='10.10.10.100'
NETMASK='24'
INTERFACE='ens32'#網絡卡介面裝置名稱
if [ ${MASTER_IP} = ${LOCAL_IP} ];then
/usr/sbin/ip addr add ${VIP}/${NETMASK} dev ${INTERFACE} #將VIP繫結到該伺服器上
/usr/sbin/arping -q -c 3 -A ${VIP} -I ${INTERFACE}
exit 0
else
/usr/sbin/ip addr del ${VIP}/${NETMASK} dev ${INTERFACE} #將VIP從該伺服器上刪除
exit 0
fi
exit 1
啟動redis-sentinel(主從伺服器全部啟動)
redis-sentinel /etc/redis-sentinel.conf #指定redis-sentinel配置檔案
檢視主(從)伺服器sentinel狀態
[[email protected] ~]# redis-cli -h 10.10.10.63 -p 6800 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=master8000,status=ok,address=10.10.10.63:8000,slaves=2,sentinels=3
4. 繫結VIP到redis主伺服器(第一次需要手動繫結,後面故障切換就不需要)
從伺服器的sentinel狀態可以看出,當前主redis伺服器ip是10.10.10.63
[[email protected] ~]# ip addr add 10.10.10.100/24 dev ens32 #繫結vip到網絡卡上
[[email protected] ~]# arping -q -c 3 -A 10.10.10.100 -I ens32 #arp抑制
[[email protected] ~]# ip a #檢視當前網絡卡ip狀態
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:ad:9c:a1 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.63/24 brd 10.10.10.255 scope global ens32
valid_lft forever preferred_lft forever
inet 10.10.10.100/24 scope global secondary ens32#vip繫結成功
valid_lft forever preferred_lft forever
inet6 fe80::709:25dd:d1e8:815c/64 scope link
valid_lft forever preferred_lft forever
5. 通過VIP(10.10.10.100)測試連線redis-server和redis-sentinel
[[email protected] ~]# redis-cli -h 10.10.10.100 -p 8000 info replication #連線redis-server
# Replication
role:master
connected_slaves:2
slave0:ip=10.10.10.65,port=8000,state=online,offset=174706,lag=1
slave1:ip=10.10.10.64,port=8000,state=online,offset=174706,lag=1
master_repl_offset:174706
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:174705
[[email protected] ~]# redis-cli -h 10.10.10.100 -p 6800 info sentinel #連線redis-sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=master8000,status=ok,address=10.10.10.63:8000,slaves=2,sentinels=3
6. 故障測試
[[email protected] ~]# redis-cli -h 10.10.10.63 -p 8000 shutdown #關閉主redis
[[email protected] ~]# redis-cli -h 10.10.10.100 -p 6800 info sentinel #通過vip檢視sentinel狀態
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
#發現主redis已經切換到10.10.10.65這臺從伺服器上了
master0:name=master8000,status=ok,address=10.10.10.65:8000,slaves=2,sentinels=3
[[email protected]wyq65 ~]# ip a #65從伺服器上檢視網絡卡ip繫結狀態
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:0b:6a:b4 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.65/24 brd 10.10.10.255 scope global ens32
valid_lft forever preferred_lft forever
inet 10.10.10.100/24 scope global secondary ens32#發現VIP已經漂移過來了
valid_lft forever preferred_lft forever
inet6 fe80::5735:b898:abf3:bdec/64 scope link
valid_lft forever preferred_lft forever
到此,使用redis主從 + 哨兵(sentinel)+ 漂移VIP的方案搭建了一個redis高可用系統,但這個系統保證的是單個redis例項的高可用,所以適合業務比較小的應用。如果業務比較大,併發量比較高,建議搭建redis叢集,比如官方redis cluster,還有開源的codings叢集。
後續只需將掛掉的redis恢復,就會自動新增到例項中