解決Nginx + Keepalived主從雙機熱備+自動切換,實現負載均衡及高可用
阿新 • • 發佈:2018-11-01
解決Nginx + Keepalived主從雙機熱備+自動切換,實現負載均衡及高可用
IP | 伺服器 | 服務 |
---|---|---|
192.168.1.10 | lb-node1 | Nginx、keepalived |
192.168.1.11 | lb-node2 | Nginx、keepalived |
[[email protected] /]# yum install -y keepalived
[[email protected] /]# yum install -y psmisc
[[email protected] ~]# systemctl start keepalived.service
[[email protected] ~]# systemctl enable keepalived.service
- 用來進行nginx是否存活的監測,並設定chmod +x check_nginx.sh
[[email protected] ~]# vim /soft/scripts/check_nginx.sh #!/bin/bash if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ] then /usr/bin/systemctl restart nginx.service #檢測到nginx宕機嘗試拉起一次 sleep 5 if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ] then killall keepalived #拉起失敗殺死keepalived,備機獲取vip fi fi
- keepadlived主配置檔案
[[email protected] ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived vrrp_script chk_nginx { script "/soft/scripts/check_nginx.sh" interval 10 #每10s檢查一次 weight -20 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 150 #優先順序,主備之間最好相差50 advert_int 1 #心跳間隔(如果1秒沒通訊,備節點馬上接管) authentication { auth_type PASS auth_pass 1111 } track_script { chk_nginx } virtual_ipaddress { 192.168.1.100/24 } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } track_script { chk_nginx } virtual_ipaddress { 192.168.1.100/24 } }
//備機的配置檔案
slave配置配置檔案與master區別:
......
......
state BACKUP #主機為MASTER,備用機為BACKUP
...
priority 100
// ip漂移測試
[[email protected] ~]# ip a |grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 192.168.1.11/32 brd 192.168.1.11 scope global noprefixroute eth0
inet 192.168.1.100/24 scope global eth0
[[email protected] ~]#
//模擬master故障,此時備機獲取192.168.1.100的VIP
[[email protected] ~]# systemctl stop keepalived.service
[[email protected] ~]# ip a |grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 192.168.1.11/32 brd 192.168.1.11 scope global noprefixroute eth0
inet 192.168.1.100/24 scope global eth0
[[email protected] ~]#
- 在兩臺Web Server上執行realserver.sh指令碼,為lo:0繫結VIP地址192.168.1.100、抑制arp廣播
#!/bin/bash
#description: Config realserver
VIP=192.168.1.100
/etc/rc.d/init.d/functions
case "$1" in
start)
/sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
/sbin/ifconfig lo:0 down
/sbin/route del $VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
// 分別在主從機上執行 sh realserver.sh start 就可實現負載均衡及高可用叢集了;
[[email protected] /soft/scripts]# ip a |grep -E "lo|eth0"
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.1.100/32 brd 192.168.1.100 scope global lo:0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 192.168.1.10/24 brd 192.168.1.255 scope global noprefixroute eth0
inet 192.168.1.100/24 scope global secondary eth0
inet6 2409:8a28:8a8:e3c0:b6d2:ec3c:3557:2609/64 scope global noprefixroute dynamic
[[email protected] /soft/scripts]#
- 實現負載均衡及高可用叢集
[[email protected] scripts]# ip a |grep -E "lo|eth0"
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.1.100/32 brd 192.168.1.100 scope global lo:0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
inet 192.168.1.11/32 brd 192.168.1.11 scope global noprefixroute eth0
[[email protected] scripts]#
- Keepalived配置檔案詳解
inet 10.0.0.11/24 scope global secondary eth0
[[email protected] ~]# /etc/init.d/keepalived stop #停止Master上Keepalived
[[email protected] ~]# ip addr|grep 10.0.0.11 #VIP已經從Master端移除
[[email protected] ~]# ip addr|grep 10.0.0.11 #Backup上Keepalived接管資源
inet 10.0.0.11/24 scope global secondary eth0
[[email protected] ~]# /etc/init.d/keepalived start #啟動Master_keepalived
[[email protected] ~]# ip addr|grep 10.0.0.11 #Master繼續接管資源
inet 10.0.0.11/24 scope global secondary eth0
1.5.9Keepalived配置檔案詳解
1 ! Configuration File for keepalived #註釋
2
3 global_defs {
4 notification_email {
5 [email protected] #5-7發郵件給誰
6
7
8 }
9 notification_email_from [email protected] #發郵件發件人
10 smtp_server 192.168.200.1 #郵件伺服器地址
11 smtp_connect_timeout 30 #超時時間
12 router_id Nginx_01 #主備ID不能一樣
13 }
14
15 vrrp_instance VI_1 { #例項名稱(建議不修)
16 state MASTER #伺服器的狀態(僅僅是傀儡)
17 interface eth0 #通訊埠
18 virtual_router_id 51 #例項的ID
19 priority 150 #優先順序,主備之間最好相差50
20 advert_int 1 #心跳間隔(如果一秒沒通訊備節點馬上接管)
21 authentication {
22 auth_type PASS #PASS認證型別,此引數備節點設定和主節點相同
23 auth_pass 1111 #密碼是1111,此引數備節點設定和主節點相同
24 }
25 virtual_ipaddress { #vip(可以多個)
26 10.0.0.11/24 #26-28配置vIP地址,繫結在eth0 因為(interface eth0)
29 }
30 }
q全域性定義塊部分:主要設定Keepalived的通知機制和標識
1、第4-9行是email通知引數。作用:當LVS發生切換或RS等有故障時,會發郵件報警。這是可選配,notifucation_email指定在keepalived發生事件時,需要發給的email地址,可以有多個,每行一個。
2、smtp_server指定傳送郵件的smtp伺服器,如果本機開啟了sendmail,就可以使用上面預設配置實現郵件傳送。
3、第10行是Lvs負載均衡器標示(rote_id)。在一個區域網內,它應該是唯一的。
4、大括號”{}” 用來分隔定義塊,因此必須成對出現。如果漏寫了,keepalived執行時,不會得到預期的結果。由於定義塊記憶體在巢狀關係,因此很容易遺漏結尾處的花括號,這點要特別注意。
qVRRP定義塊
1、第13行為VRRP例項vrrp_instance,每個Vrrp例項可以認為是一個keepalived例項,在配置中VRRP例項可以有多個。
(1)第14行例項狀態state.只有Master和Backup兩種狀態,並且需要大寫這些單詞。其中MASTER為工作狀態。BACKUP為備用狀態。當MASTER所在的伺服器失效時,BACKUP所在的系統會自動把它的狀態有BACKUP變換成MASTER,當失效的MASTER所在的系統恢復時,BACKUP從MASTER恢復到BACKUP狀態。
(2)通訊介面interface。對外提供服務的網路結構,如eth0,eth1當前主流的伺服器有2個或2個以上的網路介面,在選擇伺服器介面時,一定要搞清楚了。
(3)lvs_sync_daemon_interface。負載均衡器之間的監控介面,類似於HA HeartBeat的心跳線。
(4)第16行為虛擬路由標示virtual_route_id是一致的,同時在整個keepalived內是唯一的。
(5)第17行為優先順序priority,這是一個數字,數值愈大,優先順序越高。在同一個vrrp_instance裡,MASTER的優先順序 BACKUP。若MASTER的priority值為150,那麼BACKUP的priority只能在149或者跟小的數值(官方建議相差50)。
(6)第18行同步通知間隔advert_int。MASTER與BACKUP負載均衡器之間同步檢查的時間間隔,單位為秒。
(7)第19-22行驗證authentication.包含驗證型別和驗證密碼。型別主要有PASS、AH兩種,通常使用的型別為PASS,據說AH使用時有問題。驗證密碼為明文,同一vrrp例項MASTER與BACKUP使用相同的密碼才能正常通訊,這裡官方推薦用明文即可。
2、第23-27行為虛擬ip地址virtual_ipaddress。可以配置多個IP地址,每個地址佔一行,需要指定子網掩碼。
注意:這個ip必須與我們在lvs客戶端設定的vip相一致。