lvs+keepalived叢集
一.知識梳理
LVS+keepalived基於完成開源軟體的構架實現負載均衡高可用
1.LVS
LVS是Linux Virtual Server的簡寫，即Linux虛擬伺服器，是一個虛擬的服務集群系統。有三種負載均衡技術（VS/NAT、VS/TUN、VS/DR），八種排程演算法。
2.keeplived
keepalived在這裡主要做realserver健康檢查以及loadbalance主機和backup主機之間failover的實現。
keepalived是一個基於VRRP協議來實現WEB服務高可用方案，可以用來避免單點故障，一個web伺服器至少有兩臺伺服器執行keepalived，一臺主伺服器（MASTER），一臺備用伺服器（BACKUP），但是對外只表現一個vip，主伺服器會發送特定訊息給備用伺服器，當備用伺服器收到訊息時，備用伺服器就會接管ipswich，繼續提供服務，從而保證了高可用性。
3.kepalive工作原理
Layer3:keepalived使用layer3的方式工作時，keepalived會定期向伺服器群中傳送一個ICMP的資料包（即我們平時用的ping程式），如果發現某臺伺服器的IP地址沒有啟用，keepalived便會報告這臺伺服器是小，並將他從伺服器群中剔除。Layer3的方式是以伺服器的IP第孩子是否有效作為伺服器工作正常與否的標準
Layer4:主要以TCP埠的狀態來決定伺服器工作正常與否。如web sercer的服務埠一般是80.如果keepalived檢測到80埠沒有啟動，則keepalived將這臺伺服器從服務群中刪除。
Layer5:layer5就是工作載具體的應用層，比layer3,4要複雜一點，載網路上佔用的寬頻也要打一些。Keepalived將根據使用者的設定檢查伺服器的執行是否正常。如果設定不相符，則keepalived將把伺服器從群中踢除。
二.實現
1.lvs配置
server1
yum install -y ipvsadm
ipvsadm -A -t 172.25.38.100:80 -s rr
新增策略 指定vip 172.25.66.100:80，-t指tcp，-s rr 指定輪詢
ip addr add 172.25.66.100/24 dev eth0 ##新增虛擬ip
ipvsadm -L ##列出ipvsadm策略
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.66.100:http rr
ipvsadm -a -t 172.25.66.100:80 -r 172.25.66.2:80 -g
ipvsadm -a -t 172.25.66.100:80 -r 172.25.66.3:80 -g
將虛擬ip對映到真實的ip，-r指定realserver -t指定tcp協議 -g指dr模式
ipvsadm -L ##檢視策略已加入
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.66.100:http rr
-> server2:http Route 1 0 0
-> server3:http Route 1 0 0
/etc/init.d/ipvsadm save ##儲存策略，不儲存的話關機策略即失效
ipvsadm: Saving IPVS table to /etc/sysconfig/ipvsadm: [ OK ]
server2
ip addr add 172.25.66.100/32 dev eth0 ##在realserver上加一個和vip相同的ip地址
ip addr
[[email protected] ~]# yum install -y arptables_jf
為防止客戶端在訪問vip時直接訪問到realserver，在realserver設定一定的策略
[[email protected] ~]# arptables -A IN -d 172.25.66.100 -j DROP
訪問realserver的100ip時的請求直接丟棄
[[email protected] ~]# arptables -A OUT -s 172.25.66.100 -j mangle –mangle-ip-s 172.25.66.2
realserver發出的資訊全部轉為他的真實ip
[[email protected] ~]# /etc/init.d/arptables_jf save ##儲存策略
Saving current rules to /etc/sysconfig/arptables: [ OK ]
[[email protected] ~]# /etc/init.d/httpd start
Starting httpd: httpd: Could not reliably determine the server’s fully ####qualified domain name, using 172.25.6.2 for ServerName
[ OK ]
server3（兩個realserver做同樣的配置）
[[email protected] ~]# ip addr add 172.25.66.100/32 dev eth0
[[email protected] ~]# ip addr
[[email protected] ~]# yum install -y arptables_jf
[[email protected] ~]# arptables -A IN -d 172.25.66.100 -j DROP
[[email protected] ~]# arptables -A OUT -s 172.25.66.100 -j mangle –mangle-ip-s 172.25.66.3
[[email protected] ~]# /etc/init.d/arptables_jf save
Saving current rules to /etc/sysconfig/arptables: [ OK ]
[[email protected] ~]# /etc/init.d/httpd start
Starting httpd: httpd: Could not reliably determine the server’s fully qualified domain name, using 172.25.6.3 for ServerName
[ OK ]
此時已經實現了負載均衡了
我在這裡是直接安裝keepalived沒有進行原始碼編譯
直接開服務就行
修改配置檔案
vim keepalived.conf ##編輯主配置檔案
1 ! Configuration File for keepalived
2
3 global_defs {
4 notification_email {
5 [email protected] ##設定報警郵件地址，如果要開啟郵件報警，要開啟本機的sendmail服務
6 }
7 notification_email_from [email protected]##設定郵件的傳送地址
8 smtp_server 127.0.0.1 ##設定smpt server郵件服務端的地址
9 smtp_connect_timeout 30 ##設定連線smptserver的超時時間
10 router_id LVS_DEVEL
11 vrrp_skip_check_adv_addr
12 vrrp_strict
13 }
14
15 vrrp_instance VI_1 {
16 state MASTER ##指定keepalived的角色，MASTER表示此主機為主伺服器，若為BACKUP則表示備用伺服器
17 interface eth0 ##指定HA檢測網路的介面
18 virtual_router_id 51 ##虛擬路由標識，MASTER和BACKUP的必須相同
19 priority 100 ##定義優先順序，數字越大優先順序越高，因此MASTER的必須年比BACKUP的數字大，否則就會有衝突
20 advert_int 1 ##設定MASTER和BACKUP負載均衡之間同步檢查的時間間隔，單位為秒
21 authentication { ##設定驗證型別和密碼
22 auth_type PASS
23 auth_pass 1111
24 }
25 virtual_ipaddress { ##指定虛擬ip
26 172.25.66.100
27 }
28 }
29 ##虛擬伺服器定義部分
30 virtual_server 172.25.66.100 80 { ##設定虛擬伺服器ip地址及埠，兩者以空格隔開
31 delay_loop 6 ##設定執行情況檢查時間，單位為秒
32 lb_algo rr ##設定負載均衡演算法，這裡設定為rr演算法，即輪詢演算法
33 lb_kind DR ##設定lvs實現負載均衡機制，有NAT，TUN，DR三種模式
34 #persistence_timeout 50 ##會話保留時間，單位為秒
35 protocol TCP ##指定轉發協議型別
36
37
38
39 real_server 172.25.66.2 80 { ##配置realserver，需要指定真實ip與其埠，用空格隔開
40 weight 1 ##配置服務節點的權重值
41 TCP_CHECK { ##realserver檢查狀態設定部分
42 connect_timeout 3 ##3s無響應時超時
43 nb_get_retry 3 ##重複次數
44 delay_before_retry 3 ##重複間隔
45 }
46 }
47 real_server 172.25.66.3 80 {
48 weight 1
49 TCP_CHECK {
50 connect_timeout 3
51 nb_get_retry 3
52 delay_before_retry 3
53 }
54 }
55 }
另外一端也要安裝keepalived 用作備用機
[[email protected] keepalived]# vim keepalived.conf
1 ! Configuration File for keepalived
2
3 global_defs {
4 notification_email {
5 [email protected]
6 }
7 notification_email_from [email protected]
8 smtp_server 127.0.0.1
9 smtp_connect_timeout 30
10 router_id LVS_DEVEL
11 vrrp_skip_check_adv_addr
12 vrrp_strict
13 }
14
15 vrrp_instance VI_1 {
16 state BACKUP ##備用伺服器
17 interface eth0
18 virtual_router_id 51
19 priority 50 ##優先順序必須低於主伺服器
20 advert_int 1
21 authentication {
22 auth_type PASS
23 auth_pass 1111
24 }
25 virtual_ipaddress {
26 172.25.66.100
27 }
28 }
29
30 virtual_server 172.25.66.100 80 {
31 delay_loop 6
32 lb_algo rr
33 lb_kind DR
34 #persistence_timeout 50
35 protocol TCP
36
37
38
39 real_server 172.25.66.2 80 {
40 weight 1
41 TCP_CHECK {
42 connect_timeout 3
43 nb_get_retry 3
44 delay_before_retry 3
45 }
46 }
47 real_server 172.25.66.3 80 {
48 weight 1
49 TCP_CHECK {
50 connect_timeout 3
51 nb_get_retry 3
52 delay_before_retry 3
53 }
54 }
55 }
[[email protected] keepalived]# /etc/init.d/keepalived start
Starting keepalived: [ OK ]
[[email protected] keepalived]# tail -f /var/log/messages
Jul 26 13:46:25 server1 Keepalived_healthcheckers[24222]: Removing service [172.25.66.3]:80 from VS [172.25.66.100]:80
Jul 26 13:46:25 server1 Keepalived_healthcheckers[24222]: Remote SMTP server [127.0.0.1]:25 connected.
Jul 26 13:46:25 server1 Keepalived_healthcheckers[24222]: SMTP alert successfully sent.
Jul 26 13:46:28 server1 Keepalived_healthcheckers[24222]: TCP connection to [172.25.66.2]:80 failed.
Jul 26 13:46:28 server1 Keepalived_healthcheckers[24222]: Check on service [172.25.66.2]:80 failed after 1 retry.
Jul 26 13:46:28 server1 Keepalived_healthcheckers[24222]: Removing service [172.25.66.2]:80 from VS [172.25.66.100]:80
Jul 26 13:46:28 server1 Keepalived_healthcheckers[24222]: Lost quorum 1-0=1 > 0 for VS [172.25.66.100]:80
Jul 26 13:46:28 server1 Keepalived_healthcheckers[24222]: Remote SMTP server [127.0.0.1]:25 connected.
Jul 26 13:46:28 server1 Keepalived_healthcheckers[24222]: SMTP alert successfully sent.
Jul 26 13:46:28 server1 Keepalived_vrrp[24223]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 172.25.66.100
[[email protected] keepalived]# ip addr ##接管vip
[[email protected] keepalived]# ipvsadm -l
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.66.100:http rr
-> server2:http Route 1 0 3
-> server3:http Route 1 0 4
測試：
server2和server3實現lvs
server1和server4實現keepalived
1）當server1和test2的keepalived服務都正常時,server1的keepalived工作（預設master工作，這裡server1被設定為master）
[[email protected] Desktop]$ curl 172.25.66.100
www.westos.com-server1
[[email protected] Desktop]$ curl 172.25.66.100
www.westos.com-server2
[[email protected] Desktop]$ arp -an | grep 100
? (172.25.66.100) at 52:54:00:c1:37:57 [ether] on br0
[[email protected] local]# ip addr
2）當master即server1掛掉時，test2的keepalived工作
[[email protected] local]# /etc/init.d/keepalived stop
Stopping keepalived: [ OK ]
[[email protected] Desktop]$ curl 172.25.66.100
www.westos.com-server2
[[email protected] Desktop]$ arp -an | grep 100
? (172.25.66.100) at 52:54:00:7a:98:49 [ether] on br0
[[email protected] keepalived]# ip addr
