二十九節課預習
Linux叢集架構（二）
目錄
八、LVS DR模式搭建
九、keepalived + LVS
十、擴充套件
八、LVS DR模式搭建
1、實驗環境：
四臺機器：
client: 192.168.76.150
Director節點： (ens32 192.168.76.155 vip ens32:0 192.168.76.158)
Real server1： (ens32 192.168.76.156 vip lo:0 192.168.76.158)
Real server2： (ens32 192.168.76.157 vip lo:0 192.168.76.158)
2、安裝
//兩臺real server需安裝web服務。之前已經裝過，略過//在director安裝ipvsadm軟體包，可參考lvs nat部分
[[email protected] ~]# yum -y install ipvsadm
3、在director上配置指令碼
[[email protected] ~]# vim /usr/local/sbin/lvs-dr.sh
#!/bin/bash
echo 1 > /proc/sys/net/ipv4/ip_feorward
ipv=/usr/sbin/ipvsadm
vip=192.168.76.158
rs1=192.168.76.156
rs2=192.168.76.157
ifconfig ens32:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip dev ens32:0$ipv -C$ipv -A -t $vip:80 -s rr$ipv -a -t $vip:80 -r $rs1:80 -g -w 3$ipv -a -t $vip:80 -r $rs2:80 -g -w 1
//賦予755許可權，
[[email protected] ~]# chmod 755 /usr/local/sbin/lvs-dr.sh
//執行指令碼
[[email protected] ~]# /usr/local/sbin/lvs-dr.sh
//檢視狀態
[[email protected] ~]# chmod 755 /usr/local/sbin/lvs-dr.sh^C
[[email protected] ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.76.158:80 rr
-> 192.168.76.156:80 Route 3 0 0
-> 192.168.76.157:80 Route 1 0 0
4、在兩臺real server配置指令碼
[[email protected] ~]# vim /usr/local/sbin/lvs-dr-rs.sh
#!/bin/bash
vip=192.168.76.158
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
//賦予755許可權，然後執行
[[email protected] ~]# chmod 755 /usr/local/sbin/lvs-dr-rs.sh
//執行
[[email protected] ~]# /usr/local/sbin/lvs-dr-rs.sh
5、測試
//當前採用的是rr排程演算法
Last login: Mon Jul 23 14:47:55 2018
[[email protected] ~]# ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:a2:07:b1 brd ff:ff:ff:ff:ff:ff
inet 192.168.76.150/24 brd 10.0.1.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fea2:7b1/64 scope link
valid_lft forever preferred_lft forever
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
九、keepalived + LVS
LVS可以實現負載均衡，但是不能夠進行健康檢查，如一個rs出現故障，LVS 仍然會把請求轉發給故障的rs伺服器，這就會導致請求的無效性。keepalive 軟體可以進行健康檢查，而且能同時實現 LVS 的高可用性，解決 LVS 單點故障的問題，其實 keepalive 就是為 LVS 而生的。
1、實驗環境
4臺節點
Keepalived1 + lvs1(Director1)：192.168.76.155
Keepalived2 + lvs2(Director2)：192.168.76.159
Real server1：192.168.76.156
Real server2：192.168.76.157
VIP:192.168.76.158
2.軟體安裝
//Keepalived + lvs兩個節點安裝
[[email protected] ~]# yum install ipvsadm keepalived -y
//兩個read server安裝nignx, 之前環境已經安裝過，此處略
3.設定配置指令碼
//兩臺real server節點建立指令碼
[[email protected] ~]# vim /usr/local/sbin/lvs-dr-rs.sh#!/bin/bash
vip=192.168.76.158
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
//賦予755許可權，然後執行
[[email protected] ~]# chmod 755 /usr/local/sbin/lvs-dr-rs.sh
//執行
[[email protected] ~]# /usr/local/sbin/lvs-dr-rs.sh
//兩臺keepalived節點配置
//master節點配置檔案
[[email protected] ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
#備用伺服器上為 BACKUP
state MASTER
interface ens32
virtual_router_id 51
#備用伺服器上為90
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass aminglinux
}
virtual_ipaddress {
192.168.76.158
}
}
virtual_server 192.168.76.158 80 {
#(每隔10秒查詢realserver狀態)
delay_loop 10
#(lvs 演算法)
lb_algo wrr
#(DR模式)
lb_kind DR
#(同一IP的連線60秒內被分配到同一臺realserver)
#實驗環境註釋掉，不然看不到rr的效果
#persistence_timeout 60
#(用TCP協議檢查realserver狀態)
protocol TCP
real_server 192.168.76.156 80 {
#(權重)
weight 1
TCP_CHECK {
#(10秒無響應超時)
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.76.157 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
//backup節點
[[email protected] ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
#備用伺服器上為 BACKUP
state BACKUP
interface ens32
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass aminglinux
}
virtual_ipaddress {
192.168.76.158
}
}
virtual_server 192.168.76.158 80 {
#(每隔10秒查詢realserver狀態)
delay_loop 10
#(lvs 演算法)
lb_algo rr
#(DR模式)
lb_kind DR
#(同一IP的連線60秒內被分配到同一臺realserver)
#persistence_timeout 60
#(用TCP協議檢查realserver狀態)
protocol TCP
real_server 192.168.76.156 80 {
#(權重)
weight 1
TCP_CHECK {
#(10秒無響應超時)
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.76.157 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
4.在keepalived兩個節點開啟轉發功能
[[email protected] ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
5.在兩個節點啟動keepalive,
[[email protected] ~]# systemctl start keepalived.service
[[email protected] ~]# systemctl start keepalived.service
6.測試
//測試1：手動關閉192.168.76.156節點的nginx，在客戶端上去測試訪問
//在192.168.76.156上操作
[[email protected] ~]# /usr/local/nginx/sbin/nginx -s stop
[[email protected] ~]# lsof -i :80
//在192.168.76.150客戶端上測試
Last login: Mon Jul 23 14:49:10 2018 from 10.0.1.229
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!//結果正常，不會出現訪問192.168.76.156節點，一直訪問的是192.168.76.157節點的內容。
//測試2 手動重新開啟 192.168.76.156 節點的nginx， 在客戶端上去測試訪問
//在192.168.76.156上操作
[[email protected] ~]# lsof -i :80
[[email protected] ~]# /usr/local/nginx/sbin/nginx
[[email protected] ~]# lsof -i :80
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
nginx 2969 root 6u IPv4 48805 0t0 TCP *:http (LISTEN)
nginx 2970 nginx 6u IPv4 48805 0t0 TCP *:http (LISTEN)
nginx 2971 nginx 6u IPv4 48805 0t0 TCP *:http (LISTEN)
//在192.168.76.150上測試
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
//結果正常，按照 rr 排程演算法訪問192.168.76.156節點和192.168.76.157節點內容。
//測試 keepalived 的HA特性
//ip addr檢視，此時192.168.76.158的vip在主上
valid_lft forever preferred_lft forever
[[email protected] ~]# ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:85:24:8c brd ff:ff:ff:ff:ff:ff
inet 192.168.76.155/24 brd 10.0.1.255 scope global ens32
valid_lft forever preferred_lft forever
inet 192.168.76.158/32 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe85:248c/64 scope link
valid_lft forever preferred_lft forever
//停止master上的keepalived
[[email protected] ~]# systemctl stop keepalived.service
[[email protected] ~]#
//在dr2上檢視，vip搶佔過來了
[[email protected] ~]# ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:dd:53:4e brd ff:ff:ff:ff:ff:ff
inet 192.168.76.159/24 brd 10.0.1.255 scope global ens32
valid_lft forever preferred_lft forever
inet 192.168.76.158/32 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::c388:e67a:4ac3:6566/64 scope link
valid_lft forever preferred_lft forever
//在192.168.76.150上測試
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
I am Lvs-backend1!!!
[[email protected] ~]# curl 192.168.76.158
I am lvs-backend2!!!
[[email protected] ~]# curl 192.168.76.158
//可以正常訪問後端的網站,驗證了keepalived的特性
//重新開啟master上的keepalived
[[email protected] ~]# systemctl start keepalived.service
[[email protected] ~]# ip add1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:85:24:8c brd ff:ff:ff:ff:ff:ff
inet 192.168.76.155/24 brd 10.0.1.255 scope global ens32
valid_lft forever preferred_lft forever
inet 192.168.76.158/32 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe85:248c/64 scope link
valid_lft forever preferred_lft forever
十、擴充套件
heartbeat和keepalived比較
http://blog.csdn.net/yunhua_lee/article/details/9788433
DRBD工作原理和配置
http://502245466.blog.51cto.com/7559397/1298945
mysql+keepalived
http://lizhenliang.blog.51cto.com/7876557/1362313
lvs 三種模式詳解
http://www.it165.net/admin/html/201401/2248.html
lvs幾種演算法
http://www.aminglinux.com/bbs/thread-7407-1-1.html
關於arp_ignore和 arp_announce
http://www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html
lvs原理相關的
http://blog.csdn.net/pi9nc/article/details/23380589
haproxy+keepalived
http://blog.csdn.net/xrt95050/article/details/40926255
nginx、lvs、haproxy比較
http://www.csdn.net/article/2014-07-24/2820837
keepalived中自定義指令碼 vrrp_script
http://my.oschina.net/hncscwc/blog/158746
lvs dr模式只使用一個公網ip的實現方法
http://storysky.blog.51cto.com/628458/338726