實驗準備：

1）平臺使用RHEL6.5的64位系統平臺

2）所需軟件為ipvsadm、keepalived、httpd

3）節點如下：

direct1 192.168.3.166

direct2 192.168.3.168

rip1 192.168.3.33

rip2 192.168.3.34

VIP 192.168.3.35

1、安裝lvs-1.26

1) yum -y install gcc gcc-c++

2) yum -y install libnl* libpopt*

3) yum -y install popt-devel

4) rpm -ivh popt-static-1.13-7.el6.x86_64.rpm

5) tar -xf ipvsadm-1.26.tar.gz

6) cd ipvsadm-1.26

7) make && make install

2、安裝keepalived

1）yum -y install openssl openssl-devel

2）yum install -y libnfnetlink-devel

3) tar -xf keepalived-1.3.5.tar.gz

3）cd keepalived-1.3.5

4) ./configure --prefix=/usr/local/keepalived

5) make && make install

6)配置keepalived文件的路徑

cp /opt/keepalived-1.3.5/keepalived/etc/init.d/keepalived /etc/init.d/

cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

mkdir /etc/keepalived

cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

3、在其中一個direct上配置/etc/keepalived/keepalived.conf，當前是在LVS的MASTER端

global_defs {

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id LVS1

}

vrrp_sync_group test {

group {

test_1

}

}

vrrp_instance test_1 {

state MASTER

interface eth0

lvs_sync_daemon_inteface eth0

virtual_router_id 51

priority 100

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

virtual_ipaddress {

192.168.3.35

}

}

virtual_server 192.168.3.35 80 {

delay_loop 6

lb_algo rr

lb_kind DR

persistence_timeout 0

protocol TCP

real_server 192.168.3.33 80 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

real_server 192.168.3.34 80 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

}

4、編輯好配置文件後，可以把這個文件復制到LVS的BACKUP端，即另一個direct節點，然後再BACKUP端修改state 為BACKUP,priority為99即可，如下：

global_defs {

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id LVS2

}

vrrp_sync_group test {

group {

test_1

}

}

vrrp_instance test_1 {

state BACKUP

interface eth0

lvs_sync_daemon_inteface eth0

virtual_router_id 51

priority 99

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

virtual_ipaddress {

192.168.3.35

}

}

virtual_server 192.168.3.35 80 {

delay_loop 6

lb_algo rr

lb_kind DR

persistence_timeout 0

protocol TCP

real_server 192.168.3.33 80 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

real_server 192.168.3.34 80 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

}

5、在兩個direct上啟動keepalived服務

/etc/init.d/keepalived start

6、當keepalived啟動完成之後，可以檢查虛擬IP地址是否已經生產，且虛擬IP地址位於MASTER上。

由於在配置文件中將direct1設置為MASTER，因此，只需要在direct上執行如下命令，即可檢查配置是否成功，如下：

# ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 7a:d2:18:41:d3:c8 brd ff:ff:ff:ff:ff:ff

inet 192.168.3.166/24 brd 192.168.3.255 scope global eth0

inet 192.168.3.35/32 scope global eth0

inet6 fe80::78d2:18ff:fe41:d3c8/64 scope link

valid_lft forever preferred_lft forever

7、測試VIP的漂移

1)當MASTER發生故障時，測試VIP是否會漂移到BACKUP上去。

可以在MASTER上執行如下命令來模擬BACKUP發生故障：

/etc/init.d/keepalived stop

2)之後去BACKUP端執行以下命令：

# ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 46:6c:ca:46:84:18 brd ff:ff:ff:ff:ff:ff

inet 192.168.3.168/24 brd 192.168.3.255 scope global eth0

inet 192.168.3.35/32 scope global eth0

inet6 fe80::446c:caff:fe46:8418/64 scope link

valid_lft forever preferred_lft forever

3）以上即說明，當MASTER發生故障時，VIP就會漂移到BACKUP上去

8、keepalived的高可用已經實現，之後就是要配置與LVS有關的內容，需在兩臺真實服務器上操作

在兩臺真實服務器上安裝apache，並且啟動httpd服務

在這最好是編輯一下兩臺真實服務器的httpd的默認根路徑下的文件，使兩個文件的內容不同，以便於測試。

1）以上工作做好，開始在兩臺真實服務器上配置VIP，如下

vim /etc/init.d/realserver.sh

#!/bin/bash

# description: Config realserver lo and apply noarp

. /etc/rc.d/init.d/functions

VIP=192.168.3.35

host=`/bin/hostname`

case "$1" in

start)

/sbin/ifconfig lo down

/sbin/ifconfig lo up

echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

# sysctl -p >/dddev/null 2>&1

# echo "RealServer Start OK"

#/sbin/ifconfig lo:0$VIP broadcast $VIP netmask 255.255.255.255up

#/sbin/route add -host $VIP dev lo:0

/sbin/ifconfig lo:0 192.168.3.35 broadcast 192.168.3.35 netmask 255.255.255.255 up

/sbin/route add -host 192.168.3.35 dev lo:0

;;

stop)

/sbin/ifconfig lo:0 down

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

echo "RealServer Stoped"

;;

status)

# Status of LVS-DR real server.

islothere=`/sbin/ifconfig lo:0 | grep 192.168.3.35`

isrothere=`netstat -rn | grep "lo:0" | grep 192.168.3.35`

if [ ! "$islothere" -o ! "isrothere" ];then

# Either the route or the lo:0 device

# not found.

echo "LVS-DR real server Stopped."

else

echo "LVS-DR real server Running."

fi

;;

*)

# Invalid entry.

echo "$0: Usage: $0 {start|status|stop}"

exit 1

;;

esac

2）配置文件寫好之後，加權限再運行

chmod +x /etc/init.d/realserver.sh

/etc/init.d/realserver.sh start

3）然後再運行ifconfig，可查看到VIP已配置好

#ifconfig

eth0 Link encap:Ethernet HWaddr 42:58:57:BE:25:BE

inet addr:192.168.3.33 Bcast:192.168.3.255 Mask:255.255.255.0

inet6 addr: fe80::4058:57ff:febe:25be/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:2515650 errors:0 dropped:0 overruns:0 frame:0

TX packets:16369 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:164807767 (157.1 MiB) TX bytes:1947231 (1.8 MiB)

Interrupt:165

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:4920 errors:0 dropped:0 overruns:0 frame:0

TX packets:4920 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:339452 (331.4 KiB) TX bytes:339452 (331.4 KiB)

lo:0 Link encap:Local Loopback

inet addr:192.168.3.35 Mask:255.255.255.255

UP LOOPBACK RUNNING MTU:16436 Metric:1

4）兩臺真實服務器上的VIP的配置都是一致的

5）在MASTER上查看IPVS規則是否成功生成

#ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 192.168.3.35:80 rr

-> 192.168.3.33:80 Route 1 0 0

-> 192.168.3.34:80 Route 1 0 0

9、測試

1）在瀏覽器上輸入http://192.168.3.35，然後觀察頁面內容，多刷新幾次，就會發現頁面內容會發生變化，即成功調用兩臺真實服務器的htppd默認網頁的內容。

2）當後端的真實服務器發生故障或運行在真實服務器上的服務發生故障時，則與這個真實服務器相關的IPVS規則會被刪除。

如在rip1上執行如下命令，來模擬httpd服務發生故障

/usr/local/apache/bin/apachectl stop

然後再在作為MASTER上的服務器上查看IPVS規則，使用如下命令即可查看IPVS規則，這時我的direct1是作為MASTER。

#ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 192.168.3.35:80 rr

-> 192.168.3.34:80 Route 1 0 0

這時會發現與rip1相關的IPVS規則被刪除，測試成功。

補充：

本人在做好LVS+Keepalived之後，真實服務器也做好的相關的配置，但是在MASTER運行 ipvsadm -L -n 時，就只有一個真實服務器被添加到列表中，另一臺沒有添加成功！

經過網上搜索，排查錯誤，最後找到原因，即在兩個節點上配置keepalived.conf文件時，有一個地方不規範，即添加realserver時，語法出現錯誤

real_server 192.168.3.33 80 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

real_server 192.168.3.34 80 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

connect_port 80

}

}

這裏的TCP_CHECK後面一定要有空格，是TCP_CHECK { ，TCP_CHECK與{ 之間存在空格，而不是TCP_CHECK{ ，中間沒有空格，就會出現以上的錯誤。

LVS+Keepalived+Apache