Keepalived+Haproxy負載均衡
HAProxy是一個使用C語言編寫的自由及開放源代碼軟件,其提供高可用性、負載均衡,以及基於TCP和HTTP的應用程序代理。 HAProxy特別適用於那些負載特大的web站點,這些站點通常又需要會話保持或七層處理。HAProxy運行在當前的硬件上,完全可以支持數以萬計的並發連接。並且它的運行模式使得它可以很簡單安全的整合進您當前的架構中, 同時可以保護你的web服務器不被暴露到網絡上。
環境:
HAProxy服務器:192.168.200.101、192.168.200.102
虛擬服務器(VIP):192.168.200.100、192.168.200.110
DNS輪詢: 192.168.200.100、192.168.200.110
1. 安裝HAProxy:#兩節點HAProxy配置一致
wget http://www.haproxy.org/download/1.7/src/haproxy-1.7.8.tar.gz
tar xzvf haproxy-1.7.8.tar.gz
cd haproxy-1.7.8/
make TARGET=linux31 CPU=x86_64 PREFIX=/usr/local/haproxy
#uname -r中查看內核版本號為 3.10.0-514.el7.x86_64 則TARGET=linux31 CPU=x86_64
make install PREFIX=/usr/local/haproxy
創建haproxy用戶和組
groupadd haproxy
useradd -g haproxy -s /sbin/nologin haproxy
創建配置文件
mkdir /usr/local/haproxy/conf/
vim /usr/local/haproxy/conf/haproxy.cfg
#全局配置 global #設置日誌 log 127.0.0.1 local3 info #haproxy安裝目錄 chroot /usr/local/haproxy #用戶與用戶組 user haproxy group haproxy #守護進程方式啟動 daemon #進程數量 nbproc 1 #每個進程最大連接數 maxconn 65535 #默認設置 defaults log global #模式(tcp:三層|http:七層|health:只返回ok) mode http option httplog #服務端保持長連接 option http-pretend-keepalive #不記錄健康檢查日誌 option dontlognull #每次請求完畢後關閉http通道 option httpclose #服務端響應後主動關閉請求連接,不檢查客戶端應答確認 option forceclose #如果後端服務器宕機,強制切換到其他服務器 option redispatch #丟棄由於客戶端等待時間過長但仍在等待隊列中的請求 option abortonclose #傳遞client端的IP地址給server端,並寫入“X-Forward_for”首部中 option forwardfor except 127.0.0.0/8 #記錄客戶端訪問的目的地IP option originalto #同一IP地址的所有請求發送到同一服務器 balance source #三次連接失敗,判斷服務不可用 retries 3 #檢測超時時間 timeout check 5s #http請求超時時間 timeout http-request 5s #一個請求在隊列中超時時間 timeout queue 10s timeout connect 5000 timeout client 50000 timeout server 50000 #前端配置,http_front名稱自定義 frontend http_front bind *:80 mode http option httplog option dontlognull option httpclose #acl規則 #創建一個acl acl_http_www.a.com, 用於判斷主機名是否為www.a.com,-i 忽略大小寫 acl acl_www.a.com hdr_end(host) -i www.a.com acl acl_bbs.a.com hdr_end(host) -i bbs.a.com #判斷ua是否是Android acl acl_m.a.com hdr_reg(User-Agent) -i android #判斷url文件的結尾 acl acl_path_end path_end -i .php .php5 #如果acl_www.a.com規則被觸發,則將客戶端請求分發到web1 use_backend web1 if acl_www.a.com use_backend web2 if acl_bbs.a.com use_backend m if acl_m.a.com use_backend php if acl_path_end #默認頁面defailt_site default_backend default_site #haproxy統計頁面 listen admin_stats bind 0.0.0.0:8080 stats enable mode http #統計頁面url stats uri /haproxy?stats #登錄頁面提示信息 stats realm "haproxy status page" #用戶名 密碼 stats auth admin:admin #隱藏版本信息 stats hide-version #通過認證才能管理 stats admin if TRUE #自動刷新時間 stats refresh 10s default_backend http_back #後端配置,http_back名稱自定義 backend http_back #負載均衡模式 #source 根據源IP #static-rr 根據權重 #leastconn 最少連接優先處理 #url 根據請求url #url_param 根據請求url參數 #rdp-cookie 根據cookie(name)來鎖定並hash每一次請求 #hdr(name) 根據http請求頭來鎖定每一次http請求 #roundrobin 輪詢方式 balance roundrobin #設置健康檢查頁面 option httpchk GET /index.html #傳遞客戶端真實IP option forwardfor header X-Forwarded-For #inter 2000 健康檢查時間間隔2秒 #rise 3 檢測3次認為正常運行 #fall 3 失敗3次認為不可用 #weight 30 權重30 backend web1 server web11 192.168.200.108:80 check inter 2000 rise 3 fall 3 weight 30 server web12 192.168.200.103:80 check inter 2000 rise 3 fall 3 weight 30 backend web2 server web21 192.168.200.104:80 check inter 2000 rise 3 fall 3 weight 30 server web22 192.168.200.105:80 check inter 2000 rise 3 fall 3 weight 30 backend php server php1 192.168.200.106:80 check inter 2000 rise 3 fall 3 weight 30 backend m server m1 192.168.200.107:80 check inter 2000 rise 3 fall 3 weight 30 backend default_site server web00 192.168.200.109:80 check inter 2000 rise 3 fall 3 weight 30
日誌設置
vim /etc/rsyslog.conf
15 $ModLoad imudp
16 $UDPServerRun 514
92 local3.* /var/log/haproxy
systemctl restart rsyslog
啟動
/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/conf/haproxy.cfg
開機啟動
echo "/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/conf/haproxy.cfg" >> /etc/rc.d/rc.local
chmod +x /etc/rc.d/rc.local
安裝keepalived
yum install -y libnl-devel popt-devel openssl-devel popt-static libnfnetlink libnfnetlink-devel kernel-devel
reboot
cd /usr/local/src
wget http://www.keepalived.org/software/keepalived-1.3.4.tar.gz
tar -xzvf keepalived-1.3.4.tar.gz
cd keepalived-1.3.4
./configure
make && make install
編譯的時候出現下面的提示,說明keepalived和內核結合了,否則請檢查是否安裝上面的軟件包
cp /usr/local/src/keepalived-1.3.4/keepalived/etc/init.d/keepalived /etc/init.d/
cp /usr/local/src/keepalived-1.3.4/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp /usr/local/src/keepalived-1.3.4/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/src/keepalived-1.3.4/bin/keepalived /usr/sbin/
配置keepalived-MASTER
vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server smtp.domain.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_http_port {
script "/etc/keepalived/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state MASTER
interface ens32
virtual_router_id 51
mcast_src_ip 192.168.200.101
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_http_port
}
virtual_ipaddress {
192.168.200.100
}
}
vrrp_instance VI_2 {
state BACKUP
interface ens32
virtual_router_id 52
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.200.110
}
}
配置keepalived-BACKUP
global_defs {
notification_email {
}
notification_email_from [email protected]
smtp_server smtp.domain.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_http_port {
script "/etc/keepalived/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state BACKUP
interface ens32
virtual_router_id 51
mcast_src_ip 192.168.200.102
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_http_port
}
virtual_ipaddress {
192.168.200.100
}
}
vrrp_instance VI_2 {
state BACKUP
interface ens32
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.200.110
}
}
腳本(兩節點相同)
vim /etc/keepalived/check_haproxy.sh
#!/bin/bash
if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
/etc/init.d/haproxy start
fi
sleep 2
if [ $(ps -C haproxy --no-header | wc -l) -eq 0 ]; then
/etc/init.d/keepalived stop
fi
啟動
systemctl start keepalived
systemctl enable keepalived
ip addr | grep "192.168.200"
inet 192.168.200.101/24 brd 192.168.200.255 scope global ens32
inet 192.168.200.100/32 scope global ens32
ip addr | grep "192.168.200"
inet 192.168.200.102/24 brd 192.168.200.255 scope global ens32
inet 192.168.200.110/32 scope global ens32
當其中一臺主機宕機後:
ip addr | grep "192.168.200"
inet 192.168.200.101/24 brd 192.168.200.255 scope global ens32
inet 192.168.200.100/32 scope global ens32
inet 192.168.200.110/32 scope global ens32
Keepalived+Haproxy負載均衡