|NO.Z.00012|——————————|^^ 構建 ^^|——|HAC構建.V3|-------------------------------------------|5臺server|
阿新 • • 發佈：2022-03-22
[Applications：HAC構建.V3] [Applications.LBC&HAC$HPC] [HAC|HA高可用|Keepalived原理|LVS+NAT+LVS健康檢查指令碼] [LVS-DR+keepalived|Heartbeat+Nginx|5臺server]
一、LVS-DR+keepalived模式：10.10.10.11部署Keepalived服務（LVS主master）
### --- 10.10.10.11部署Keepalived服務（LVS主master） ### --- 將Keepalived.iso檔案上傳到10.10.10.11伺服器的root目錄下 ### --- 建立一個/mnt/iso1目錄用於掛載Keepalived.iso檔案 [root@server11 ~]# mkdir /mnt/iso1 [root@server11 ~]# mount -o loop Keepalived.iso /mnt/iso1/ ~~~ 將所有資料拷貝到當前目錄 [root@server11 ~]# cp -a /mnt/iso1/* . // 將所有資料拷貝到當前目錄 [root@server11 ~]# ls keepalived-1.2.2.tar.gz rkhunter-1.3.8.tar.gz // 得到Keepalived的原始碼包
### --- 需要原始碼編譯安裝；安裝 gcc環境 [root@server11 ~]# yum install -y gcc gcc-c++ ~~~ 原始碼編譯安裝Keepalived [root@server11 ~]# tar -zxvf keepalived-1.2.2.tar.gz [root@server11 ~]# cd keepalived-1.2.2 ~~~ 安裝相關依賴 [root@server11 keepalived-1.2.2]# yum install -y kernel-devel openssl-devel popt-devel gcc*
~~~ 生成Makefile檔案 [root@server11 keepalived-1.2.2]# ./configure --prefix=/ --with-kernel-dir=/usr/src/kernels/2.6.32-642.el6.x86_64/ Keepalived configuration ------------------------ Keepalived version : 1.2.2 Compiler : gcc Compiler flags : -g -O2 Extra Lib : -lpopt -lssl -lcrypto Use IPVS Framework : Yes IPVS sync daemon support : Yes IPVS use libnl : No Use VRRP Framework : Yes Use Debug flags : No [root@server11 keepalived-1.2.2]# make && make install
~~~ 把啟動指令碼做軟連線，設定為開機自啟
[root@server11 keepalived-1.2.2]# ll /etc/init.d/keepalived
-rwxr-xr-x 1 root root 1288 Jan 21 00:35 /etc/init.d/keepalived
~~~ 新增Keepalived自啟規則
[root@server11 keepalived-1.2.2]# chkconfig --add keepalived // 新增Keepalived自啟規則
[root@server11 keepalived-1.2.2]# chkconfig keepalived on // 設定開機自啟### --- 修改Keepalived配置檔案
[root@server11 keepalived-1.2.2]# vim /etc/keepalived/keepalived.conf
global_defs {
# notification_email { // 刪除
# [email protected]
# [email protected]
# [email protected]
# }
# notification_email_from [email protected]
# smtp_server 192.168.200.1
# smtp_connect_timeout 30
router_id R1 // 保留router_id,可以這個叢集服務在這臺裝置的別名設定主機的名稱，可以自己定義，但是同一個組下名稱不可以相同。
vrrp_instance VI_1 {
state MASTER // 設定是主伺服器，還是從伺服器，master是主伺服器
interface eth0 // 通過eth0網絡卡做心跳檢測
virtual_router_id 66 // 以及虛擬的組，只有在同一個組裡，它才是同一個高可用環境，組ID必須一致
priority 80 // 權重：最大不超過150，組合組之間最好差距為50，切換可能會更順暢一點。官方建議；寫80，留一定的餘地
advert_int 1 // 檢測間隔為1秒
authentication { // 認證：主和從之間的需要填寫一定的身份認證，此處為密碼認證
auth_type PASS // 認證賬戶：
auth_pass 1111 // 認證密碼
}
virtual_ipaddress { // 幾群的IP地址
10.10.10.100 // 此環境只有一個叢集~~~ Keepalived以上為虛擬路由相關的協議設定OK
~~~ Keepalived以下為 虛擬服務的設定
virtual_server 10.10.10.100 80 { // 叢集相關的引數；地址:埠號80：10.10.10.100:80
delay_loop 6 // 檢測迴圈次數
lb_algo rr // 當前使用的演算法rr
lb_kind DR // 當前的模式為DR模式
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP // 當前的檢測方案為TCP連線模式
real_server 192.168.201.100 443 { // 真實伺服器
weight 1 // 權重，此環境使用的是r，沒有用wr，所以寫1：1即可
TCP_CHECK {
connect_port 80 // TCP的檢測方案，檢測的埠是80
connect_timeout 3 // 檢測的操作時間是3秒
nb_get_retry 3 // 重試3次
delay_before_retry 4 // 重試以後間隔4秒 ### --- 啟動Keepalived服務
[root@server11 keepalived-1.2.2]# service keepalived start
Starting keepalived: [ OK ]
~~~ 通過日誌去檢視一下
[root@server11 keepalived-1.2.2]# cat /var/log/messages
Jan 21 01:21:47 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.10.10.100 //ARP監聽到eth0，使用的是100地址
Jan 21 01:21:52 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.10.10.100 ### --- keepalived.conf配置檔案標準，上面為引數說明
[root@server11 keepalived-1.2.2]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id R1 // 設定當前裝置的名稱
}
vrrp_instance VI_1 { // 設定伺服器當前的名稱，
state MASTER
interface eth0
virtual_router_id 66
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress { //虛擬IP的地址
10.10.10.100
}
}
virtual_server 10.10.10.100 80 { //設定虛擬網路
delay_loop 2
lb_algo rr
lb_kind DR
protocol TCP
real_server 10.10.10.13 80 { //真實伺服器RS1
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 4
}
}
real_server 10.10.10.14 80 { //真實伺服器RS2
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 4
}
}
} ### --- 10.10.10.12部署Keepalived服務(LVS主master)
### --- 新增路由優化方案：關閉網絡卡的重定向功能
[root@server12 ~]# vim /etc/sysctl.conf
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.eth0.send_redirects = 0
[root@server12 ~]# sysctl -p### --- 啟動10.10.10.100心跳IP地址
~~~ 第一個問題：因為10.10.10.100在HA-server1中已經啟動了，在HA-server2中是起不來的；能不能啟動時在網絡卡的啟動指令碼設定的，只需要把網絡卡啟動指令碼的拒絕給關閉即可。
~~~ 第二個問題：同一個區域網下有兩個10.10.10.100地址，它會怎麼處理，它就會在HA-server1和HA-server2中來回切換，交換機會重新整理它的Mac地址
~~~ 第三個問題：為什麼要加入Keepalived：Keepalived會繫結你的ARP許可權，若是HA-server1下的10.10.10.100是存活狀態，HA-server2下的10.10.10.100是不允許被使用的。若是檢測到HA-server1下的10.10.10.100不在了，HA-server2就可以使用10.10.10.100的IP許可權了。就可以和外網進行通訊。
[root@server12 ~]# cd /etc/sysconfig/network-scripts/
[root@server12 network-scripts]# cp -a ifcfg-eth0 ifcfg-eth0:0
[root@server12 network-scripts]# vim !$
DEVICE=eth0:0
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.10.10.100
NETMASK=255.255.255.0### --- 故障現象：啟動網絡卡會報錯，已經有服務在使用該地址了
[root@server12 network-scripts]# ifup eth0:0
Determining if ip address 10.10.10.100 is already in use for device eth0...
Error, some other host (00:0C:29:FA:34:71) already uses address 10.10.10.100.
### --- 故障分析：
[root@server12 ~]# vim /etc/sysconfig/network-scripts/ifup-eth
~~~ ：256行尋找修改引數，不同版本中它的位置是不一樣的，找到如下內容
~~~ 上面引數表示通過ARP協議檢測在相同佇列裡有沒有相同IP資訊，若有相同就離開，返回值為1
if ! ARPING=$(/sbin/arping -c 2 -w ${ARPING_WAIT:-3} -D -I ${REALDEVICE} ${ipaddr[$idx]}) ; then
ARPINGMAC=$(echo $ARPING | sed -ne 's/.*\[\(.*\)\].*/\1/p')
net_log $"Error, some other host ($ARPINGMAC) already uses address ${ipaddr[$idx]}."
exit 1
### --- 解決方案：註釋掉這幾行內容
# if ! ARPING=$(/sbin/arping -c 2 -w ${ARPING_WAIT:-3} -D -I ${REALDEVICE} ${ipaddr[$idx]}) ; then
# ARPINGMAC=$(echo $ARPING | sed -ne 's/.*\[\(.*\)\].*/\1/p')
# net_log $"Error, some other host ($ARPINGMAC) already uses address ${ipaddr[$idx]}."
# exit 1
# fi
~~~ 註釋資訊後重新啟動ifcfg-eth0:0網絡卡
[root@server12 ~]# ifup eth0:0 // 啟動成功
Determining if ip address 10.10.10.100 is already in use for device eth0...
[root@server12 ~]# ifconfig
eth0:0 Link encap:Ethernet HWaddr 00:0C:29:DA:AF:AC
inet addr:10.10.10.100 Bcast:10.10.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1### --- 構建Keepalived：將Keepalived.iso上傳到伺服器
[root@server12 ~]# mkdir /mnt/iso1
[root@server12 ~]# mount -o loop Keepalived.iso /mnt/iso1/
[root@server12 ~]# cp -a /mnt/iso1/* .
[root@server12 ~]# tar -zxvf keepalived-1.2.2.tar.gz
[root@server12 ~]# cd keepalived-1.2.2### --- 安裝make編譯安裝依賴包並
[root@server12 keepalived-1.2.2]# yum install -y kernel-devel openssl-devel popt-devel gcc*
~~~ 生成makefile檔案
[root@server12 keepalived-1.2.2]# ./configure --prefix=/ --with-kernel-dir=/usr/src/kernels/2.6.32-642.el6.x86_64/
Keepalived configuration
------------------------
Keepalived version : 1.2.2
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lpopt -lssl -lcrypto
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
IPVS use libnl : No
Use VRRP Framework : Yes
Use Debug flags : No~~~ 安裝
[root@server12 keepalived-1.2.2]# make && make install
~~~ 啟動Keepalived
[root@server12 keepalived-1.2.2]# chkconfig --add keepalived
[root@server12 keepalived-1.2.2]# chkconfig keepalived on### --- 修改Keepalived.conf配置檔案
~~~ 直接在HA-server1下推入配置檔案
[root@server11 ~]# scp /etc/keepalived/keepalived.conf [email protected]:/etc/keepalived/keepalived.conf
[root@server12 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id R2 // 組號修改為R2,，同一個叢集下不可一直
}
vrrp_instance VI_1 {
state SLAVE // 名稱為SLAVE或者BACKUP
interface eth0
virtual_router_id 66 // 組號保持一致
priority 20 // 權重為20
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}### --- 安裝ipvsadm
[root@server12 ~]# yum install -y ipvsadm
[root@server12 ~]# service ipvsadm start
[root@server12 ~]# chkconfig ipvsadm on
### --- 啟動Keepalived服務
[root@server12 ~]# service keepalived start
Starting keepalived: [ OK ]### --- 驗證是否配置成功：
[root@server12 ~]# ipvsadm -Ln // 第一次沒有檢視到
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@server12 ~]# ipvsadm -Ln // 間隔兩秒重新查詢後出現
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.10.10.100:80 rr
-> 10.10.10.13:80 Route 1 0 0
-> 10.10.10.14:80 Route 1 0 0三、驗證：通過IE瀏覽器訪問http://10.10.10.100/ 四、通過命令驗證
### --- 驗證LVS-DR+keepalived負載排程
[root@server11 ~]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 10.10.10.100:80 42 210 0 30051 0
-> 10.10.10.13:80 21 105 0 15015 0
-> 10.10.10.14:80 21 105 0 15036 0~~~ 重新整理網頁；發現HA-server2的資料沒有發生變化，而HA-server1 的資料發生變化，說明HA-server1正在負載均衡
[root@server11 ~]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 10.10.10.100:80 56 280 0 40068 0
-> 10.10.10.13:80 28 140 0 20020 0
-> 10.10.10.14:80 28 140 0 20048 0~~~ 把HA-server1的網路斷掉;HA-server2的資料發生變化，說明HA-server2切換為負載排程器實現負載排程。
[root@server12 ~]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 10.10.10.100:80 82 410 0 58670 0
-> 10.10.10.13:80 11 55 0 7865 0
-> 10.10.10.14:80 11 55 0 7875 0### --- LVS-DR+keepalived結合後，Keepalived會後端的伺服器進行健康狀態檢查，停掉HA-server3服務的Apache服務，檢視狀態
[root@server13 ~]# service httpd stop
[root@server11 ~]# ipvsadm -Ln --stats //10.10.10.13服務已經不存在了
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 10.10.10.100:80 56 280 0 40068 0
-> 10.10.10.14:80 28 140 0 20048 0
[root@server13 ~]# service httpd start
[root@server11 ~]# ipvsadm -Ln --stats //10.10.10.13服務又重新恢復
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 10.10.10.100:80 56 280 0 40068 0
-> 10.10.10.13:80 0 0 0 0 0
-> 10.10.10.14:80 28 140 0 20048 0### --- 重點說明：
~~~ 在此環境下真實伺服器RS1/RS2宕機不會對業務造成中斷
~~~ LVS-M/LVS-S宕機不會對業務造成中斷
~~~ 在此環境下不管是哪個節點宕機都不會對業務造成中斷：
~~~ LVS-DR+keepalived高可用叢集構建完成。===============================END===============================
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart ——W.S.Landor
來自為知筆記(Wiz)