Nginx+keepalived雙機熱備
Nginx是一款輕量級的Web 服務器/反向代理服務器及電子郵件(IMAP/POP3)代理服務器,並在一個BSD-like 協議下發行。其特點是占有內存少,並發能力強,事實上nginx的並發能力確實在同類型的網頁服務器中表現較好,中國大陸使用nginx網站用戶有:百度、京東、新浪、網易、騰訊、淘寶等。
二、測試環境
下面拿2臺虛擬機進行環境測試,實驗環境為centos6.6 x86_64,具體用途和ip如下
服務器類型 | IP地址 |
Keepalived vip | 192.168.214.70 |
Nginx1 | 192.168.214.76 |
Nginx2 | 192.168.214.77 |
三、安裝nginx
這邊給大家使用源碼包來安裝
groupadd nginx
useradd -g nginx -s /sbin/nologin nginx
mkdir /var/log/nginx&&chown -R nginx:nginx /var/log/nginx
mkdir /usr/local/www&&chown -R nginx:nginx /usr/local/www
安裝nginx所需pcre庫
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz
tar zxvf pcre-8.38.tar.gz
cd pcre-8.38
./configure
make && make install
wget http://nginx.org/download/nginx-1.14.0.tar.gz
cd nginx-1.14.0
./configure --prefix=/usr/local/nginx--user=nginx --group=nginx \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_flv_module \
--with-http_gzip_static_module
make && make install
ln -s /usr/local/lib/libpcre.so.1 /lib64
使用/usr/local/nginx/sbin/nginx來啟動nginx服務
訪問下默認nginx頁面
四、修改linux文件句柄數
使用ulimit -a查看默認為1024
open files (-n) 1024
1、直接在終端修改 ulimit -SHn 65535
2、修改linux系統參數
vi /etc/security/limits.conf 添加保存後,重啟機器就永久生效了
* soft nofile 65535
* hard nofile 65535
五、配置nginx
這邊給大家提供了一個最簡單的配置文件給大家參考
cat /usr/local/nginx/conf/nginx.conf user nginx nginx; worker_processes 8; error_log logs/error.log; #error_log logs/error.log notice; #error_log logs/error.log info; pid /usr/local/nginx/nginx.pid; worker_rlimit_nofile 65535; events { use epoll; worker_connections 65535; } http { include mime.types; default_type application/octet-stream; #charset gb2312; server_names_hash_bucket_size 128; client_header_buffer_size 32k; large_client_header_buffers 4 32k; client_max_body_size 8m; #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' # '$status $body_bytes_sent "$http_referer" ' # '"$http_user_agent" "$http_x_forwarded_for"'; #access_log logs/access.log main; sendfile on; tcp_nopush on; #keepalive_timeout 0; keepalive_timeout 60; tcp_nodelay on; gzip on; gzip_min_length 1k; gzip_buffers 4 16k; gzip_http_version 1.0; gzip_comp_level 2; gzip_types text/plain application/x-javascript text/css application/xml; gzip_vary on; server { listen 80; server_name 192.168.214.76; index index.html index.htm; root /usr/local/www; #charset koi8-r; location /nginx_status { stub_status on; access_log off; } access_log /var/log/nginx/$server_name.log combined; } }
六、安裝keepalived
yum install -y keepalived
chkconfig keepalived on
註:在centos7系列系統中開機自動啟動使用systemctl enable keepalived
七、keepalived文件配置
查看192.168.214.76主keepalived配置
cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server mail.test.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_http_port { script "/usr/local/scripts/nginx_pid.sh" interval 2 weight 2 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } track_interface { eth0 } track_script { chk_http_port } virtual_ipaddress { 192.168.214.70 } }
查看192.168.214.77備的keepalived配置
! Configuration File for keepalived global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server mail.test.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_http_port { script "/usr/local/scripts/nginx_pid.sh" interval 2 weight 2 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 99 advert_int 1 authentication { auth_type PASS auth_pass 1111 } track_interface { eth0 } track_script { chk_http_port } virtual_ipaddress { 192.168.214.70 } }
最後附上Nginx的檢測腳本,當主服務器探測到nginx服務停止後,嘗試開啟nginx服務,失敗後關閉主服務器的keepalived服務,把vip切換到備上
cat nginx_pid.sh #!/bin/bash A=`ps -C nginx --no-header |wc -l` if [ $A -eq 0 ];then /usr/local/nginx/sbin/nginx sleep 3 if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then killall keepalived fi fi
八、啟動keepalived服務及查看相關信息
在192.168.214.76上通過ip addr 查看,vip192.168.214.70已經綁定在eth0網口上了
[root@localhost scripts]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000
link/ether 52:54:00:55:b2:d4 brd ff:ff:ff:ff:ff:ff
inet 192.168.214.76/24 brd 192.168.214.255 scope global eth0
inet 192.168.214.70/32 scope global eth0
inet6 fe80::5054:ff:fe55:b2d4/64 scope link
valid_lft forever preferred_lft forever
3: eth1:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000
link/ether 52:54:00:85:11:95 brd ff:ff:ff:ff:ff:ff
inet 192.168.211.76/24 brd 192.168.211.255 scope global eth1
inet6 fe80::5054:ff:fe85:1195/64 scope link
valid_lft forever preferred_lft forever
在214.76上查看日誌信息,看到已成功進入keepalived主機模式
[root@localhost scripts]# tail -f/var/log/messages
May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Registering Kernel netlink command channel
May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Opening file '/etc/keepalived/keepalived.conf'.
May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Configuration is using : 7681 Bytes
May 17 16:29:00 localhostKeepalived_healthcheckers[1747]: Using LinkWatch kernel netlink reflector...
May 17 16:29:01 localhost Keepalived_vrrp[1748]:VRRP_Instance(VI_1) Transition to MASTER STATE
May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) EnteringMASTER STATE
May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) setting protocol VIPs.
May 17 16:29:02 localhostKeepalived_vrrp[1748]: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 192.168.214.70
May 17 16:29:02 localhostKeepalived_healthcheckers[1747]: Netlink reflector reports IP 192.168.214.70 added
May 17 16:29:07 localhost Keepalived_vrrp[1748]:VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.214.70
在214.77上查看日誌信息,看到已成功進入keepalived備機模式
May 17 16:29:06 localhostKeepalived_vrrp[22914]: Opening file '/etc/keepalived/keepalived.conf'.
May 17 16:29:06 localhost Keepalived_vrrp[22914]:Configuration is using : 66478 Bytes
May 17 16:29:06 localhostKeepalived_vrrp[22914]: Using LinkWatch kernel netlink reflector...
May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) EnteringBACKUP STATE
May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP sockpool: [ifindex(2), proto(112), unicast(0),fd(10,11)]
May 17 16:29:06 localhostKeepalived_healthcheckers[22912]: Using LinkWatch kernel netlink reflector...
九、keepalived測試
使用vip192.168.214.70訪問nginx
最後,我們模擬下192.168.214.76nginx服務宕機,且無法通過腳本恢復,看下vip地址是否會漂移過去,nginx頁面是否能正常訪問。
通過ip addr命令看到vip已漂移到了192.168.214.77nginx服務器上,成功實現了熱備。
[root@localhost scripts]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000
link/ether 52:54:00:1b:a2:11 brd ff:ff:ff:ff:ff:ff
inet 192.168.214.77/24 brd 192.168.214.255 scope global eth0
inet 192.168.214.70/32 scope global eth0
inet6 fe80::5054:ff:fe1b:a211/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP>mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:64:47:7d brd ff:ff:ff:ff:ff:ff
inet 192.168.211.77/24 brd 192.168.211.255 scope global eth1
inet6 fe80::5054:ff:fe64:477d/64 scope link
valid_lft forever preferred_lft forever
在214.77上查看日誌信息,看到從之前的備機模式已成功進入keepalived主機模式
[root@localhost scripts]# tail -f/var/log/messages
May 17 16:29:06 localhostKeepalived_vrrp[22914]: Using LinkWatch kernel netlink reflector...
May 17 16:29:06 localhost Keepalived_vrrp[22914]:VRRP_Instance(VI_1) Entering BACKUP STATE
May 17 16:29:06 localhostKeepalived_vrrp[22914]: VRRP sockpool: [ifindex(2), proto(112), unicast(0),fd(10,11)]
May 17 16:29:06 localhostKeepalived_healthcheckers[22912]: Using LinkWatch kernel netlink reflector...
May 17 16:49:47 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Transitionto MASTER STATE
May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) EnteringMASTER STATE
May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) setting protocol VIPs.
May 17 16:49:48 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.214.70
May 17 16:49:48 localhostKeepalived_healthcheckers[22912]: Netlink reflector reports IP 192.168.214.70added
May 17 16:49:53 localhostKeepalived_vrrp[22914]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for192.168.214.70
如果想了解更多,請關註我們的公眾號
公眾號ID:opdevos
掃碼關註
Nginx+keepalived雙機熱備