1. 程式人生 > >MySQL架構之keepalived+haproxy+mysql

MySQL架構之keepalived+haproxy+mysql

MySQL的高可用方案一般有如下幾種:
keepalived+雙主,MHA,PXC,MMM,Heartbeat+DRBD等,比較常用的是keepalived+雙主,MHA和PXC。

HAProxy是一款免費的提供高可用性、負載均衡以及基於TCP(第四層)和HTTP(第七層)應用的代理軟體,藉助HAProxy可以快速並且可靠的提供基於TCP和HTTP應用的代理解決方案。
Keepalived主要作用是LoadBalance master和LoadBalance backup之間的健康檢查,實現故障轉換。
Mysql Replication主要作用是提高mysql並處理資料的能力以及實現容災備份的作用。

架構規劃:
keepalived+haproxy+mysql

伺服器及其IP地址規劃:
240 mysql_b2 192.168.0.240
242 mysql_b1 192.168.0.242
247 haproxy1 192.168.0.247
249 haproxy2 192.168.0.249
243 VIP          192.168.0.243
248 client       192.168.0.248

軟體版本:
mysql:8.0.12
keepalived:2.0.8
haproxy:1.8.14
mydumper:0.9.1

軟體下載連結:
wget http://fossies.org/linux/misc/haproxy-1.8.14.tar.gz
wget http://www.keepalived.org/software/keepalived-2.0.8.tar.gz

伺服器準備:

docker network create --subnet=192.168.0.0/16 staticnet
docker run -d --privileged -v `pwd`/mysql_data:/data -p 3000:3306 --name mysql_b2 --hostname mysql_b2 --net staticnet --ip 192.168
.0.240 eiki/mysql:8.0.12 /usr/sbin/init docker run -d --privileged -v `pwd`/mysql_data:/data -p 4000:3306 --name mysql_b1 --hostname mysql_b1 --net staticnet --ip 192.168.0.242 eiki/mysql:8.0.12 /usr/sbin/init docker run -d --privileged -v `pwd`/mysql_data:/data -p 5001:8899 --name haproxy1 --hostname haproxy1 --net staticnet --ip 192.168.0.247 eiki/centos:7.5 /usr/sbin/init docker run -d --privileged -v `pwd`/mysql_data:/data -p 5002:8899 --name haproxy2 --hostname haproxy2 --net staticnet --ip 192.168.0.249 eiki/centos:7.5 /usr/sbin/init

Mysql主從複製配置:

過程略,參考請見https://www.cnblogs.com/EikiXu/p/9811764.html
MyDumper的安裝,參考https://www.cnblogs.com/EikiXu/p/9816058.html

1.在master伺服器上建立mysql使用者(授權複製賬戶):
192.168.0.242:

create user 'replica'@'%' IDENTIFIED BY 'rooT_000';
grant REPLICATION SLAVE ON *.* TO 'replica'@'%';

2.編輯master伺服器的mysql配置檔案my.cnf:

[mysql]
prompt = [\\[email protected]\\h][\\d]>\\_
port = 3306
socket = /usr/local/mysql/data/mysql.sock

[mysqld]
server-id = 1 ###指定伺服器的ID
port = 3306
mysqlx_port = 33060
mysqlx_socket = /usr/local/mysql/data/mysqlx.sock
datadir = /usr/local/mysql/data
socket = /usr/local/mysql/data/mysql.sock
pid-file = /usr/local/mysql/data/mysqld.pid
log-error = error.log
slow-query-log = 1
slow-query-log-file = slow.log
long_query_time = 0.2
log-bin = mysql-bin ###開啟二進位制日誌
relay-log = relay.log
binlog_format =ROW
relay_log_recovery = 1
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
init_connect ='SET NAMES utf8mb4'
innodb_buffer_pool_size = 1G
join_buffer_size = 128M
sort_buffer_size = 2M
read_rnd_buffer_size = 2M
log_timestamps = SYSTEM
lower_case_table_names = 1
default-authentication-plugin =mysql_native_password
max_allowed_packet = 500M ###控制其通訊緩衝區的最大長度


plugin_load = "validate_password.so;rpl_semi_sync_master=semisync_master.so;rpl_semi_sync_slave=semisync_slave.so"
rpl_semi_sync_master_enabled = 0
rpl_semi_sync_master_timeout = 1000


auto_increment_increment=2
auto_increment_offset=1

binlog-ignore = mysql //忽略mysql和information_schema 資料庫
binlog-ignore = information_schema
binlog-do-db = blog //同步資料庫,默認同步所有資料庫

replicate-do-db=test

3.檢視master狀態;

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 | 1642 | | | |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.04 sec)

4.在slave端建立資料庫blog,匯出master端的blog庫,匯入到此庫,並修改mysql主配置檔案my.cnf server-id= 10 重啟mysql資料庫
192.168.0.240:
mydumper -h 192.168.0.242 --regex="test.*" -u root -p 'abc123' -r 300000 -G -E -R -v 3 -t 5 -o /data/backup20181031 > /data/mydumper20181031.log 2>&1 &
myloader -S /usr/local/mysql/data/mysql.sock -u root -p 'abc123' -v 3 -t 10 -e y -d /data/backup20181031 > /data/myloader20181031.log 2>&1 &

CHANGE MASTER TO
MASTER_HOST='192.168.0.242',
MASTER_USER='replica',
MASTER_PASSWORD='rooT_000',
MASTER_LOG_FILE='mysql-bin.000001',
MASTER_LOG_POS=1642;

5.mysql主從同步測試,show slave status\G;能看到Slave_IO_Running和Slave_SQL_Running都為YES即可。

[[email protected]][(none)]> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.0.242
Master_User: replica
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000002
Read_Master_Log_Pos: 441
Relay_Log_File: relay.000007
Relay_Log_Pos: 655
Relay_Master_Log_File: mysql-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB: test,test
Replicate_Ignore_DB: 
Replicate_Do_Table: 
Replicate_Ignore_Table: 
Replicate_Wild_Do_Table: 
Replicate_Wild_Ignore_Table: 
Last_Errno: 0
Last_Error: 
Skip_Counter: 0
Exec_Master_Log_Pos: 441
Relay_Log_Space: 1020
Until_Condition: None
Until_Log_File: 
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File: 
Master_SSL_CA_Path: 
Master_SSL_Cert: 
Master_SSL_Cipher: 
Master_SSL_Key: 
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error: 
Last_SQL_Errno: 0
Last_SQL_Error: 
Replicate_Ignore_Server_Ids: 
Master_Server_Id: 1
Master_UUID: 3b27e7d5-d5d9-11e8-8004-0242ac110002
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind: 
Last_IO_Error_Timestamp: 
Last_SQL_Error_Timestamp: 
Master_SSL_Crl: 
Master_SSL_Crlpath: 
Retrieved_Gtid_Set: 
Executed_Gtid_Set: 
Auto_Position: 0
Replicate_Rewrite_DB: 
Channel_Name: 
Master_TLS_Version: 
Master_public_key_path: 
Get_master_public_key: 0
1 row in set (0.01 sec)

Haproxy安裝及其配置,master和backup安裝完全一樣的,配置部分差異:


useradd -M -s /sbin/nologin haproxy ###建立haproxy程式使用者以及配置檔案目錄
wget https://www.haproxy.org/download/1.8/src/haproxy-1.8.14.tar.gz
tar-zxvf haproxy-1.8.14.tar.gz
cd haproxy-1.8.14
make TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 PREFIX=/usr/local/haproxy
make install PREFIX=/usr/local/haproxy
cd /usr/local/haproxy
mkdir conf logs //在此目錄下面建立conf,logs目錄分別存放HAproxy的配置檔案,PID檔案和日誌檔案。

建立使用者haproxy
grant all privileges on *.* to 'haproxy'@'%' identified by 'haproxy';
flush privileges;

vim /usr/local/haproxy/conf/haproxy.cfg

global
log /dev/log local0 info
log /dev/log local1 notice
user haproxy
group haproxy
defaults
log global
retries 2
timeout connect 3000
timeout server 5000
timeout client 5000
listen mysql-cluster
bind 0.0.0.0:3306
mode tcp
#option mysql-check user haproxy
option tcp-check
balance roundrobin
server mysql1 192.168.0.242:3306 check weight 2
server mysql2 192.168.0.240:3306 check weight 2
listen mysql-clusterstats
bind 0.0.0.0:8899
mode http
stats enable
stats uri /
stats realm Strictly\ Private
stats auth status:mypass
stats auth admin:mysqladmin #狀態檢視頁面登陸帳號密碼

vim /etc/rsyslog.d/haproxy.conf

if ($programname == 'haproxy' and $syslogseverity-text == 'info') then -/var/log/haproxy181/haproxy181-info.log
& ~
if ($programname == 'haproxy' and $syslogseverity-text == 'notice') then -/var/log/haproxy181/haproxy181-notice.log
& ~

/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg -c ###驗證引數檔案配置有效性

/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg ###啟動haproxy

Haproxy開啟系統日誌支援。

vim /etc/syslog.conf

#新增:
local3.* /var/log/haproxy.log
local0.* /var/log/haproxy.log
vim /etc/sysconfig/syslog
#修改:
SYSLOGD_OPTIONS="-r -m 0"

#重新啟動syslog服務

/etc/init.d/syslogrestart


[[email protected] haproxy-1.8.9]# ss -antulp | grep haproxy
配置haproxy開機自啟動
# chmod +x /etc/rc.d/rc.local
echo '/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg' >> /etc/rc.d/rc.local


為了方便系統在開機時載入,還可以建立啟動指令碼:
vim /etc/rc.d/init.d/haproxy

#haproxy啟動指令碼
#!/bin/bash
BASE_DIR="/usr/local/haproxy"
ARGV="[email protected]"
start()
{
echo"START HAPoxy SERVERS"
$BASE_DIR/sbin/haproxy-f $BASE_DIR/conf/haproxy.conf
}
stop()
{
echo"STOP HAPoxy Listen"
kill-TTOU $(cat$BASE_DIR/logs/haproxy.pid)
echo"STOP HAPoxy process"
kill-USR1 $(cat$BASE_DIR/logs/haproxy.pid)
}
case$ARGV in
start)
start
ERROR=$?
;;
stop)
stop
ERROR=$?
;;
restart)
stop
start
ERROR=$?
;;
*)
echo"hactl.sh [start|restart|stop]"
esac
exit$ERROR

[[email protected] src]# chmod +x /etc/rc.d/init.d/haproxy

[[email protected] src]# chkconfig --add haproxy

登入地址:
http://192.168.0.247:8899
檢視全部伺服器是不是都是UP
-------測試是否是負載平衡
for i in $(seq 1 10); do mysql -uroot -pabc123 -h192.168.0.243 -e 'select @@server_id;'; done | egrep '[0-9]'

停止服務:
# killall haproxy


keepalived 安裝:

安裝依賴包:
yum -y install kernel kernel-devel* popt popt-devel libssl-dev libnl libnl-devel openssl openssl-* ipvsadm libnfnetlink-devel

詳細安裝步驟:
tar xzvf keepalived-2.0.8.tar.gz -C /usr/local/src
cd /usr/local/src/keepalived-2.0.8/
./configure --prefix=/usr/local/keepalived --sysconf=/etc
make && make install
cp /usr/local/src/keepalived-2.0.8/keepalived/etc/init.d/keepalived /etc/init.d/
cp /usr/local/src/keepalived-2.0.8/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/src/keepalived-2.0.8/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

chkconfig --add keepalived
chkconfig --level 2345 keepalived on
vi /etc/keepalived/keepalived.conf

route add 192.168.0.0 mask 255.255.255.0 10.200.22.128

keepalived主:

global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server root
smtp_connect_timeout 30
router_id NodeA
}
vrrp_script check_haproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
interval 3 #指令碼執行間隔
weight 2 #執行指令碼後優先順序變更:5表示優先順序+5;-5則表示優先順序-5
}

vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.243
}
track_script {
check_haproxy
}
notify_master /etc/keepalived/scripts/state_master.sh
notify_backup /etc/keepalived/scripts/state_backup.sh
notify_fault /etc/keepalived/scripts/state_fault.sh
}

  

keepalived從:

global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server root
smtp_connect_timeout 30
router_id NodeA
}
vrrp_script check_haproxy {
script "/etc/keepalived/scripts/check_haproxy.sh"
interval 3 #指令碼執行間隔
weight 2 #執行指令碼後優先順序變更:5表示優先順序+5;-5則表示優先順序-5
}

vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.243
}
track_script {
check_haproxy
}
notify_master /etc/keepalived/scripts/state_master.sh
notify_backup /etc/keepalived/scripts/state_backup.sh
notify_fault /etc/keepalived/scripts/state_fault.sh
}

  

vi /etc/keepalived/scripts/check_haproxy.sh

#!/bin/bash
#nginx="/usr/local/haproxy/sbin/haproxy"

PID=`ps -C haproxy --no-heading|wc -l`
if [ "${PID}" = "0" ];
then
/etc/init.d/haproxy start
sleep 1
LOCK=`ps -C haproxy --no-heading|wc -l`
if [ "${LOCK}" = "0" ];
then
/etc/init.d/keepalived stop
fi
fi

 

vi /etc/keepalived/scripts/state_master.sh

#!/bin/bash
echo -e >> $LOGFILE
host=haproxy01 #設定當前的主機名
LOGFILE="/var/log/keepalived-state.log"
echo "[Master]" >> $LOGFILE
date >> $LOGFILE
echo "The ${host} Starting to become master server...." >> $LOGFILE 2>&1

echo "Please run the “ipvsadm -Ln” check the keepalived state ..." >> $LOGFILE
echo ".........................................................................!">> $LOGFILE
echo >>$LOGFILE

vi /etc/keepalived/scripts/state_backup.sh

#!/bin/bash
echo -e >> $LOGFILE
host=haproxy02 #設定當前的主機名
LOGFILE="/var/log/keepalived-state.log"
echo "[Backup]" >> $LOGFILE
date >> $LOGFILE
echo "The ${host} Starting to become Backup server...." >> $LOGFILE 2>&1

echo "Please run the “ipvsadm -Ln” check the state ..." >> $LOGFILE
echo "........................................................................!">> $LOGFILE
echo >> $LOGFILE

vi /etc/keepalived/scripts/state_fault.sh

#!/bin/bash
echo -e >> $LOGFILE
host=haproxy01 #設定當前的主機名
LOGFILE="/var/log/keepalived-state.log"
echo "[fault errot ]" >> $LOGFILE
date >> $LOGFILE
echo "The ${host} is fault error...." >> $LOGFILE 2>&1
echo "Please check the server state ..." >> $LOGFILE
echo "........................................................................!">> $LOGFILE
echo >> $LOGFILE