MySQL中介軟體proxysql實現讀寫分離
環境說明:
IP | 角色 | 應用 | 平臺 |
---|---|---|---|
192.168.161.96 | 讀寫分離解析主機 | proxysql | rhel7.4 |
192.168.161.164 | master | mysql | rhel7.4 |
192.168.161.167 | slave | mysql | rhel7.4 |
前提要求:
- 關閉防火牆
- 關閉selinux
- 安裝MySQL並配置主從
在192.168.161.96上做讀寫分離解析主機
安裝ProxySQL
//配置proxysql的yum源
[[email protected] ~]# cat <<EOF | tee /etc/yum.repos.d/proxysql.repo
> [proxysql_repo]
> name=ProxySQL
> baseurl=http://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/7
> gpgcheck=1
> gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key
> EOF
[proxysql_repo]
name=ProxySQL
baseurl=http://repo.proxysql.com/ProxySQL/proxysql-1.4.x/centos/7
gpgcheck=1
gpgkey=http://repo.proxysql.com/ProxySQL/repo_pub_key
//安裝proxysql
[ [email protected] ~]# yum -y install proxysql
//啟動proxysql並設定開機自動啟動
//proxysql預設只提供了sysv風格的啟動指令碼,所以想設定開機自啟則需藉助於chkconfig工具
[[email protected] ~]# systemctl start proxysql
[[email protected] ~]# chkconfig proxysql on
- mysql主庫新增proxysql可以增刪改查的賬號
mysql> grant all on *.* to 'proxysql'@'192.168.161.96' identified by 'pwproxysql''; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec)
登入proxysql管理端
安裝mariadb
[[email protected] ~]# yum -y install mariadb
[[email protected] ~]# export MYSQL_PS1="(\[email protected]\h:\p) [\d]> "
[[email protected] ~]# mysql -uadmin -padmin -h127.0.0.1 -P6032
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.5.30 (ProxySQL Admin Module)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
([email protected]:6032) [(none)]>
([email protected]:6032) [(none)]> show databases;
+-----+---------------+-------------------------------------+
| seq | name | file |
+-----+---------------+-------------------------------------+
| 0 | main | |
| 2 | disk | /var/lib/proxysql/proxysql.db |
| 3 | stats | |
| 4 | monitor | |
| 5 | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.00 sec)
資料庫說明:
- main 記憶體配置資料庫,表裡存放後端db例項、使用者驗證、路由規則等資訊。表名以 runtime開頭的表示proxysql當前執行的配置內容,不能通過dml語句修改,只能修改對應的不以 runtime 開頭的(在記憶體)裡的表,然後 LOAD 使其生效, SAVE 使其存到硬碟以供下次重啟載入
- disk 是持久化到硬碟的配置,sqlite資料檔案
- stats 是proxysql執行抓取的統計資訊,包括到後端各命令的執行次數、流量、processlist、查詢種類彙總/執行時間等等
- monitor 庫儲存 monitor 模組收集的資訊,主要是對後端db的健康/延遲檢查
- stats_history 統計資訊歷史庫
Proxysql管理端新增後端連線mysql主從資料庫的配置
([email protected]:6032) [(none)]> show tables from main;
+--------------------------------------------+
| tables |
+--------------------------------------------+
| global_variables |
| mysql_collations |
| mysql_group_replication_hostgroups |
| mysql_query_rules |
| mysql_query_rules_fast_routing |
| mysql_replication_hostgroups |
| mysql_servers |
| mysql_users |
| proxysql_servers |
| runtime_checksums_values |
| runtime_global_variables |
| runtime_mysql_group_replication_hostgroups |
| runtime_mysql_query_rules |
| runtime_mysql_query_rules_fast_routing |
| runtime_mysql_replication_hostgroups |
| runtime_mysql_servers |
| runtime_mysql_users |
| runtime_proxysql_servers |
| runtime_scheduler |
| scheduler |
+--------------------------------------------+
20 rows in set (0.00 sec)
runtime_ 開頭的是執行時的配置,這些是不能修改的。要修改 ProxySQL 的配置,需要修改了非 runtime_ 表,修改後必須執行 LOAD … TO RUNTIME 才能載入到 RUNTIME 生效,執行 save … to disk 才能將配置持久化儲存到磁碟
下面語句中沒有先切換到 main 庫也執行成功了,因為 ProxySQL 內部使用的 SQLite3 資料庫引擎,和 MySQL 的解析方式是不一樣的。即使執行了 USE main 語句也是無任何效果的,但不會報錯
使用 insert 語句新增 mysql 主機到 mysql_servers 表中,其中:hostgroup_id 1 表示寫組,2表示讀組
([email protected]:6032) [(none)]> insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(1,'192.168.161.164',3306,1,'Write Group');
Query OK, 1 row affected (0.00 sec)
([email protected]:6032) [(none)]> insert into mysql_servers(hostgroup_id,hostname,port,weight,comment) values(2,'192.168.161.167',3306,1,'Read Group');
Query OK, 1 row affected (0.00 sec)
([email protected]:6032) [(none)]> select * from mysql_servers;
+--------------+-----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+-------------+
| hostgroup_id | hostname | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+-----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+-------------+
| 1 | 192.168.161.164 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | Write Group |
| 2 | 192.168.161.167 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | Read Group |
+--------------+-----------------+------+--------+--------+-------------+-----------------+---------------------+---------+----------------+-------------+
2 rows in set (0.00 sec)
修改後,需要載入到RUNTIME,並儲存到disk
([email protected]:6032) [(none)]> load mysql servers to runtime;
Query OK, 0 rows affected (0.00 sec)
([email protected]:6032) [(none)]> save mysql servers to disk;
Query OK, 0 rows affected (0.05 sec)
在 proxysql 主機的 mysql_users 表中新增剛才在 master 上建立的賬號 proxysql,proxysql 客戶端需要使用這個賬號來訪問資料庫
default_hostgroup 預設組設定為寫組,也就是1;
當讀寫分離的路由規則不符合時,會訪問預設組的資料庫;
([email protected]:6032) [(none)]> insert into mysql_users(username,password,default_hostgroup,transaction_persistent)values('proxysql','pwproxysql',1,1);
Query OK, 1 row affected (0.00 sec)
([email protected]:6032) [(none)]> select * from mysql_users \G
*************************** 1. row ***************************
username: proxysql
password: pwproxysql
active: 1
use_ssl: 0
default_hostgroup: 1
default_schema: NULL
schema_locked: 0
transaction_persistent: 1
fast_forward: 0
backend: 1
frontend: 1
max_connections: 10000
1 row in set (0.00 sec)
([email protected]:6032) [(none)]> load mysql users to runtime;
Query OK, 0 rows affected (0.00 sec)
([email protected]:6032) [(none)]> save mysql users to disk;
Query OK, 0 rows affected (0.01 sec)
新增健康檢測的帳號
在mysql的 master 端新增屬於proxysql的只讀賬號
mysql> GRANT SELECT ON *.* TO 'monitor'@'192.168.161.%' IDENTIFIED BY 'monitor';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.03 sec)
在proxysql主機端修改變數設定健康檢測的賬號
([email protected]:6032) [(none)]> set mysql-monitor_username='monitor';
Query OK, 1 row affected (0.01 sec)
([email protected]:6032) [(none)]> set mysql-monitor_password='monitor';
Query OK, 1 row affected (0.00 sec)
([email protected]:6032) [(none)]> load mysql variables to runtime;
Query OK, 0 rows affected (0.00 sec)
([email protected]:6032) [(none)]> save mysql variables to disk;
Query OK, 97 rows affected (0.01 sec)
新增讀寫分離的路由規則
需求:
- 將 select 查詢語句全部路由至 hostgroup_id=2 的組(也就是讀組)
- 但是 select * from tb for update 這樣的語句是會修改資料的,所以需要單獨定義,將它路由至 hostgroup_id=1 的組(也就是寫組)
- 其他沒有被規則匹配到的組將會被路由至使用者預設的組(mysql_users 表中的 default_hostgroup)
([email protected]:6032) [(none)]> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(1,1,'^SELECT.*FOR UPDATE$',1,1);
Query OK, 1 row affected (0.00 sec)
([email protected]:6032) [(none)]> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply)values(2,1,'^SELECT',2,1);
Query OK, 1 row affected (0.00 sec)
([email protected]:6032) [(none)]> select rule_id,active,match_digest,destination_hostgroup,apply from mysql_query_rules;
+---------+--------+----------------------+-----------------------+-------+
| rule_id | active | match_digest | destination_hostgroup | apply |
+---------+--------+----------------------+-----------------------+-------+
| 1 | 1 | ^SELECT.*FOR UPDATE$ | 1 | 1 |
| 2 | 1 | ^SELECT | 2 | 1 |
+---------+--------+----------------------+-----------------------+-------+
2 rows in set (0.00 sec)
([email protected]:6032) [(none)]> load mysql query rules to runtime;
Query OK, 0 rows affected (0.00 sec)
([email protected]:6032) [(none)]> load admin variables to runtime;
Query OK, 0 rows affected (0.00 sec)
([email protected]:6032) [(none)]> save mysql query rules to disk;
Query OK, 0 rows affected (0.01 sec)
([email protected]:6032) [(none)]> save admin variables to disk;
Query OK, 31 rows affected (0.03 sec)
驗證讀寫分離
登入 proxysql 客戶端
登入使用者是剛才我們在 mysql_user 表中建立的使用者,埠為6033
[[email protected] ~]# mysql -uproxysql -ppwproxysql -h127.0.0.1 -P6033
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.5.30 (ProxySQL)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
([email protected]:6033) [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.05 sec)
嘗試修改資料庫和查詢
建立2個數據庫並查詢一下表
([email protected]:6033) [(none)]> create database xixi;
Query OK, 1 row affected (0.00 sec)
([email protected]:6033) [(none)]> create database haha;
Query OK, 1 row affected (0.00 sec)
([email protected]:6033) [(none)]> select user,host from mysql.user;
+---------------+----------------+
| user | host |
+---------------+----------------+
| monitor | 192.168.161.% |
| proxysql | 192.168.161.96 |
| mysql.session | localhost |
| mysql.sys | localhost |
| root | localhost |
+---------------+----------------+
5 rows in set (0.00 sec)
驗證讀寫分離是否成功
proxysql有個類似審計的功能,可以檢視各類SQL的執行情況,其需要在proxysql管理端執行
[[email protected] ~]# mysql -uadmin -padmin -h127.0.0.1 -P6032
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.30 (ProxySQL Admin Module)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
([email protected]:6032) [(none)]> select * from stats_mysql_query_digest;
+-----------+--------------------+----------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| hostgroup | schemaname | username | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time |
+-----------+--------------------+----------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| 1 | information_schema | proxysql | 0xEB4549A4A010C504 | create database haha | 1 | 1543034576 | 1543034576 | 2522 | 2522 | 2522 |
| 1 | information_schema | proxysql | 0x2C9014AFB3100E16 | create database xixi | 1 | 1543034570 | 1543034570 | 2132 | 2132 | 2132 |
| 2 | information_schema | proxysql | 0x0F02B330C823D739 | select user,host from mysql.user | 1 | 1543034594 | 1543034594 | 3562 | 3562 | 3562 |
+-----------+--------------------+----------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
6 rows in set (0.00 sec)
從上面的 hostgroup 和 digest_text 值來看,所有的寫操作都被路由至1組,讀操作都被路由至2組,其中1組為寫組,2組為讀組!
由此可見,讀寫分離成功