MySQL 8.0.11 innodb cluster 運維管理手冊之二--集群搭建
作者 方連超
基礎環境
系統：centos 7.5
Mysql:8.0.11 二進制包
Mysqlshell: 8.0.11 rpm 包
Mysql router: 8.0.11 二進制包
架構：
192.168.181.101 myrouter1 Keepalived、MySQL-shell、MySQL-Router、MySQL-client
192.168.181.102 myrouter2 Keepalived、MySQL-shell、MySQL-Router、MySQL-client
192.168.181.103 mysql3 MySQL服務端、MySQL-shell
192.168.181.104 mysql4 MySQL服務端、MySQL-shell
192.168.181.105 mysql5 MySQL服務端、MySQL-shell
準備工作：
修改/etc/hosts (5個節點都要做), 要和主機名一致
192.168.181.101 mysql1
192.168.181.102 mysql2
192.168.181.103 mysql3
192.168.181.104 mysql4
192.168.181.105 ysql5
軟件包依賴
yum -y install gcc glibc libaio libstdc++ libstdc libncurses ld-linux
防火墻和selinux關閉：
(1)、關閉SElinux
setenforce 0
修改/etc/selinux/config
vim /etc/selinux/config
SELINUX=disabled(2)、關閉防火墻
systemctl stop firewalld systemctl disable firewalld
sysctl.conf 優化：
cat>>/etc/sysctl.conf <<EOF
fs.aio-max-nr = 1048576
fs.file-max = 681574400
kernel.shmmax = 137438953472
kernel.shmmni = 4096
kernel.sem = 250 32000 100 200
net.ipv4.ip_local_port_range = 9000 65000
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
EOFlimit 優化：
cat>>/etc/security/limits.conf <<EOF
mysql soft nproc 65536
mysql hard nproc 65536
mysql soft nofile 65536
mysql hard nofile 65536
EOFcat>>/etc/pam.d/login <<EOF
session required /lib/security/pam_limits.so
session required pam_limits.so
EOFcat>>/etc/profile<<EOF
if [ $USER = "mysql" ]; then
ulimit -u 16384 -n 65536
fi
EOFsource /etc/profile
安裝mysql軟件（sql 節點）
環境變量
useradd mysql
cat >>/home/mysql/.bash_profile<<EOF
export PATH=/usr/local/mysql/bin:/usr/local/mysql/lib:$PATH
EOF安裝mysql 軟件(sql 節點)：
cd /data
tar -xzvf mysql-8.0.11-el7-x86_64.tar.gz -C /usr/local
mv /usr/local/mysql-8.0.11-el7-x86_64 /usr/local/mysql
chown -R mysql.mysql /usr/local/mysql初始化mysql（一個主節點）
mkdir -p /data/mysql_3306_test/{data,log,binlog,conf,tmp}
chown -R mysql.mysql /data/mysql_3306_test參數文件（自己優化）
su – mysql
Vim /data/mysql_3306_test/conf/my.cnf
[mysqld]
lower_case_table_names = 1
user = mysql
server_id = 1104 #各個實例全局唯一
port = 3310
enforce_gtid_consistency = ON
gtid_mode = ON
binlog_checksum = none
default_authentication_plugin = mysql_native_password #為了兼容以前的驅動
datadir = /data/mysql_3310_test/data
pid-file = /data/mysql_3310_test/tmp/mysqld.pid
socket = /data/mysql_3310_test/tmp/mysqld.sock
tmpdir = /data/mysql_3310_test/tmp/
skip-name-resolve = ON
table_open_cache = 2000
#################innodb########################
innodb_data_home_dir = /data/mysql_3310_test/data
innodb_data_file_path = ibdata1:512M;ibdata2:512M:autoextend
innodb_buffer_pool_size = 2000M
innodb_flush_log_at_trx_commit = 1
innodb_io_capacity = 600
innodb_lock_wait_timeout = 120
innodb_log_buffer_size = 8M
innodb_log_file_size = 200M
innodb_log_files_in_group = 3
innodb_max_dirty_pages_pct = 85
innodb_read_io_threads = 8
innodb_write_io_threads = 8
innodb_thread_concurrency = 32
innodb_file_per_table
innodb_rollback_on_timeout
innodb_undo_directory = /data/mysql_3310_test/data
innodb_log_group_home_dir = /data/mysql_3310_test/data
###################session###########################
join_buffer_size = 8M
key_buffer_size = 256M
bulk_insert_buffer_size = 8M
max_heap_table_size = 96M
tmp_table_size = 96M
read_buffer_size = 8M
sort_buffer_size = 2M
max_allowed_packet = 64M
read_rnd_buffer_size = 32M
############log set###################
log-error = /data/mysql_3310_test/log/mysqld.err
log-bin = /data/mysql_3310_test/binlog/binlog
log_bin_index = /data/mysql_3310_test/binlog/binlog.index
max_binlog_size = 500M
slow_query_log_file = /data/mysql_3310_test/log/slow.log
slow_query_log = 1
long_query_time = 10
log_queries_not_using_indexes = ON
log_throttle_queries_not_using_indexes = 10
log_slow_admin_statements = ON
log_output = FILE,TABLE
master_info_file = /data/mysql_3310_test/binlog/master.info
##########################mgr set##############################
mysqlx_port=33101 #要設置，因為組復制協議走的就是這個端口，多實例下要調整
mysqlx_socket=/data/mysql_3310_test/tmp/mysqlx.sock
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
loose-group_replication_start_on_boot=off
loose-group_replication_local_address= "mysql4:33101"
loose-group_replication_group_seeds= "mysql3:33101,mysql4:33101,mysql5:33101"
loose-group_replication_bootstrap_group= off
loose-group_replication_ip_whitelist="192.168.181.0/24"
loose-group_replication_single_primary_mode = on #我們在次都要單主模式雲因看後面
loose-group_replication_auto_increment_increment=1 #這個默認是7，我們單主模式設置為1
單主模式和多主模式註意
多主模式下：
loose-group_replication_single_primary_mode = off
操作流程：業務端連接IP處理 -> GROUP內成員逐個依次主動退出GROUP （全部退出才行）-> 關閉 group_replication_single_primary_mode參數-> 逐個啟動GROUP內的SERVER
Set global group_replication_single_primary_mode=off
初始化部署：
mysqld --defaults-file=/data/mysql_3306_test/conf/my.cnf --initialize-insecure --user=mysql
mysqld --defaults-file=/data/mysql_3310_test/conf/my.cnf --initialize-insecure --user=mysql設置密碼先要啟動：
mysqladmin --defaults-file=my.cnf password 跟密碼
進入數據庫
mysql -uroot -p -S /data/mysql_3306_test/tmp/mysqld.sock創建本地用戶
create user root@‘127.0.0.1‘ identified by ‘password‘;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE, CREATE ROLE, DROP ROLE ON *.* TO `root`@`127.0.0.1` WITH GRANT OPTION;
GRANT PROXY ON ‘‘@‘‘ TO ‘root‘@‘127.0.0.1‘ WITH GRANT OPTION;
GRANT BACKUP_ADMIN,BINLOG_ADMIN,CONNECTION_ADMIN,ENCRYPTION_KEY_ADMIN,GROUP_REPLICATION_ADMIN,PERSIST_RO_VARIABLES_ADMIN,REPLICATION_SLAVE_ADMIN,RESOURCE_GROUP_ADMIN,RESOURCE_GROUP_USER,ROLE_ADMIN,SET_USER_ID,SYSTEM_VARIABLES_ADMIN,XA_RECOVER_ADMIN ON *.* TO `root`@`127.0.0.1` WITH GRANT OPTION;
create user root@‘192.168.181.%‘ identified by ‘password‘;
GRANT all on *.* TO root@‘192.168.181.%‘ WITH GRANT OPTION;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE, CREATE ROLE, DROP ROLE ON *.* TO `root`@`192.168.181.%` WITH GRANT OPTION;
GRANT BACKUP_ADMIN,BINLOG_ADMIN,CONNECTION_ADMIN,ENCRYPTION_KEY_ADMIN,GROUP_REPLICATION_ADMIN,PERSIST_RO_VARIABLES_ADMIN,REPLICATION_SLAVE_ADMIN,RESOURCE_GROUP_ADMIN,RESOURCE_GROUP_USER,ROLE_ADMIN,SET_USER_ID,SYSTEM_VARIABLES_ADMIN,XA_RECOVER_ADMIN ON *.* TO root@‘192.168.181.%‘ WITH GRANT OPTION;將實例復制拷貝到其他節點：
關閉主庫節點
打包
傳遞
解壓
安裝mysql shell route
rpm -ivh mysql-shell-8.0.11-1.el7.x86_64.rpm
rpm -ivh mysql-router-8.0.11-1.el7.x86_64
配置集群
使用mysql命令
(1)、檢查並配置實例(每個mysql節點)
mysqlsh --log-level=DEBUG3 日誌位置 ~/.mysqlsh/mysqlsh.log
檢查實例
dba.configureLocalInstance(‘[email protected]:3306‘);
dba.chekInstanceConfiguration(‘[email protected]:3306‘)(2)、創建cluster集群(確認每個SQL節點的實例都完成上述實例配置並且驗證成功)
在任意一臺mysql實例節點執行以下命令：
mysqlsh
shell.connect(‘root@mysql1:3306‘)
var cluster = dba.createCluster(‘qwCluster‘);如果創建成功輸出的信息中會有類似“Cluster successfully created.”的語句
將另外兩個節點加入到Cluster集群中
cluster.addInstance(‘root@mysql4:3306‘);
cluster.addInstance(‘root@mysql5:3306‘);集群驗證
mysql.shell 查看
cluster.status()
navacat 連接mysql 查看
select @@hostname
問題
多實例環境不要用 3306端口
多實例環境下，某個實例采用了默認的3306端口，會導致經常性的誤操作。切記
本環境建議的端口
3300 3310 3320 … 3390
一臺主機最多部署10個實例
如何重置實驗環境
- 主節點：
dba.dropMetadataSchema() mysqlshell 清空集群 mysql> stop group_replication; mysql> reset master; （清空日誌，確保和從庫的表沒有沖突奧，） mysql> reset slave - 其他節點(主要清理和主庫的主從信息， 確保主庫和從庫的表沒有沖突奧)
mysql> stop group_replication; mysql> reset master; mysql> reset slave主機名和 /etc/hosts中名字不一致
[Repl] Slave I/O for channel ‘group_replication_recovery‘: error connecting to master ‘mysql_innodb_cluster_r0430970923@mysql3:3306‘ - retry-time: 60 retries: 1, Error_code: MY-002005
[ERROR] [MY-011582] [Repl] Plugin group_replication reported: ‘There was an error when connecting to the donor server. Please check that group_replication_recovery channel credentials and all MEMBER_HOST column values of performance_schema.replication_group_members table are correct and DNS resolvable.‘
[ERROR] [MY-011583] [Repl] Plugin group_replication reported: ‘For details please check performance_schema.replication_connection_status table and error log messages of Slave I/O for channel group_replication_recovery.‘
這個問題郁悶了我很久，
Hostname 是
mysql4
mysql5
mysql6
而我在 /etc/hosts中是
192.168.181.103 mysql-1
192.168.181.103 mysql-2
192.168.181.103 mysql-3
主庫的日誌應用卡在某個位置無法應用到從庫，
[ERROR] [MY-010586] [Repl] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log ‘binlog.000007‘ position 151
[ERROR] [MY-010584] [Repl] Slave SQL for channel ‘group_replication_applier‘: Error executing row event: ‘Unknown database ‘mysql_innodb_cluster_metadata‘‘, Error_code: MY-001049
重建master：MySQL>reset master
組復制的限制
- 事物鎖缺失問題：
組復制建議，事物隔離級別，read commit - 序列化隔離級別：多主模式不支持
- 並發DDL和DML: 多主模式下，不支持 一邊對一個表進行DDL,另一邊進行更新，這樣對於DDL在其他實例上操作有未檢出的風險
- 外鍵級聯約束：多主模式下，多級外鍵依賴對引起多級操作， 因此可能導致未知沖突，建議打開 group_replication_enforce_update_everywhere_checks=ON
- 大事物，超過5秒未提交，會導致組通信失敗，
- 多主模式下：select * for update 會導致 死鎖。因為這個鎖並非全組共享。
- 部分復制不支持：組復制下，設置部分復制，會過濾事物，導致組事物不一致。
- Mysql 8.0.11 group_replication_enforce_update_everywhere_checks=ON 多主模式下不支持。
- 停止復制的情況下，某個節點執行命令後再啟動，會因為本地有私有事物，無法加入集群。需要全局 reset master 重新開始集群復制。
MySQL 8.0.11 innodb cluster 運維管理手冊之二--集群搭建