1. 程式人生 > 實用技巧 >分庫分表 3:主從複製 (從入門到精通)

分庫分表 3:主從複製 (從入門到精通)


《SpringCloud Nginx 高併發核心程式設計》 環境搭建 - 系列

元件 連結地址
【必須】 虛擬機器Linux 開發環境準備 https://www.cnblogs.com/crazymakercircle/p/14194688.html
Linux openresty 安裝 Linux openresty 安裝
【必須】Linux Redis 安裝(帶視訊) Linux Redis 安裝(帶視訊)
【必須】Linux Zookeeper 安裝(帶視訊) Linux Zookeeper 安裝, 帶視訊
Windows Redis 安裝(帶視訊) Windows Redis 安裝(帶視訊)
RabbitMQ 離線安裝(帶視訊) RabbitMQ 離線安裝(帶視訊)
ElasticSearch 安裝, 帶視訊 ElasticSearch 安裝, 帶視訊
Nacos 安裝(帶視訊) Nacos 安裝(帶視訊)
【必須】Eureka Eureka 入門,帶視訊
【必須】springcloud Config 入門,帶視訊 springcloud Config 入門,帶視訊
【必須】SpringCloud 腳手架打包與啟動 SpringCloud腳手架打包與啟動
Linux 自啟動 假死自啟動 定時自啟 Linux 自啟動 假死啟動

Sharding-JDBC 從入門到精通 - 目錄

元件 連結地址
準備一: 在window安裝虛擬機器叢集 vagrant+java+springcloud+redis+zookeeper映象下載(&製作詳解))
而且:在虛擬機器上需要安裝 mysql centos mysql 筆記(內含vagrant mysql 映象)
分庫分表 Sharding-JDBC 從入門到精通之一 Sharding-JDBC 入門實戰
分庫分表 Sharding-JDBC 從入門到精通之二 Sharding-JDBC 基本原理
分庫分表 Sharding-JDBC 從入門到精通之三 MYSQL主從複製
分庫分表 Sharding-JDBC 從入門到精通之四 讀寫分離
分庫分表 Sharding-JDBC 從入門到精通之原始碼 git

MYSQL主從複製配置

練習的場景

主從實戰的資料庫場景:

建議有三虛擬機器:cdh1、 cdh2、 cdh3(cdh3可以不開啟)。

cdh1 上的 user_db為主,cdh1 上的 user_db為從,僅僅開啟這兩個庫的複製,其他的庫如store,不開啟主從複製。
有關學習環境的快速搭建,請參考本系列部落格的準備一: 在window安裝虛擬機器叢集 。

配置主資料庫

登入主庫

mysql  ‐uroot ‐p123456

建立用於主從複製的賬號

set global validate_password_policy=0;
set global validate_password_length=1;
create user db_sync identified by '123456';

建立庫和表


mysql> CREATE DATABASE user_db DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

mysql> use user_db;
Database changed

mysql> create table t_user_0( id bigint , name varchar(40) );
Query OK, 0 rows affected (0.02 sec)

授權主備複製專用賬號

GRANT REPLICATION SLAVE ON *.* TO 'db_sync'@'%' IDENTIFIED BY 'db_sync'; 

重新整理許可權

FLUSH PRIVILEGES;

修改 MySQL 配置文件 /etc/my.cnf,在 [mysqld] 段新增以下配置:

[mysqld] 
#開啟日誌,啟用二進位制日誌
log‐bin = mysql‐bin 

#設定伺服器標識ID,主從不能一致 ,每臺伺服器唯一
server‐id = 1      

#設定需要同步的資料庫 
binlog‐do‐db=user_db 

#遮蔽不需要同步的資料庫 
binlog‐ignore‐db=mysql
binlog‐ignore‐db=information_schema
binlog‐ignore‐db=performance_schema
binlog‐ignore‐db=dolphinscheduler
binlog‐ignore‐db=sys
binlog‐ignore‐db=store 

引數說明:

log-bin
  該引數只要配置就表示開啟了MySQL的bin log日誌功能,注意改引數的值是我們自定義的,我們自定義的值將作為bin log的名稱的字首資訊喲,我們可以使用MySQL命令"show variables like '%log_bin%';"檢視咱們的配置。

server-id
  該引數可以指定資料庫伺服器的唯一標識。在同一個複製組下的所有例項的server_id都必須是唯一的,而且取值必須是正整數,取值範圍是1~(232)−1

可以通過show databases,檢視所有的庫,然後找出不需要同步的資料庫

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| dolphinscheduler   |
| mysql              |
| performance_schema |
| store              |
| sys                |
+--------------------+
6 rows in set (0.01 sec)

完整配置如下:

[root@cdh1 ~]# cat /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html

[mysqld]
log-bin=user-mysql-bin
server-id=1

binlog-do-db=user_db
binlog-ignore-db=mysql
binlog-ignore-db=information_schema
binlog-ignore-db=performance_schema
binlog-ignore-db=dolphinscheduler
binlog-ignore-db=sys
binlog-ignore-db=store

skip-name-resolve
character_set_server=utf8
init_connect='SET NAMES utf8'
lower_case_table_names=1
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

配置從伺服器

登入從庫

mysql  ‐uroot ‐p123456

建立用於主從複製的賬號

set global validate_password_policy=0;
set global validate_password_length=1;
create user db_sync identified by '123456';

建立庫和表

mysql> CREATE DATABASE user_db DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

mysql> use user_db;
Database changed

mysql> create table t_user_0( id bigint , name varchar(40) );
Query OK, 0 rows affected (0.02 sec)

授權主備複製專用賬號

GRANT REPLICATION SLAVE ON *.* TO 'db_sync'@'%' IDENTIFIED BY '123456'; 

重新整理許可權

FLUSH PRIVILEGES;

修改 MySQL 配置文件 /etc/my.cnf,在 [mysqld] 段新增以下配置:

[mysqld] 

log‐bin=mysql‐bin  #開啟日誌,啟用二進位制日誌
server‐id=2 #設定伺服器標識ID,主從不能一致 ,每臺伺服器唯一

#設定需要同步的資料庫 
replicate_wild_do_table=user_db.% 
#遮蔽系統庫同步 
replicate_wild_ignore_table=mysql.%
replicate_wild_ignore_table=information_schema.%
replicate_wild_ignore_table=performance_schema.%
replicate_wild_ignore_table=dolphinscheduler.%
replicate_wild_ignore_table=sys.%
replicate_wild_ignore_table=store.%

重啟資料庫

systemctl restart mysqld

完整的配置檔案如下:

[root@cdh2 ~]# vim /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html

[mysqld]
log-bin=user-mysql-bin
server-id=101

replicate_wild_do_table=user_db.%
replicate_wild_ignore_table=mysql.%
replicate_wild_ignore_table=information_schema.%
replicate_wild_ignore_table=performance_schema.%
replicate_wild_ignore_table=dolphinscheduler.%
replicate_wild_ignore_table=sys.%
replicate_wild_ignore_table=store.%

skip-name-resolve
character_set_server=utf8
init_connect='SET NAMES utf8'
lower_case_table_names=1
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

主從複製的配置

設定主庫

首先連線 master db,為從庫的賬號授予複製的權利

GRANT REPLICATION SLAVE ON . TO 'db_sync'@'%' IDENTIFIED BY '123456';

如果許可權不夠,就執行一下下面的兩句:

set global validate_password_policy=0;
set global validate_password_length=1;

重新整理許可權

FLUSH PRIVILEGES;

在主庫檢視同步的起點, 記錄下檔名以及起點 位置
show master status;

檢視資料庫狀態:

記錄 File 的值( user-mysql-bin.000001 )和 Position 的值(154),等會配置 slave 伺服器的時候要用。

設定從庫

接下來在 slave db上,配置主從複製:

先停止同步

STOP SLAVE; 

修改從庫的master配置,指向到主庫,配置過程中,需要使用上一步記錄的檔名以及位點

mysql>change master to
master_host='cdh1',
master_user='db_sync',
master_log_file='user-mysql-bin.000001',
master_log_pos=154,
master_port=3306,
master_password='123456';
Query OK, 0 rows affected, 2 warnings (0.03 sec)
mysql> start slave;
Query OK, 0 rows affected (0.01 sec)

解釋下配置引數

master_host='x.x.x.x' // 這裡填 master 主機 ip
master_log_file='user-mysql-bin.000001', // 這裡填寫 File 的值
master_log_pos=154,// 這裡填寫 Position 的值。

啟動同步

START SLAVE; 

檢視同步

show slave status\G; 

結果如下:

mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: cdh1
                  Master_User: db_sync
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: user-mysql-bin.000002
          Read_Master_Log_Pos: 154
               Relay_Log_File: cdh2-relay-bin.000004
                Relay_Log_Pos: 377
        Relay_Master_Log_File: user-mysql-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table: user_db.%
  Replicate_Wild_Ignore_Table: mysql.%,information_schema.%,performance_schema.%,dolphinscheduler.%,sys.%,store.%
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 154
              Relay_Log_Space: 2531
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: fb8d483c-47eb-11eb-ba17-0800276c3e95
             Master_Info_File: /var/lib/mysql/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
1 row in set (0.00 sec)

檢視Slave_IO_Runing和Slave_SQL_Runing欄位值都為Yes,表示同步配置成功。如果不為Yes,請排
查相關異常。

如果不小心配置錯, 輸入 mysql> stop slave; 然後重新錄入一遍就可以了。

問題:

問題1: 由於是克隆的虛擬機器,所以mysql的uuid相同,報錯如下:

The slave I/O thread stops because master and slave have equal MySQL server UUID

原因:主從複製的mysql例項的uuid不能相同。

修改方法:在從庫上,從my.cnf找到data目錄,然後定位到auto.cnf目錄,找到 auto.cnf檔案,進行uuid的調整。

具體如下:

$ vim /var/lib/mysql/auto.cnf
[auto]
server-uuid=fb8d483c-47eb-11eb-ba17-0800276c3e96

修改之後:

[root@cdh2 ~]#  cat /var/lib/mysql/auto.cnf
[auto]
server-uuid=fb8d483c-47eb-11eb-ba17-0800276c3e96

最終,確保兩個mysql 例項的uuid不同。

然後,重啟改動過的例項。

主從複製測試

1 將主庫的t_user_0表複製一份t_user_0_copy1,稍等一會,重新整理從庫,發現從庫也有了一個t_user_0_copy1表,表明從庫複製了主庫的t_user_0_copy1表。

2 修改主庫的t_user_0_copy1,名稱為t_user_1,大約過去10s,重新整理從庫,從庫的t_user_0_copy1名稱改為t_user_1

回到◀瘋狂創客圈

瘋狂創客圈 - Java高併發研習社群,為大家開啟大廠之門