1. 程式人生 > >MySql GR啟動時節點狀態卡在RECOVERING

MySql GR啟動時節點狀態卡在RECOVERING

問題描述:

[email protected]:(none) 04:47:06>SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 574d6e9a-fc15-11e7-9fcf-000c29879d51 | centos129   |        3306 | ONLINE       |
| group_replication_applier | b376058d-5762-11e7-baa0-000c29e6b568 | centos128   |        3306 | RECOVERING   |
| group_replication_applier | e87b5168-fc15-11e7-9fcf-000c29879d51 | centos130   |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+

並且錯誤日誌顯示內容如下:

2018-05-15T08:08:18.041338Z 54 [ERROR] Error reading packet from server for channel 'group_replication_recovery': The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. (server_errno=1236)
2018-05-15T08:08:18.041384Z 54 [ERROR] Slave I/O for channel 'group_replication_recovery': Got fatal error 1236 from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.', Error_code: 1236

2018-05-15T08:10:18.511895Z 18 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='centos129', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2018-05-15T08:10:18.523029Z 18 [ERROR] Plugin group_replication reported: 'Maximum number of retries when trying to connect to a donor reached. Aborting group replication recovery.'
2018-05-15T08:10:18.523049Z 18 [Note] Plugin group_replication reported: 'Terminating existing group replication donor connection and purging the corresponding logs.'
2018-05-15T08:10:18.531944Z 18 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2018-05-15T08:10:18.547788Z 18 [ERROR] Plugin group_replication reported: 'Fatal error during the Recovery process of Group Replication. The server will leave the group.'
2018-05-15T08:10:18.549152Z 0 [Note] Plugin group_replication reported: 'getstart group_id 4317e324'
2018-05-15T08:10:22.045063Z 0 [Note] Plugin group_replication reported: 'state 4330 action xa_terminate'
2018-05-15T08:10:22.045482Z 0 [Note] Plugin group_replication reported: 'new state x_start'
2018-05-15T08:10:22.045511Z 0 [Note] Plugin group_replication reported: 'state 4257 action xa_exit'
2018-05-15T08:10:22.045634Z 0 [Note] Plugin group_replication reported: 'Exiting xcom thread'
2018-05-15T08:10:22.045655Z 0 [Note] Plugin group_replication reported: 'new state x_start'

檢視centos128的狀態:

[email protected]:qn 05:02:15>select * from performance_schema.replication_group_member_stats \G ;
*************************** 1. row ***************************
                      CHANNEL_NAME: group_replication_applier
                           VIEW_ID: 15263657103992362:28
                         MEMBER_ID: b376058d-5762-11e7-baa0-000c29e6b568
       COUNT_TRANSACTIONS_IN_QUEUE: 0
        COUNT_TRANSACTIONS_CHECKED: 0
          COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS: 
    LAST_CONFLICT_FREE_TRANSACTION:

  此時沒有顯示centos128上的gtid資訊。檢視其它節點狀態:

centos129:

[email protected]:qn 04:47:47>select * from performance_schema.replication_group_member_stats \G ;
*************************** 1. row ***************************
                      CHANNEL_NAME: group_replication_applier
                           VIEW_ID: 15263657103992362:28
                         MEMBER_ID: 574d6e9a-fc15-11e7-9fcf-000c29879d51
       COUNT_TRANSACTIONS_IN_QUEUE: 0
        COUNT_TRANSACTIONS_CHECKED: 1
          COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-151,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678
    LAST_CONFLICT_FREE_TRANSACTION: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:142
1 row in set (0.00 sec)

centos130:

[email protected]:(none) 05:05:06>select * from performance_schema.replication_group_member_stats \G ;
*************************** 1. row ***************************
                      CHANNEL_NAME: group_replication_applier
                           VIEW_ID: 15263657103992362:33
                         MEMBER_ID: e87b5168-fc15-11e7-9fcf-000c29879d51
       COUNT_TRANSACTIONS_IN_QUEUE: 5
        COUNT_TRANSACTIONS_CHECKED: 1
          COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-151,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678
    LAST_CONFLICT_FREE_TRANSACTION: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:142

檢視節點的GTID狀態:

centos128:

[email protected]:(none) 04:48:02>show global variables like '%gtid%' ;
+---------------------------------------------------+---------+
| Variable_name                                     | Value   |
+---------------------------------------------------+---------+
| binlog_gtid_simple_recovery                       | ON      |
| enforce_gtid_consistency                          | ON      |
| group_replication_allow_local_disjoint_gtids_join | OFF     |
| group_replication_gtid_assignment_block_size      | 1000000 |
| gtid_executed                                     |         |
| gtid_executed_compression_period                  | 1000    |
| gtid_mode                                         | ON      |
| gtid_owned                                        |         |
| gtid_purged                                       |         |
| session_track_gtids                               | OFF     |
+---------------------------------------------------+---------+

centos129和centos130一樣:

[email protected]:qn 04:44:32>show global variables like '%gtid%' ;
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| Variable_name                                     | Value                                                                                    |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| binlog_gtid_simple_recovery                       | ON                                                                                       |
| enforce_gtid_consistency                          | ON                                                                                       |
| group_replication_allow_local_disjoint_gtids_join | OFF                                                                                      |
| group_replication_gtid_assignment_block_size      | 1000000                                                                                  |
| gtid_executed                                     | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-149,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| gtid_executed_compression_period                  | 1000                                                                                     |
| gtid_mode                                         | ON                                                                                       |
| gtid_owned                                        |                                                                                          |
| gtid_purged                                       | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| session_track_gtids                               | OFF                                    

根據centos129與centos130的gtid_purged ,更改centos128的gtid_purged使其按gtid_purged開始複製:

[email protected]:(none) 04:53:03>set global gtid_purged='aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,b376058d-5762-11e7-baa0-000c29e6b568:1-47678' ;
Query OK, 0 rows affected (0.24 sec)

[email protected]:(none) 04:53:17>show global variables like '%gtid%' ;
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| Variable_name                                     | Value                                                                                    |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| binlog_gtid_simple_recovery                       | ON                                                                                       |
| enforce_gtid_consistency                          | ON                                                                                       |
| group_replication_allow_local_disjoint_gtids_join | OFF                                                                                      |
| group_replication_gtid_assignment_block_size      | 1000000                                                                                  |
| gtid_executed                                     | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| gtid_executed_compression_period                  | 1000                                                                                     |
| gtid_mode                                         | ON                                                                                       |
| gtid_owned                                        |                                                                                          |
| gtid_purged                                       | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| session_track_gtids                               | OFF                                                                                      |
+---------------------------------------------------+------------------------------------------------------------------------------------------+

開啟centos128的GR:

[email protected]:(none) 04:53:22>CHANGE MASTER TO MASTER_USER='rpl_user', MASTER_PASSWORD='rpl_pass' FOR CHANNEL 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.02 sec)

[email protected]:(none) 04:54:17>show global variables like '%gtid%' ;
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| Variable_name                                     | Value                                                                                    |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
| binlog_gtid_simple_recovery                       | ON                                                                                       |
| enforce_gtid_consistency                          | ON                                                                                       |
| group_replication_allow_local_disjoint_gtids_join | OFF                                                                                      |
| group_replication_gtid_assignment_block_size      | 1000000                                                                                  |
| gtid_executed                                     | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| gtid_executed_compression_period                  | 1000                                                                                     |
| gtid_mode                                         | ON                                                                                       |
| gtid_owned                                        |                                                                                          |
| gtid_purged                                       | aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa:1-135,
b376058d-5762-11e7-baa0-000c29e6b568:1-47678 |
| session_track_gtids                               | OFF                                                                                      |
+---------------------------------------------------+------------------------------------------------------------------------------------------+
10 rows in set (0.00 sec)

[email protected]:(none) 04:54:21>START GROUP_REPLICATION; 
Query OK, 0 rows affected (1.93 sec)

[email protected]:(none) 04:54:35>SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 574d6e9a-fc15-11e7-9fcf-000c29879d51 | centos129   |        3306 | ONLINE       |
| group_replication_applier | b376058d-5762-11e7-baa0-000c29e6b568 | centos128   |        3306 | ONLINE       |
| group_replication_applier | e87b5168-fc15-11e7-9fcf-000c29879d51 | centos130   |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
3 rows in set (0.00 sec)