rds下載後進行恢復(xb格式)_sql_thread方式
環境:
OS:Centos 7
mysql:5.6.40
部署機器:192.168.1.14
背景說明:
線上對一個大表執行了語句insert into a select * from b,b表資料2000多萬的記錄,導致binlog遠遠超過了max_binlog_size設定的值(大事物不會根據設定的值進行切換,等該事物完成後才會切換)
後面使用者有誤操作,要求進行時間點恢復,要求恢復的時間點需要用到該日誌檔案,常規的採用mysqlbinlog應用日誌一直報Got a packet bigger than 'max_allowed_packet' bytes的錯誤
該方法無解後,嘗試採用sql_thread的方法應用日誌,步驟如下:
1.停掉當前的資料庫
停掉資料庫
/opt/mysql56_rds/bin/mysqladmin -h localhost -S /home/mysql/data/mysql.sock -uroot shutdown
2.建立新的資料庫目錄
[root@localhost mysql5640]#cd /home/mysql/
[root@localhost mysql5640]#mv data bakdata ##備份原有的資料庫目錄
建立空的data目錄,用於新的資料庫使用
[root@localhost mysql5640]#mkdir data
3.解壓阿里雲rds下載的備份集
阿里雲上的rds常用的壓縮格式有gz格式和xb格式,我們這裡這個例項是xb格式,不同的格式採用不同的恢復方式
安裝好 XtraBackup 之後,使用 xbstream 命令將備份檔案解包到目標目錄。
[root@localhost bin]# /opt/xtrabackup-2.4.7/bin/xbstream -x -v -C /home/mysql/data</soft/rds/hins18998840_data_20210914055629_qp.xb ##說明/opt/mysql5640/data是資料還原的目錄,hins18998840_data_20210914055629_qp.xb為rds下載檔案
4.解壓還原
/opt/xtrabackup-2.4.7/bin/xtrabackup --decompress --remove-original --target-dir=/home/mysql/data
引數說明:
--remove-original##意思是還原後,將原有的qp字尾的檔案刪除掉
5.應用日誌
/opt/xtrabackup-2.4.7/bin/innobackupex --defaults-file=/opt/mysql56_rds/conf/my.cnf --apply-log /home/mysql/data
配置檔案引數
[root@localhost conf]# more my.cnf
[mysqld]
port=23306
server-id=130
datadir=/home/mysql/data
socket=/home/mysql/data/mysql.sock
character-set-server=utf8
max_connections = 1500
skip-external-locking
key_buffer_size=16M
max_allowed_packet=16M
myisam_sort_buffer_size=16M
query_cache_size=32M
read_buffer_size=2M
sort_buffer_size=2M
interactive_timeout=86400
wait_timeout=86400
innodb_file_per_table=1
innodb_buffer_pool_size=128M
event_scheduler=1
binlog_format=row
log-bin=/opt/mysql56_rds/binlog/binlog.bin
expire_logs_days=1
max_binlog_size=128m
character_set_server=utf8mb4
collation-server=utf8mb4_general_ci
init_connect='SET collation_connection = utf8mb4_general_ci'
init_connect='SET NAMES utf8mb4'
skip-grant-tables=1
gtid_mode=on
log_slave_updates=1
enforce_gtid_consistency=ON
sync_binlog=0 ##關閉雙1
innodb_flush_log_at_trx_commit=0 ##關閉雙1
max_allowed_packet=1073741824
wait_timeout=86400
interactive_timeout=86400
log-warnings=2
relay-log-index=/opt/mysql56_rds/mysqllog/relaylog/slave-relay-bin.index ##從庫配置
relay-log=/opt/mysql56_rds/mysqllog/relaylog/relaylog-binlog ##從庫配置
skip-slave-start
[client]
port=23306
loose-default-character-set = utf8
default-character-set=utf8
[mysql]
no-auto-rehash
port=3306
max_allowed_packet=1073741824
[mysqldump]
max_allowed_packet=1073741824
6.修改目錄許可權
[root@localhost home]#cd /home
[root@localhost home]#chown -R mysql:mysql ./mysql/
[root@localhost home]#pwd
7.啟動mysql
/opt/mysql56_rds/bin/mysqld_safe --defaults-file=/opt/mysql56_rds/conf/my.cnf --user=mysql &
發現啟動報錯誤資訊:
2021-09-22 15:12:48 94824 [ERROR] Error creating master info: Error removing old repository.
2021-09-22 15:12:48 94824 [ERROR] Failed to create or recover replication info repository.
解決辦法:
use mysql
drop table slave_master_info;
drop table slave_relay_log_info;
drop table slave_worker_info;
drop table innodb_index_stats;
drop table innodb_table_stats;
source /opt/mysql56_rds/share/mysql_system_tables.sql;
然後重啟動資料庫
/opt/mysql56_rds/bin/mysqladmin -h localhost -S /home/mysql/data/mysql.sock -uroot shutdown
/opt/mysql56_rds/bin/mysqld_safe --defaults-file=/opt/mysql56_rds/conf/my.cnf --user=mysql &
8.使用sql_thread進行日誌應用
登陸資料庫執行如下命令,模擬從庫
登陸資料庫:
/opt/mysql56_rds/bin/mysql -h localhost -uroot -P23306 -S /home/mysql/data/mysql.sock
set global relay_log_info_repository='FILE';
change master to master_host='1',master_password='1',master_user='1',master_log_file='1',master_log_pos=4;
show variables like '%master_auto_position%';
解決辦法:
change master to master_auto_position=0;
8.關閉資料庫
這一步很重要
/opt/mysql56_rds/bin/mysqladmin -h localhost -S /home/mysql/data/mysql.sock -uroot shutdown
9.拷貝binlog到relay目錄
cp /soft/rds/binlog/mysql-bin.000175 /opt/mysql56_rds/mysqllog/relaylog/
cp /soft/rds/binlog/mysql-bin.000176 /opt/mysql56_rds/mysqllog/relaylog/
修改檔名,修改成relaylog的格式
[root@localhost relaylog]# cd /opt/mysql56_rds/mysqllog/relaylog
[root@localhost relaylog]# mv mysql-bin.000175 relaylog-binlog.000001
[root@localhost relaylog]# mv mysql-bin.000176 relaylog-binlog.000002
然後修改許可權
[root@localhost relaylog]#chown mysql:mysql relaylog-binlog.000001
[root@localhost relaylog]#chown mysql:mysql relaylog-binlog.000002
然後修改slave-relay-bin.index檔案,將需要應用的binlog都寫上去
[root@localhost relaylog]# cd /opt/mysql56_rds/mysqllog/relaylog
[root@localhost relaylog]# more slave-relay-bin.index
/opt/mysql56_rds/mysqllog/relaylog/relaylog-binlog.000001
/opt/mysql56_rds/mysqllog/relaylog/relaylog-binlog.000002
10.修改relay-log.info檔案指定從那個pos開始
[root@localhost data]# cd /home/mysql/data
[root@localhost data]# more relay-log.info
7
/opt/mysql56_rds/mysqllog/relaylog/relaylog-binlog.000001
4
1
4
0
0
1
修改檔案的第二和第三行,第二行是指從那個檔案開始,第三行是指從那個pos開始,修改後的內容如下:
[root@localhost data]# more relay-log.info
7
/opt/mysql56_rds/mysqllog/relaylog/relaylog-binlog.000001
594508
1
4
0
0
11.啟動資料庫和啟動sql_thread程序
啟動資料庫
/opt/mysql56_rds/bin/mysqld_safe --defaults-file=/opt/mysql56_rds/conf/my.cnf --user=mysql &
登陸資料庫
/opt/mysql56_rds/bin/mysql -h localhost -uroot -P23306 -S /home/mysql/data/mysql.sock
檢視當前的從庫狀態
mysql> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: 1
Master_User: 1
Master_Port: 3236
Connect_Retry: 60
Master_Log_File: 1
Read_Master_Log_Pos: 4
Relay_Log_File: relaylog-binlog.000001
Relay_Log_Pos: 594508
Relay_Master_Log_File: 1
Slave_IO_Running: No
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 4
Relay_Log_Space: 2320569428
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 0
Master_UUID:
Master_Info_File: /home/mysql/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 2534e8d3-fb7b-11eb-9563-0c42a1f03f4e:1-63275039,
ca9c7c42-8524-11ea-8c66-7cd30ae4344c:1-382132157,
caf13323-8524-11ea-8c66-506b4bbe201c:1-7962515
Executed_Gtid_Set: 7db1c8b0-1b74-11ec-8fb3-d094667047fb:1-87
Auto_Position: 0
1 row in set (0.00 sec)
ERROR:
No query specified
發現這裡就是我們修改的pos,應用會從這裡開始
Relay_Log_Pos: 594508
啟動sql_thread程序,這裡只啟動sql_thread程序,不要啟動io程序
mysql>start slave sql_thread;
再次檢視從庫狀態
mysql> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: 1
Master_User: 1
Master_Port: 3236
Connect_Retry: 60
Master_Log_File: 1
Read_Master_Log_Pos: 4
Relay_Log_File: relaylog-binlog.000001
Relay_Log_Pos: 637936
Relay_Master_Log_File: 1
Slave_IO_Running: No
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 637936
Relay_Log_Space: 2320569428
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 728631
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 0
Master_UUID:
Master_Info_File: /home/mysql/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Opening tables
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 2534e8d3-fb7b-11eb-9563-0c42a1f03f4e:1-63275039,
ca9c7c42-8524-11ea-8c66-7cd30ae4344c:1-382132157,
caf13323-8524-11ea-8c66-506b4bbe201c:1-7962515
Executed_Gtid_Set: 2534e8d3-fb7b-11eb-9563-0c42a1f03f4e:62910819-62910911,
7db1c8b0-1b74-11ec-8fb3-d094667047fb:1-87
Auto_Position: 0
1 row in set (0.00 sec)
ERROR:
No query specified
Seconds_Behind_Master的值一直在變化,耐心等待,Seconds_Behind_Master=0 說明日誌應用完成了
-- The End --