binlog-rollback.pl 在線恢復update 和delete不加條件誤操作sql
阿新 • • 發佈:2018-05-22
binlog-rollback.pl工具一、binlog-rollback.pl工具介紹
是perl開發的腳本工具,此工具主要是生成反向的DML sql語句:
#基於row模式的binlog,生成DML(insert/update/delete)的rollback語句 #通過mysqlbinlog -v 解析binlog生成可讀的sql文件 #提取需要處理的有效sql #"### "開頭的行.如果輸入的start-position位於某個event group中間,則會導致"無法識別event"錯誤 #將INSERT/UPDATE/DELETE 的sql反轉,並且1個完整sql只能占1行 #INSERT: INSERT INTO => DELETE FROM, SET => WHERE #UPDATE: WHERE => SET, SET => WHERE #DELETE: DELETE FROM => INSERT INTO, WHERE => SET #用列名替換位置@{1,2,3} #通過desc table獲得列順序及對應的列名 #特殊列類型value做特別處理 #逆序
#註意:
#表結構與現在的表結構必須相同[謹記]
#由於row模式是冪等的,並且恢復是一次性,所以只提取sql,不提取BEGIN/COMMIT
#只能對INSERT/UPDATE/DELETE進行處理
#線上誤操作後,一定要立刻對操作的表進行加鎖,避免其他數據繼續寫入
二、MySQL server必須設置以下參數:
cat /etc/my.cnf [client] port = 3306 socket = /tmp/mysql.sock default-character-set=utf8 [mysqld] server_id = 1 log_bin = /var/log/mysql/mysql-bin.log max_binlog_size = 1G binlog_format = row binlog_row_image = full character-set-server = utf8
三、此工具使用需要創建一個mysql的管理用戶來操作
不建議直接使用mysql的root賬戶來操作,建議授權一個最小權限的用戶來操作
登錄mysql創建最小的授權用戶admin
user需要的最小權限集合:
select, super/replication client, replication slave
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO
權限說明
select:需要讀取server端information_schema.COLUMNS表,獲取表結構的元信息,拼接成可視化的sql語句 super/replication client:兩個權限都可以,需要執行‘SHOW MASTER STATUS‘, 獲取server端的binlog列表 replication slave:通過BINLOG_DUMP協議獲取binlog內容的權限
MySQL [(none)]> GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO admin@‘127.0.0.1‘ identified by ‘admin‘;
四、binlog-rollback.pl工具參數介紹
[root@git-server opt]# perl binlog-rollback.pl -help
<Error>:please input binlog file
==========================================================================================
Command line options :
--help # OUT : print help info
-f, --srcfile # IN : binlog file. [required]
-o, --outfile # OUT : output sql file. [required]
-h, --host # IN : host. default ‘127.0.0.1‘
-u, --user # IN : user. [required]
-p, --password # IN : password. [required]
-P, --port # IN : port. default ‘3306‘
--start-datetime # IN : start datetime
--stop-datetime # IN : stop datetime
--start-position # IN : start position
--stop-position # IN : stop position
-d, --database # IN : database, split comma
-T, --table # IN : table, split comma. [required] set -d
-i, --ignore # IN : ignore binlog check contain DDL(CREATE|ALTER|DROP|RENAME)
--debug # IN : print debug information
Sample :
shell> perl binlog-rollback.pl -f ‘mysql-bin.000001‘ -o ‘/tmp/t.sql‘ -u ‘user‘ -p ‘pwd‘
shell> perl binlog-rollback.pl -f ‘mysql-bin.000001‘ -o ‘/tmp/t.sql‘ -u ‘user‘ -p ‘pwd‘ -i
shell> perl binlog-rollback.pl -f ‘mysql-bin.000001‘ -o ‘/tmp/t.sql‘ -u ‘user‘ -p ‘pwd‘ --debug
shell> perl binlog-rollback.pl -f ‘mysql-bin.000001‘ -o ‘/tmp/t.sql‘ -h ‘192.168.1.2‘ -u ‘user‘ -p ‘pwd‘ -P 3307
shell> perl binlog-rollback.pl -f ‘mysql-bin.000001‘ -o ‘/tmp/t.sql‘ -u ‘user‘ -p ‘pwd‘ --start-position=107
shell> perl binlog-rollback.pl -f ‘mysql-bin.000001‘ -o ‘/tmp/t.sql‘ -u ‘user‘ -p ‘pwd‘ --start-position=107 --stop-position=10000
shell> perl binlog-rollback.pl -f ‘mysql-bin.000001‘ -o ‘/tmp/t.sql‘ -u ‘user‘ -p ‘pwd‘ -d ‘db1,db2‘
shell> perl binlog-rollback.pl -f ‘mysql-bin.0000*‘ -o ‘/tmp/t.sql‘ -u ‘user‘ -p ‘pwd‘ -d ‘db1,db2‘ -T ‘tb1,tb2‘
操作報錯如下:
[root@git-server opt]# perl binlog-rollback.pl -f ‘/data/mysql/data/mysql-bin.000002‘ -o ‘/tmp/t.sql‘ -u ‘admin‘ -p ‘admin‘
mysqlbinlog: unknown variable ‘default-character-set=utf8‘
mysqlbinlog: unknown variable ‘default-character-set=utf8‘
mysqlbinlog: unknown variable ‘default-character-set=utf8‘
解決辦法:
[root@git-server opt]# grep mysqlbinlog binlog-rollback.pl
#通過mysqlbinlog -v 解析binlog生成可讀的sql文件
$MYSQLBINLOG = qq{mysqlbinlog --no-defaults -v}; ###添加上參數--no-defaults 解決問題
五、測試delete,和update誤操作後的sql語提取和數據的恢復
5.1delete刪除忘記添加條件恢復測試
MySQL [zixun3]> select * from zx_scores;
+----+--------+------+----------+-----------+
| id | titles | icon | integral | isdefault |
+----+--------+------+----------+-----------+
| 2 | 列兵 | 1 | 0 | 1 |
| 3 | 班長 | 2 | 1000 | 1 |
| 4 | 少尉 | 3 | 2000 | 1 |
| 5 | 中尉 | 4 | 3000 | 1 |
| 6 | 上尉 | 5 | 4000 | 1 |
| 7 | 少校 | 6 | 5000 | 1 |
| 8 | 中校 | 7 | 6000 | 1 |
| 9 | 上校 | 8 | 9000 | 1 |
| 10 | 少將 | 9 | 14000 | 1 |
| 11 | 中將 | 10 | 19000 | 1 |
| 12 | 上將 | 11 | 24000 | 1 |
| 15 | 大將 | 12 | 29000 | 1 |
+----+--------+------+----------+-----------+
12 rows in set (0.00 sec)
MySQL [zixun3]> show master status\G
*************************** 1. row ***************************
File: mysql-bin.000003
Position: 592
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.00 sec)
MySQL [zixun3]> flush logs;
Query OK, 0 rows affected (0.10 sec)
MySQL [zixun3]> show master status\G
*************************** 1. row ***************************
File: mysql-bin.000004
Position: 120
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.00 sec)
delete刪除時,不加任何條件,導致誤刪除
MySQL [zixun3]> delete from zx_scores;
Query OK, 12 rows affected (0.00 sec)
MySQL [zixun3]>
[root@git-server opt]# perl binlog-rollback.pl -f ‘/data/mysql/data/mysql-bin.000004‘ -o ‘/tmp/insert.sql‘ -u ‘admin‘ -p ‘admin‘ -h 127.0.0.1 -i
Warning: Using a password on the command line interface can be insecure.
[root@git-server opt]# cat /tmp/insert.sql
INSERT INTO `zixun3`.`zx_scores` SET `id`=15, `titles`=‘大將‘, `icon`=12, `integral`=29000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=12, `titles`=‘上將‘, `icon`=11, `integral`=24000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=11, `titles`=‘中將‘, `icon`=10, `integral`=19000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=10, `titles`=‘少將‘, `icon`=9, `integral`=14000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=9, `titles`=‘上校‘, `icon`=8, `integral`=9000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=8, `titles`=‘中校‘, `icon`=7, `integral`=6000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=7, `titles`=‘少校‘, `icon`=6, `integral`=5000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=6, `titles`=‘上尉‘, `icon`=5, `integral`=4000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=5, `titles`=‘中尉‘, `icon`=4, `integral`=3000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=4, `titles`=‘少尉‘, `icon`=3, `integral`=2000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=3, `titles`=‘班長‘, `icon`=2, `integral`=1000, `isdefault`=1;
INSERT INTO `zixun3`.`zx_scores` SET `id`=2, `titles`=‘列兵‘, `icon`=1, `integral`=0, `isdefault`=1;
MySQL [zixun3]> source /tmp/insert.sql
MySQL [zixun3]> select * from zx_scores;
+----+--------+------+----------+-----------+
| id | titles | icon | integral | isdefault |
+----+--------+------+----------+-----------+
| 2 | 列兵 | 1 | 0 | 1 |
| 3 | 班長 | 2 | 1000 | 1 |
| 4 | 少尉 | 3 | 2000 | 1 |
| 5 | 中尉 | 4 | 3000 | 1 |
| 6 | 上尉 | 5 | 4000 | 1 |
| 7 | 少校 | 6 | 5000 | 1 |
| 8 | 中校 | 7 | 6000 | 1 |
| 9 | 上校 | 8 | 9000 | 1 |
| 10 | 少將 | 9 | 14000 | 1 |
| 11 | 中將 | 10 | 19000 | 1 |
| 12 | 上將 | 11 | 24000 | 1 |
| 15 | 大將 | 12 | 29000 | 1 |
+----+--------+------+----------+-----------+
12 rows in set (0.00 sec)
delete操作恢復完成
5.2update更新時忘記加條件限制的數據恢復
MySQL [zixun3]> update zx_scores set titles=‘班長‘;
Query OK, 11 rows affected (0.00 sec)
Rows matched: 12 Changed: 11 Warnings: 0
MySQL [zixun3]> select * from zx_scores;
+----+--------+------+----------+-----------+
| id | titles | icon | integral | isdefault |
+----+--------+------+----------+-----------+
| 2 | 班長 | 1 | 0 | 1 |
| 3 | 班長 | 2 | 1000 | 1 |
| 4 | 班長 | 3 | 2000 | 1 |
| 5 | 班長 | 4 | 3000 | 1 |
| 6 | 班長 | 5 | 4000 | 1 |
| 7 | 班長 | 6 | 5000 | 1 |
| 8 | 班長 | 7 | 6000 | 1 |
| 9 | 班長 | 8 | 9000 | 1 |
| 10 | 班長 | 9 | 14000 | 1 |
| 11 | 班長 | 10 | 19000 | 1 |
| 12 | 班長 | 11 | 24000 | 1 |
| 15 | 班長 | 12 | 29000 | 1 |
+----+--------+------+----------+-----------+
12 rows in set (0.00 sec)
[root@git-server opt]# perl binlog-rollback.pl -f ‘/data/mysql/data/mysql-bin.000004‘ -o ‘/tmp/update.sql‘ -u ‘admin‘ -p ‘admin‘ -h 127.0.0.1 -i
Warning: Using a password on the command line interface can be insecure.
由於mysql-bin.000004 中記錄了所有的sql域名,而且binlog-rollback.pl 次工具是過濾mysql-bin.000004 中所有的DML語句 insert,update,delete,所以需要把單獨的update語句過濾出來
[root@git-server opt]# grep -i update /tmp/update.sql >/tmp/3.sql
[root@git-server opt]# cat /tmp/3.sql
UPDATE `zixun3`.`zx_scores` SET `id`=15, `titles`=‘大將‘, `icon`=12, `integral`=29000, `isdefault`=1 WHERE `id`=15 AND `titles`=‘班長‘ AND `icon`=12 AND `integral`=29000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=12, `titles`=‘上將‘, `icon`=11, `integral`=24000, `isdefault`=1 WHERE `id`=12 AND `titles`=‘班長‘ AND `icon`=11 AND `integral`=24000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=11, `titles`=‘中將‘, `icon`=10, `integral`=19000, `isdefault`=1 WHERE `id`=11 AND `titles`=‘班長‘ AND `icon`=10 AND `integral`=19000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=10, `titles`=‘少將‘, `icon`=9, `integral`=14000, `isdefault`=1 WHERE `id`=10 AND `titles`=‘班長‘ AND `icon`=9 AND `integral`=14000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=9, `titles`=‘上校‘, `icon`=8, `integral`=9000, `isdefault`=1 WHERE `id`=9 AND `titles`=‘班長‘ AND `icon`=8 AND `integral`=9000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=8, `titles`=‘中校‘, `icon`=7, `integral`=6000, `isdefault`=1 WHERE `id`=8 AND `titles`=‘班長‘ AND `icon`=7 AND `integral`=6000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=7, `titles`=‘少校‘, `icon`=6, `integral`=5000, `isdefault`=1 WHERE `id`=7 AND `titles`=‘班長‘ AND `icon`=6 AND `integral`=5000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=6, `titles`=‘上尉‘, `icon`=5, `integral`=4000, `isdefault`=1 WHERE `id`=6 AND `titles`=‘班長‘ AND `icon`=5 AND `integral`=4000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=5, `titles`=‘中尉‘, `icon`=4, `integral`=3000, `isdefault`=1 WHERE `id`=5 AND `titles`=‘班長‘ AND `icon`=4 AND `integral`=3000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=4, `titles`=‘少尉‘, `icon`=3, `integral`=2000, `isdefault`=1 WHERE `id`=4 AND `titles`=‘班長‘ AND `icon`=3 AND `integral`=2000 AND `isdefault`=1;
UPDATE `zixun3`.`zx_scores` SET `id`=2, `titles`=‘列兵‘, `icon`=1, `integral`=0, `isdefault`=1 WHERE `id`=2 AND `titles`=‘班長‘ AND `icon`=1 AND `integral`=0 AND `isdefault`=1;
MySQL [zixun3]> source //tmp/3.sql ;
MySQL [zixun3]> select * from zx_scores;
+----+--------+------+----------+-----------+
| id | titles | icon | integral | isdefault |
+----+--------+------+----------+-----------+
| 2 | 列兵 | 1 | 0 | 1 |
| 3 | 班長 | 2 | 1000 | 1 |
| 4 | 少尉 | 3 | 2000 | 1 |
| 5 | 中尉 | 4 | 3000 | 1 |
| 6 | 上尉 | 5 | 4000 | 1 |
| 7 | 少校 | 6 | 5000 | 1 |
| 8 | 中校 | 7 | 6000 | 1 |
| 9 | 上校 | 8 | 9000 | 1 |
| 10 | 少將 | 9 | 14000 | 1 |
| 11 | 中將 | 10 | 19000 | 1 |
| 12 | 上將 | 11 | 24000 | 1 |
| 15 | 大將 | 12 | 29000 | 1 |
+----+--------+------+----------+-----------+
12 rows in set (0.00 sec)
完成恢復
簡單吧,同時此工具還可以結合位置點和時間點進行恢復,請關註後面的博文。
binlog-rollback.pl 在線恢復update 和delete不加條件誤操作sql