提取mysql binlog日誌內容
阿新 • • 發佈：2019-01-05
整理自網路
（mysqlbinlog命令 -v 顯示每個欄位插入的值， -v -v顯示每個欄位的註釋，包括資料型別）
MySQL binlog日誌記錄了MySQL資料庫從啟用日誌以來所有對當前資料庫的變更。
binlog日誌屬於二進位制檔案，實現基於時間點的恢復或者不完全恢復時必須要用到二進位制日誌。
下面給出了提取binlog日誌的例子：
1.檢視mysql資料庫版本
mysql> show variables like 'version'; +---------------+------------+ | Variable_name | Value | +---------------+------------+ | version | 5.6.28-log | +---------------+------------+ 1 row in set (0.00 sec)
2.檢視binlog格式
mysql> show variables like 'binlog_format';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | ROW |
+---------------+-------+
1 row in set (0.00 sec)3.檢視當前binlog檔案，binlog列表最後一行。
mysql> show binary logs; +------------------+-----------+ | Log_name | File_size | +------------------+-----------+ | mysql-bin.000001 | 65426 | | mysql-bin.000002 | 1176885 | | mysql-bin.000003 | 2064 | | mysql-bin.000004 | 120 | | mysql-bin.000005 | 2848 | | mysql-bin.000006 | 975 | | mysql-bin.000007 | 631 | | mysql-bin.000008 | 191 | +------------------+-----------+ 8 rows in set (0.00 sec)
mysql> show master status\G
*************************** 1. row ***************************
File: mysql-bin.000008
Position: 191
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set: 5933d78c-a22b-11e5-af00-0800273fac50:1-13
1 row in set (0.00 sec)4.建立表，插入資料。
create table binlogtest(id serial,name varchar(20)); insert into binlogtest values (1,'lijingkuan'); insert into binlogtest values (2,'test')
5.在資料庫中查詢mysql binlog中的events；
show binlog events from 191？應該可以，但執行出錯，原因未知。
mysql> show binlog events in 'mysql-bin.000008';
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------------------+
| mysql-bin.000008 | 4 | Format_desc | 123 | 120 | Server ver: 5.6.28-log, Binlog ver: 4 |
| mysql-bin.000008 | 120 | Previous_gtids | 123 | 191 | 5933d78c-a22b-11e5-af00-0800273fac50:1-13 |
| mysql-bin.000008 | 191 | Gtid | 123 | 239 | SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:14' |
| mysql-bin.000008 | 239 | Query | 123 | 368 | use `lucien`; create table binlogtest(id serial,name varchar(20)) |
| mysql-bin.000008 | 368 | Gtid | 123 | 416 | SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:15' |
| mysql-bin.000008 | 416 | Query | 123 | 490 | BEGIN |
| mysql-bin.000008 | 490 | Rows_query | 123 | 560 | # insert into binlogtest values (1,'lijingkuan') |
| mysql-bin.000008 | 560 | Table_map | 123 | 618 | table_id: 71 (lucien.binlogtest) |
| mysql-bin.000008 | 618 | Write_rows | 123 | 673 | table_id: 71 flags: STMT_END_F |
| mysql-bin.000008 | 673 | Xid | 123 | 704 | COMMIT /* xid=31 */ |
| mysql-bin.000008 | 704 | Gtid | 123 | 752 | SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:16' |
| mysql-bin.000008 | 752 | Query | 123 | 826 | BEGIN |
| mysql-bin.000008 | 826 | Rows_query | 123 | 890 | # insert into binlogtest values (2,'test') |
| mysql-bin.000008 | 890 | Table_map | 123 | 948 | table_id: 71 (lucien.binlogtest) |
| mysql-bin.000008 | 948 | Write_rows | 123 | 997 | table_id: 71 flags: STMT_END_F |
| mysql-bin.000008 | 997 | Xid | 123 | 1028 | COMMIT /* xid=42 */ |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------------------+從上可以看出，一個事務在binlog中產生多行記錄。
事務1：建立表
191-----239 獲取GTID
239-----368 建立表語句
事務2：插入第一條資料
368-----416 獲取GTID
416-----490 事務開始
490-----560 事務執行的SQL語句
560-----618
618-----673
673-----704 事務提交（包含事務ID資訊）
事務2：插入第二條資料
同事務1.
6.用mysqlbinlog檢視binlog日誌中相應條目
[[email protected] binlogs]# mysqlbinlog -v -v mysql-bin.000008
.......................................................
.......................................................
# at 191
#151222 14:55:13 server id 123 end_log_pos 239 CRC32 0x208cd83a GTID [commit=yes]
SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:14'/*!*/;
# at 239
#151222 14:55:13 server id 123 end_log_pos 368 CRC32 0x57661241 Query thread_id=4 exec_time=1 error_code=0
use `lucien`/*!*/;
SET TIMESTAMP=1450767313/*!*/;
SET @@session.pseudo_thread_id=4/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1073741824/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
create table binlogtest(id serial,name varchar(20))
/*!*/;
# at 368
#151222 14:58:36 server id 123 end_log_pos 416 CRC32 0x0cc43ac4 GTID [commit=yes]
SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:15'/*!*/;
# at 416
#151222 14:58:36 server id 123 end_log_pos 490 CRC32 0xa5038708 Query thread_id=5 exec_time=0 error_code=0
SET TIMESTAMP=1450767516/*!*/;
BEGIN
/*!*/;
# at 490
#151222 14:58:36 server id 123 end_log_pos 560 CRC32 0xc79548d7 Rows_query
# insert into binlogtest values (1,'lijingkuan')
# at 560
#151222 14:58:36 server id 123 end_log_pos 618 CRC32 0xbf120b46 Table_map: `lucien`.`binlogtest` mapped to number 71
# at 618
#151222 14:58:36 server id 123 end_log_pos 673 CRC32 0x2ce894d0 Write_rows: table id 71 flags: STMT_END_F
BINLOG '
nPR4Vh17AAAARgAAADACAACAAC5pbnNlcnQgaW50byBiaW5sb2d0ZXN0IHZhbHVlcyAoMSwnbGlq
aW5na3Vhbicp10iVxw==
nPR4VhN7AAAAOgAAAGoCAAAAAEcAAAAAAAEABmx1Y2llbgAKYmlubG9ndGVzdAACCA8CPAACRgsS
vw==
nPR4Vh57AAAANwAAAKECAAAAAEcAAAAAAAEAAgAC//wBAAAAAAAAAApsaWppbmdrdWFu0JToLA==
'/*!*/;
### INSERT INTO `lucien`.`binlogtest`
### SET
### @1=1 /* LONGINT meta=0 nullable=0 is_null=0 */
### @2='lijingkuan' /* VARSTRING(60) meta=60 nullable=1 is_null=0 */
# at 673
#151222 14:58:36 server id 123 end_log_pos 704 CRC32 0xc3dcf0b3 Xid = 31
COMMIT/*!*/;
# at 704
#151222 15:05:49 server id 123 end_log_pos 752 CRC32 0x231697ad GTID [commit=yes]
SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:16'/*!*/;
# at 752
#151222 15:05:49 server id 123 end_log_pos 826 CRC32 0xd45eac59 Query thread_id=6 exec_time=0 error_code=0
SET TIMESTAMP=1450767949/*!*/;
BEGIN
/*!*/;
# at 826
#151222 15:05:49 server id 123 end_log_pos 890 CRC32 0x02dc15d0 Rows_query
# insert into binlogtest values (2,'test')
# at 890
#151222 15:05:49 server id 123 end_log_pos 948 CRC32 0x2d629ce6 Table_map: `lucien`.`binlogtest` mapped to number 71
# at 948
#151222 15:05:49 server id 123 end_log_pos 997 CRC32 0xdf63c14e Write_rows: table id 71 flags: STMT_END_F
BINLOG '
TfZ4Vh17AAAAQAAAAHoDAACAAChpbnNlcnQgaW50byBiaW5sb2d0ZXN0IHZhbHVlcyAoMiwndGVz
dCcp0BXcAg==
TfZ4VhN7AAAAOgAAALQDAAAAAEcAAAAAAAEABmx1Y2llbgAKYmlubG9ndGVzdAACCA8CPAAC5pxi
LQ==
TfZ4Vh57AAAAMQAAAOUDAAAAAEcAAAAAAAEAAgAC//wCAAAAAAAAAAR0ZXN0TsFj3w==
'/*!*/;
### INSERT INTO `lucien`.`binlogtest`
### SET
### @1=2 /* LONGINT meta=0 nullable=0 is_null=0 */
### @2='test' /* VARSTRING(60) meta=60 nullable=1 is_null=0 */
# at 997
#151222 15:05:49 server id 123 end_log_pos 1028 CRC32 0x6c55e57a Xid = 42
COMMIT/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog *//*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET [email protected]_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;7.提取某類SQL語句，如insert
[[email protected] binlogs]# mysqlbinlog mysql-bin.000008 | grep insert
/*!40019 SET @@session.max_insert_delayed_threads=0*/;[[email protected] binlogs]# mysqlbinlog -v mysql-bin.000008 | grep insert
/*!40019 SET @@session.max_insert_delayed_threads=0*/;[[email protected] binlogs]# mysqlbinlog -v -v mysql-bin.000008 | grep insert
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
# insert into binlogtest values (1,'lijingkuan')
# insert into binlogtest values (2,'test')
8.演示mysqlbinlog方式提取binlog
提取指定的binlog日誌
# mysqlbinlog /dbdata/binlogs/mysqlbin.000001
# mysqlbinlog /dbdata/binlogs/mysqlbin.000001 |grep insert ------row 模式無法獲得sql語句，能看到二進位制的binlog，無法閱讀</span>
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
insert into tb values(2,'jack')
提取指定position位置的binlog日誌
# mysqlbinlog --start-position="120" --stop-position="332" /dbdata/binlogs/mysqlbin.000001
提取指定位置的多個binlog日誌檔案
# mysqlbinlog --start-position="120" --stop-position="332" /dbdata/binlogs/mysqlbin.000001 /dbdata/binlogs/mysqlbin.000002 |more
提取指定資料庫binlog並轉換字符集到UTF8
# mysqlbinlog --database=lucien --set-charset=utf8 /dbdata/binlogs/mysqlbin.000001 /dbdata/binlogs/mysqlbin.000002 >test.sql
遠端提取使用row格式的binlog日誌並輸出到本地檔案
# mysqlbinlog -uroot -p123456 -h192.168.56.124 --read-from-remote-server -v -v mysqlbin.000008 >row.sql
9.mysql binlog日誌分析
以事務2為例：
（1）獲得事務GTID
SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:15'/*!*/;
（2）事務開始時間
SET TIMESTAMP=1450767516/*!*/;
BEGIN
（3）sqlevent起點
# at 490 ：為事件的起點，是以490位元組開始。
（4）sqlevent發生的時間點（一個事務中可能有多個sqlevent？）
#151222 14:58:36 事件發生的時間
（5）sqlevent結束時間點，花費時間，錯誤程式碼，事務ID，事務執行緒
end_log_pos 490 CRC32 0xa5038708
Query thread_id=5 exec_time=0 error_code=0
Xid = 31