文章目錄

• Replication進階(九) MySQL如何自動清理過期的binlog

Replication進階(九) MySQL如何自動清理過期的binlog

可以通過expire_logs_days設定binlog的過期時間，超過此天數的binlog會被自動purge掉，這個purge的動作是怎麼完成的呢？難道MySQL單獨開闢了一個執行緒來不停的看binlog是不是過期了？如果是你，又會如何設計呢？之所以探究這個問題是因為在設計logserver時，也存在相同的問題，日誌服務不可能永久保留binlog，和MySQL一樣，存在過期清理策略。

今天就來看下MySQL到底是怎麼清理過期的binlog的。

• 資料庫重啟時
  在資料庫啟動初始化的過程中，會對已存在的binlog進行檢測，如果符合過期條件，則會自動進行清理

~/github/mysql-server/sql/mysqld.cc:init_server_components()

4232 #ifdef HAVE_REPLICATION                      
4233   if (opt_bin_log && expire_logs_days)       
4234   {
4235     time_t purge_time= server_start_time - expire_logs_days*24*60*60;
4236     if (purge_time >= 0)
4237       mysql_bin_log.purge_logs_before_date(purge_time, true);
4238   }
4239 #endif

 

• 人為操作：flush logs，觸發過期檢測，自動清理

Thread 23 "mysqld" hit Breakpoint 2, MYSQL_BIN_LOG::purge_logs_before_date (this=0x1df3e00 <mysql_bin_log>, purge_time=1496221928, auto_purge=true)
    at /data/mysql-5.7.18/sql/binlog.cc:6313
6313	{
(gdb) bt
#0  MYSQL_BIN_LOG::purge_logs_before_date (this=0x1df3e00 <mysql_bin_log>, purge_time=1496221928, auto_purge=true) at /data/mysql-5.7.18/sql/binlog.cc:6313
#1  0x0000000000e0db68 in MYSQL_BIN_LOG::rotate_and_purge (this=0x1df3e00 <mysql_bin_log>, 
 
[email protected]=0x7fde00000ae0, 
    [email protected]=true) at /data/mysql-5.7.18/sql/binlog.cc:7236
#2  0x0000000000c6c6ab in reload_acl_and_cache ([email protected]=0x7fde00000ae0, options=16130, [email protected]=0x0, 
    [email protected]=0x7fde38271780) at /data/mysql-5.7.18/sql/sql_reload.cc:161
#3  0x0000000000c37bbb in mysql_execute_command ([email protected]=0x7fde00000ae0, [email protected]=true)
    at /data/mysql-5.7.18/sql/sql_parse.cc:4210
#4  0x0000000000c3e155 in mysql_parse ([email protected]=0x7fde00000ae0, [email protected]=0x7fde38272540)
    at /data/mysql-5.7.18/sql/sql_parse.cc:5612
#5  0x0000000000c3f243 in dispatch_command ([email protected]=0x7fde00000ae0, [email protected]=0x7fde38272de0, command=COM_QUERY)
    at /data/mysql-5.7.18/sql/sql_parse.cc:1461
#6  0x0000000000c40817 in do_command ([email protected]=0x7fde00000ae0) at /data/mysql-5.7.18/sql/sql_parse.cc:999
#7  0x0000000000d00488 in handle_connection ([email protected]=0x2d406b0) at /data/mysql-5.7.18/sql/conn_handler/connection_handler_per_thread.cc:300
#8  0x0000000000eebd81 in pfs_spawn_thread (arg=0x2dcf140) at /data/mysql-5.7.18/storage/perfschema/pfs.cc:2188
#9  0x00007fde59ef66ba in start_thread (arg=0x7fde38273700) at pthread_create.c:333
#10 0x00007fde5938b82d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

• 事務提交，但是會導致日誌輪換，自動清理

(gdb) bt
#0  MYSQL_BIN_LOG::purge (this=0x1df3e00 <mysql_bin_log>) at /data/mysql-5.7.18/sql/binlog.cc:7181
#1  0x0000000000e0eea6 in MYSQL_BIN_LOG::ordered_commit ([email protected]=0x1df3e00 <mysql_bin_log>, [email protected]=0x7fe9840121f0, [email protected]=false,
    skip_commit=<optimized out>) at /data/mysql-5.7.18/sql/binlog.cc:9194
#2  0x0000000000e0fecd in MYSQL_BIN_LOG::commit (this=0x1df3e00 <mysql_bin_log>, thd=0x7fe9840121f0, all=<optimized out>)
    at /data/mysql-5.7.18/sql/binlog.cc:8260
#3  0x00000000007c70c4 in ha_commit_trans ([email protected]=0x7fe9840121f0, [email protected]=false, [email protected]=false)
    at /data/mysql-5.7.18/sql/handler.cc:1795
#4  0x0000000000ce4202 in trans_commit_stmt ([email protected]=0x7fe9840121f0) at /data/mysql-5.7.18/sql/transaction.cc:458
#5  0x0000000000c3a022 in mysql_execute_command ([email protected]=0x7fe9840121f0, [email protected]=true)
    at /data/mysql-5.7.18/sql/sql_parse.cc:5044
#6  0x0000000000c3e155 in mysql_parse ([email protected]=0x7fe9840121f0, [email protected]=0x7fe9d004e540)
    at /data/mysql-5.7.18/sql/sql_parse.cc:5612
#7  0x0000000000c3f243 in dispatch_command ([email protected]=0x7fe9840121f0, [email protected]=0x7fe9d004ede0, command=COM_QUERY)
    at /data/mysql-5.7.18/sql/sql_parse.cc:1461
#8  0x0000000000c40817 in do_command ([email protected]=0x7fe9840121f0) at /data/mysql-5.7.18/sql/sql_parse.cc:999
#9  0x0000000000d00488 in handle_connection ([email protected]=0x3104e70) at /data/mysql-5.7.18/sql/conn_handler/connection_handler_per_thread.cc:300
#10 0x0000000000eebd81 in pfs_spawn_thread (arg=0x2f59a90) at /data/mysql-5.7.18/storage/perfschema/pfs.cc:2188
#11 0x00007fe9ef38d6ba in start_thread (arg=0x7fe9d004f700) at pthread_create.c:333
#12 0x00007fe9ee82282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

第四種屬於題外話，非自動清理，而是手動的讓mysql清理掉指定時間之前的日誌。

• purge binary logs to

Thread 23 "mysqld" hit Breakpoint 4, MYSQL_BIN_LOG::purge_logs_before_date ([email protected]=0x1df3e00 <mysql_bin_log>, purge_time=1496699186, 
    [email protected]=false) at /data/mysql-5.7.18/sql/binlog.cc:6313
6313	{
(gdb) bt
#0  MYSQL_BIN_LOG::purge_logs_before_date ([email protected]=0x1df3e00 <mysql_bin_log>, purge_time=1496699186, [email protected]=false)
    at /data/mysql-5.7.18/sql/binlog.cc:6313
#1  0x0000000000e0d99f in purge_master_logs_before_date ([email protected]=0x7fde00000ae0, purge_time=<optimized out>) at /data/mysql-5.7.18/sql/binlog.cc:2784
#2  0x0000000000c3b74e in mysql_execute_command ([email protected]=0x7fde00000ae0, [email protected]=true)
    at /data/mysql-5.7.18/sql/sql_parse.cc:2863
#3  0x0000000000c3e155 in mysql_parse ([email protected]=0x7fde00000ae0, [email protected]=0x7fde38272540)
    at /data/mysql-5.7.18/sql/sql_parse.cc:5612
#4  0x0000000000c3f243 in dispatch_command ([email protected]=0x7fde00000ae0, [email protected]=0x7fde38272de0, command=COM_QUERY)
    at /data/mysql-5.7.18/sql/sql_parse.cc:1461
#5  0x0000000000c40817 in do_command ([email protected]=0x7fde00000ae0) at /data/mysql-5.7.18/sql/sql_parse.cc:999
#6  0x0000000000d00488 in handle_connection ([email protected]=0x2d406b0) at /data/mysql-5.7.18/sql/conn_handler/connection_handler_per_thread.cc:300
#7  0x0000000000eebd81 in pfs_spawn_thread (arg=0x2dcf140) at /data/mysql-5.7.18/storage/perfschema/pfs.cc:2188
#8  0x00007fde59ef66ba in start_thread (arg=0x7fde38273700) at pthread_create.c:333
#9  0x00007fde5938b82d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

如上就是MySQL清理binlog的所有的觸發邏輯，並非一開始想象的單獨的執行緒操作。