Redis核心解讀–AOF與REWRITE機制
自動的bgrewriteaof
為了避免aof檔案過大,我們會週期性的做bgrewriteaof來重整aof檔案。以前我們會額外的配置crontab在業務低峰期執行這個命令,這額外的增加一個workaroud的指令碼任務在大叢集裡是很糟糕的,不易檢查,出錯無法即時發現。
於是這個自動bgrewriteaof功能被直接加到redis的內部。首先對於aof檔案,server物件新增一個欄位來記錄aof檔案的大小server.appendonly_current_size,每次aof發生變化都會維護這個欄位。
aof.c ================= 116 nwritten = write(server.appendfd,server.aofbuf,sdslen(server.aofbuf)); ..... 128 server.appendonly_current_size += nwritten;
bgrewriteaof完畢或者例項啟動載入aof資料後也會呼叫aofUpdateCurrentSize這個函式維護這個欄位,同時會記錄下此時的aof檔案的大小server.auto_aofrewrite_base_size作為基準值,用於接下來判斷aof增長率。
aof.c ================= 385 aofUpdateCurrentSize(); 386 server.auto_aofrewrite_base_size = server.appendonly_current_size;
有了當前值和基準值我們就可以判斷aof檔案的增長情況。另外還需要配置兩個引數來判斷是否需要自動觸發bgrewriteaof。
redis.h =============== int auto_aofrewrite_perc; /* Rewrite AOF if % growth is > M and... */ off_t auto_aofrewrite_min_size; /* the AOF file is at least N bytes. */
auto_aofrewrite_perc: aof檔案的大小超過基準百分之多少後觸發bgrewriteaof。預設這個值設定為100,意味著當前aof是基準大小的兩倍的時候觸發bgrewriteaof。把它設定為0可以禁用自動觸發的功能。
auto_aofrewrite_min_size: 當前aof檔案大於多少位元組後才觸發。避免在aof較小的時候無謂行為。預設大小為64mb。
兩個引數都是可以在conf裡靜態配置,或者通過config set來動態修改的。
redis 127.0.0.1:6379> config get auto-aof-rewrite-percentage 1) "auto-aof-rewrite-percentage" 2) "100" redis 127.0.0.1:6379> config get auto-aof-rewrite-min-size 1) "auto-aof-rewrite-min-size" 2) "1048576" redis 127.0.0.1:6379> config get auto-aof-rewrite-min-size 1) "auto-aof-rewrite-min-size" 2) "1048576" redis 127.0.0.1:6379> config set auto-aof-rewrite-percentage 200 OK redis 127.0.0.1:6379> config set auto-aof-rewrite-min-size 10485760 OK
然後就是觸發檢查的主邏輯,serverCron時間事件中每次都會檢查現有狀態和引數來判斷是否需要啟動bgrewriteaof。
redis.c =============== 635 if (server.bgsavechildpid == -1 && 636 server.bgrewritechildpid == -1 && 637 server.auto_aofrewrite_perc && 638 server.appendonly_current_size > server.auto_aofrewrite_min_size) 639 { 640 long long base = server.auto_aofrewrite_base_size ? 641 server.auto_aofrewrite_base_size : 1; 642 long long growth = (server.appendonly_current_size*100/base) - 100; 643 if (growth >= server.auto_aofrewrite_perc) { 644 redisLog(REDIS_NOTICE,"Starting automatic rewriting of AOF on %lld%% growth",growth); 645 rewriteAppendOnlyFileBackground(); 646 } 647 }
以上程式碼顯示,如果aof檔案增長百分率growth大於auto_aofrewrite_perc,則自動的觸發後一個bgrewriteaof。
延遲bgrewriteaof
這是個小的改進,手動觸發的bgrewriteaof的時候如果同時存在bgsave在備份,會推遲這次操走的事件,設定server.aofrewrite_scheduled=1,待到bgsave結束後的下一次serverCron裡才會觸發。
設定aofrewrite_scheduled=1
aof.c 706 void bgrewriteaofCommand(redisClient *c) { 707 if (server.bgrewritechildpid != -1) { 708 addReplyError(c,"Background append only file rewriting already in progress"); 709 } else if (server.bgsavechildpid != -1) { 710 server.aofrewrite_scheduled = 1; 711 addReplyStatus(c,"Background append only file rewriting scheduled"); 712 } else if (rewriteAppendOnlyFileBackground() == REDIS_OK) { 713 addReplyStatus(c,"Background append only file rewriting started"); 714 } else { 715 addReply(c,shared.err); 716 } 717 }
觸發bgrewriteaof
redis.c 598 if (server.bgsavechildpid == -1 && server.bgrewritechildpid == -1 && 599 server.aofrewrite_scheduled) 600 { 601 rewriteAppendOnlyFileBackground(); 602 }