《Redis設計與實現》[第二部分]單機資料庫的實現-C原始碼閱讀(三)
3、AOF持久化
關鍵字:AOF持久化:檔案寫入與同步,AOF檔案重寫,資料一致性
與RDB持久化通過儲存資料庫中的鍵值對來記錄資料庫狀態不同,AOF持久化是通過儲存redis伺服器所執行的寫命令來記錄資料庫狀態的
被寫入AOF檔案的所有命令都是以redis的命令請求協議格式儲存的,因為redis的命令請求協議是純文字格式,所以可以直接開啟一個AOF檔案,觀察裡面的內容
AOF檔案初始會自動新增一個用於指定資料庫的select命令。
伺服器在啟動時,可以通過載入和執行AOF檔案中儲存的命令來還原伺服器關閉之前的資料庫狀態
AOF持久化的實現
AOF持久化功能的實現分為命令追加(append)、檔案寫入、檔案同步(sync)三個步驟。
struct redisServer{
// ...
/* AOF persistence */
// AOF 狀態(開啟/關閉/可寫)
int aof_state; /* REDIS_AOF_(ON|OFF|WAIT_REWRITE) */
// 所使用的 fsync 策略(每個寫入/每秒/從不)
int aof_fsync; /* Kind of fsync() policy */
char *aof_filename; /* Name of the AOF file */
int aof_no_fsync_on_rewrite; /* Don't fsync if a rewrite is in prog. */
int aof_rewrite_perc; /* Rewrite AOF if % growth is > M and... */
off_t aof_rewrite_min_size; /* the AOF file is at least N bytes. */
// 最後一次執行 BGREWRITEAOF 時, AOF 檔案的大小
off_t aof_rewrite_base_size; /* AOF size on latest startup or rewrite. */
// AOF 檔案的當前位元組大小
off_t aof_current_size; /* AOF current size. */
//記錄伺服器是否延遲了bgrewriteaof即AOF重寫命令,如果值為1 ,表示有AOF重寫命令被延遲了
int aof_rewrite_scheduled; /* Rewrite once BGSAVE terminates. */
// 負責進行 AOF 重寫的子程序 ID,如果沒有執行,屬性值為-1
pid_t aof_child_pid; /* PID if rewriting process */
// AOF 重寫快取連結串列,連結著多個快取塊
list *aof_rewrite_buf_blocks; /* Hold changes during an AOF rewrite. */
// AOF 緩衝區
sds aof_buf; /* AOF buffer, written before entering the event loop */
// AOF 檔案的描述符
int aof_fd; /* File descriptor of currently selected AOF file */
// AOF 的當前目標資料庫
int aof_selected_db; /* Currently selected DB in AOF */
// 推遲 write 操作的時間
time_t aof_flush_postponed_start; /* UNIX time of postponed AOF flush */
// 最後一直執行 fsync 的時間
time_t aof_last_fsync; /* UNIX time of last fsync() */
time_t aof_rewrite_time_last; /* Time used by last AOF rewrite run. */
// AOF 重寫的開始時間
time_t aof_rewrite_time_start; /* Current AOF rewrite start time. */
// 最後一次執行 BGREWRITEAOF 的結果
int aof_lastbgrewrite_status; /* REDIS_OK or REDIS_ERR */
// 記錄 AOF 的 write 操作被推遲了多少次
unsigned long aof_delayed_fsync; /* delayed AOF fsync() counter */
// 指示是否需要每寫入一定量的資料,就主動執行一次 fsync()
int aof_rewrite_incremental_fsync;/* fsync incrementally while rewriting? */
int aof_last_write_status; /* REDIS_OK or REDIS_ERR */
int aof_last_write_errno; /* Valid if aof_last_write_status is ERR */
// ...
};
當AOF持久化功能處於開啟狀態(aof_state為REDIS_AOF_ON)時,伺服器在執行完一個寫命令之後,會以協議格式將被執行的寫命令追加到伺服器的aof_buf緩衝區的末尾
Redis的伺服器程序就是一個事件迴圈(loop),這個迴圈中的檔案事件負責接收客戶端的命令請求,以及向客戶端傳送命令回覆,而時間事件則負責執行像serverCron函式這樣需要定時執行的函式
因為伺服器在處理檔案事件時可能會執行寫命令,使得一些內容被追加到aof_buf緩衝區裡面,所以在伺服器每次結束一個事件迴圈之前,它都會呼叫flushAppendOnlyFile函式,考慮是否需要將aof_buf緩衝區中的內容寫入和儲存到AOF檔案裡面:
/* Write the append only file buffer on disk.
*
* 將 AOF 快取寫入到檔案中。
*
* Since we are required to write the AOF before replying to the client,
* and the only way the client socket can get a write is entering when the
* the event loop, we accumulate all the AOF writes in a memory
* buffer and write it on disk using this function just before entering
* the event loop again.
*
* 因為程式需要在回覆客戶端之前對 AOF 執行寫操作。
* 而客戶端能執行寫操作的唯一機會就是在事件 loop 中,
* 因此,程式將所有 AOF 寫累積到快取中,
* 並在重新進入事件 loop 之前,將快取寫入到檔案中。
*
* About the 'force' argument:
*
* 關於 force 引數:
*
* When the fsync policy is set to 'everysec' we may delay the flush if there
* is still an fsync() going on in the background thread, since for instance
* on Linux write(2) will be blocked by the background fsync anyway.
*
* 當 fsync 策略為每秒鐘儲存一次時,如果後臺執行緒仍然有 fsync 在執行,
* 那麼我們可能會延遲執行沖洗(flush)操作,
* 因為 Linux 上的 write(2) 會被後臺的 fsync 阻塞。
*
* When this happens we remember that there is some aof buffer to be
* flushed ASAP, and will try to do that in the serverCron() function.
*
* 當這種情況發生時,說明需要儘快沖洗 aof 快取,
* 程式會嘗試在 serverCron() 函式中對快取進行沖洗。
*
* However if force is set to 1 we'll write regardless of the background
* fsync.
*
* 不過,如果 force 為 1 的話,那麼不管後臺是否正在 fsync ,
* 程式都直接進行寫入。
*/
#define AOF_WRITE_LOG_ERROR_RATE 30 /* Seconds between errors logging. */
void flushAppendOnlyFile(int force) {
ssize_t nwritten;
int sync_in_progress = 0;
// 緩衝區中沒有任何內容,直接返回
if (sdslen(server.aof_buf) == 0) return;
// 策略為每秒 FSYNC
if (server.aof_fsync == AOF_FSYNC_EVERYSEC)
// 是否有 SYNC 正在後臺進行?
sync_in_progress = bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC) != 0;
// 每秒 fsync ,並且強制寫入為假
if (server.aof_fsync == AOF_FSYNC_EVERYSEC && !force) {
/* With this append fsync policy we do background fsyncing.
*
* 當 fsync 策略為每秒鐘一次時, fsync 在後臺執行。
*
* If the fsync is still in progress we can try to delay
* the write for a couple of seconds.
*
* 如果後臺仍在執行 FSYNC ,那麼我們可以延遲寫操作一兩秒
* (如果強制執行 write 的話,伺服器主執行緒將阻塞在 write 上面)
*/
if (sync_in_progress) {
// 有 fsync 正在後臺進行 。。。
if (server.aof_flush_postponed_start == 0) {
/* No previous write postponinig, remember that we are
* postponing the flush and return.
*
* 前面沒有推遲過 write 操作,這裡將推遲寫操作的時間記錄下來
* 然後就返回,不執行 write 或者 fsync
*/
server.aof_flush_postponed_start = server.unixtime;
return;
} else if (server.unixtime - server.aof_flush_postponed_start < 2) {
/* We were already waiting for fsync to finish, but for less
* than two seconds this is still ok. Postpone again.
*
* 如果之前已經因為 fsync 而推遲了 write 操作
* 但是推遲的時間不超過 2 秒,那麼直接返回
* 不執行 write 或者 fsync
*/
return;
}
/* Otherwise fall trough, and go write since we can't wait
* over two seconds.
*
* 如果後臺還有 fsync 在執行,並且 write 已經推遲 >= 2 秒
* 那麼執行寫操作(write 將被阻塞)
*/
server.aof_delayed_fsync++;
redisLog(REDIS_NOTICE,"Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.");
}
}
/* If you are following this code path, then we are going to write so
* set reset the postponed flush sentinel to zero.
*
* 執行到這裡,程式會對 AOF 檔案進行寫入。
*
* 清零延遲 write 的時間記錄
*/
server.aof_flush_postponed_start = 0;
/* We want to perform a single write. This should be guaranteed atomic
* at least if the filesystem we are writing is a real physical one.
*
* 執行單個 write 操作,如果寫入裝置是物理的話,那麼這個操作應該是原子的
*
* While this will save us against the server being killed I don't think
* there is much to do about the whole server stopping for power problems
* or alike
*
* 當然,如果出現像電源中斷這樣的不可抗現象,那麼 AOF 檔案也是可能會出現問題的
* 這時就要用 redis-check-aof 程式來進行修復。
*/
nwritten = write(server.aof_fd,server.aof_buf,sdslen(server.aof_buf));
if (nwritten != (signed)sdslen(server.aof_buf)) {
static time_t last_write_error_log = 0;
int can_log = 0;
/* Limit logging rate to 1 line per AOF_WRITE_LOG_ERROR_RATE seconds. */
// 將日誌的記錄頻率限制在每行 AOF_WRITE_LOG_ERROR_RATE 秒
if ((server.unixtime - last_write_error_log) > AOF_WRITE_LOG_ERROR_RATE) {
can_log = 1;
last_write_error_log = server.unixtime;
}
/* Lof the AOF write error and record the error code. */
// 如果寫入出錯,那麼嘗試將該情況寫入到日誌裡面
if (nwritten == -1) {
if (can_log) {
redisLog(REDIS_WARNING,"Error writing to the AOF file: %s",
strerror(errno));
server.aof_last_write_errno = errno;
}
} else {
if (can_log) {
redisLog(REDIS_WARNING,"Short write while writing to "
"the AOF file: (nwritten=%lld, "
"expected=%lld)",
(long long)nwritten,
(long long)sdslen(server.aof_buf));
}
// 嘗試移除新追加的不完整內容
if (ftruncate(server.aof_fd, server.aof_current_size) == -1) {
if (can_log) {
redisLog(REDIS_WARNING, "Could not remove short write "
"from the append-only file. Redis may refuse "
"to load the AOF the next time it starts. "
"ftruncate: %s", strerror(errno));
}
} else {
/* If the ftrunacate() succeeded we can set nwritten to
* -1 since there is no longer partial data into the AOF. */
nwritten = -1;
}
server.aof_last_write_errno = ENOSPC;
}
/* Handle the AOF write error. */
// 處理寫入 AOF 檔案時出現的錯誤
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
/* We can't recover when the fsync policy is ALWAYS since the
* reply for the client is already in the output buffers, and we
* have the contract with the user that on acknowledged write data
* is synched on disk. */
redisLog(REDIS_WARNING,"Can't recover from AOF write error when the AOF fsync policy is 'always'. Exiting...");
exit(1);
} else {
/* Recover from failed write leaving data into the buffer. However
* set an error to stop accepting writes as long as the error
* condition is not cleared. */
server.aof_last_write_status = REDIS_ERR;
/* Trim the sds buffer if there was a partial write, and there
* was no way to undo it with ftruncate(2). */
if (nwritten > 0) {
server.aof_current_size += nwritten;
sdsrange(server.aof_buf,nwritten,-1);
}
return; /* We'll try again on the next call... */
}
} else {
/* Successful write(2). If AOF was in error state, restore the
* OK state and log the event. */
// 寫入成功,更新最後寫入狀態
if (server.aof_last_write_status == REDIS_ERR) {
redisLog(REDIS_WARNING,
"AOF write error looks solved, Redis can write again.");
server.aof_last_write_status = REDIS_OK;
}
}
// 更新寫入後的 AOF 檔案大小
server.aof_current_size += nwritten;
/* Re-use AOF buffer when it is small enough. The maximum comes from the
* arena size of 4k minus some overhead (but is otherwise arbitrary).
*
* 如果 AOF 快取的大小足夠小的話,那麼重用這個快取,
* 否則的話,釋放 AOF 快取。
*/
if ((sdslen(server.aof_buf)+sdsavail(server.aof_buf)) < 4000) {
// 清空快取中的內容,等待重用
sdsclear(server.aof_buf);
} else {
// 釋放快取
sdsfree(server.aof_buf);
server.aof_buf = sdsempty();
}
/* Don't fsync if no-appendfsync-on-rewrite is set to yes and there are
* children doing I/O in the background.
*
* 如果 no-appendfsync-on-rewrite 選項為開啟狀態,
* 並且有 BGSAVE 或者 BGREWRITEAOF 正在進行的話,
* 那麼不執行 fsync
*/
if (server.aof_no_fsync_on_rewrite &&
(server.aof_child_pid != -1 || server.rdb_child_pid != -1))
return;
/* Perform the fsync if needed. */
// 總是執行 fsnyc
if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
/* aof_fsync is defined as fdatasync() for Linux in order to avoid
* flushing metadata. */
aof_fsync(server.aof_fd); /* Let's try to get this data on the disk */
// 更新最後一次執行 fsnyc 的時間
server.aof_last_fsync = server.unixtime;
// 策略為每秒 fsnyc ,並且距離上次 fsync 已經超過 1 秒
} else if ((server.aof_fsync == AOF_FSYNC_EVERYSEC &&
server.unixtime > server.aof_last_fsync)) {
// 放到後臺執行
if (!sync_in_progress) aof_background_fsync(server.aof_fd);
// 更新最後一次執行 fsync 的時間
server.aof_last_fsync = server.unixtime;
}
// 其實上面無論執行 if 部分還是 else 部分都要更新 fsync 的時間
// 可以將程式碼挪到下面來
// server.aof_last_fsync = server.unixtime;
}
flushAppendOnlyFile函式的行為由伺服器配置的appendfsync選項的值來決定“
always:伺服器在每個事件迴圈都要將aof_buf緩衝區中的所有內容寫入並同步到AOF檔案,效率最慢但最安全
everysec:伺服器在每個事件迴圈都要將aof_buf緩衝區中的所有內容寫入到AOF檔案,如果上次同步AOF檔案的時間距離現在超過1s,那麼再次對AOF檔案進行同步,並且這個同步操作是由一個執行緒專門負責執行的
no:伺服器在每個事件迴圈都要將aof_buf緩衝區中的所有內容寫入到AOF檔案,但不對AOF檔案進行同步,何時同步由作業系統決定,速度最快,安全性最低
- 預設為everysec
使用者呼叫write函式,將一些資料寫入到檔案的時候,作業系統通常會將寫入資料暫時儲存在一個記憶體緩衝區中,等到緩衝區的空間被填滿、或者超過了指定的時限之後,才真正地將緩衝區中的資料寫入到磁碟。
雖然提高了效率,但為寫入資料帶來了安全問題,如果計算機發生停機,那麼儲存在記憶體緩衝區裡面的寫入資料將會丟失。
redis提供了fsync和fdatasync兩個同步函式,它們可以強制讓作業系統立即將緩衝區中的資料寫入到硬碟中,從而確保寫入資料的安全性。
AOF檔案的載入與資料還原
redis讀取AOF檔案並還原資料庫狀態的詳細步驟如下:
建立一個不帶網路連線的偽客戶端(fake client):因為redis的命令只能在客戶端上下文中執行,而載入AOF檔案時所使用的命令直接來源於AOF檔案而不是網路連線,所以伺服器使用了一個沒有網路連線的偽客戶端來執行AOF檔案儲存的寫命令,偽客戶端執行命令的效果和帶網路連線的客戶端執行命令的效果完全一樣。
從AOF檔案中分析並讀取一條寫命令
- 使用偽客戶端執行被讀出的寫命令
- 一直執行步驟2和步驟3,直到AOF檔案中的所有寫命令都被處理完畢為止
AOF重寫
為了解決AOF檔案體積膨脹問題,redis提供了AOF檔案重寫(rewrite)功能。通過該功能,Redis伺服器可以建立一個新的AOF檔案來替代現有的AOF檔案,新舊兩個AOF檔案所儲存的資料庫狀態相同,但新AOF檔案不會包含任何浪費空間的冗餘命令,所以新AOF檔案的體積通常會比舊AOF檔案體積要小得多。
AOF檔案重寫並不需要對現有的AOF檔案進行任何讀取、分析或寫入操作,這個功能是通過讀取伺服器當前的資料庫狀態來實現的。
首先從資料庫中讀取鍵現在的值,然後用一條命令去記錄鍵值對,代替之前記錄這個鍵值對的多條命令,這就是AOF重寫功能的實現原理。
rewriteAppendOnlyFile函式的實現:
/* Write a sequence of commands able to fully rebuild the dataset into
* "filename". Used both by REWRITEAOF and BGREWRITEAOF.
*
* 將一個足以還原當前資料集的命令序列寫入到 filename 指定的檔案中。
*
* 這個函式被 REWRITEAOF 和 BGREWRITEAOF 兩個命令呼叫。
* (REWRITEAOF 似乎已經是一個廢棄的命令)
*
* In order to minimize the number of commands needed in the rewritten
* log Redis uses variadic commands when possible, such as RPUSH, SADD
* and ZADD. However at max REDIS_AOF_REWRITE_ITEMS_PER_CMD items per time
* are inserted using a single command.
*
* 為了最小化重建資料集所需執行的命令數量,
* Redis 會盡可能地使用接受可變引數數量的命令,比如 RPUSH 、SADD 和 ZADD 等。
*
* 不過單個命令每次處理的元素數量不能超過 REDIS_AOF_REWRITE_ITEMS_PER_CMD 。
*/
int rewriteAppendOnlyFile(char *filename) {
dictIterator *di = NULL;
dictEntry *de;
rio aof;
FILE *fp;
char tmpfile[256];
int j;
long long now = mstime();
/* Note that we have to use a different temp name here compared to the
* one used by rewriteAppendOnlyFileBackground() function.
*
* 建立臨時檔案
*
* 注意這裡建立的檔名和 rewriteAppendOnlyFileBackground() 建立的檔名稍有不同
*/
snprintf(tmpfile,256,"temp-rewriteaof-%d.aof", (int) getpid());
fp = fopen(tmpfile,"w");
if (!fp) {
redisLog(REDIS_WARNING, "Opening the temp file for AOF rewrite in rewriteAppendOnlyFile(): %s", strerror(errno));
return REDIS_ERR;
}
// 初始化檔案 io
rioInitWithFile(&aof,fp);
// 設定每寫入 REDIS_AOF_AUTOSYNC_BYTES 位元組
// 就執行一次 FSYNC
// 防止快取中積累太多命令內容,造成 I/O 阻塞時間過長
if (server.aof_rewrite_incremental_fsync)
rioSetAutoSync(&aof,REDIS_AOF_AUTOSYNC_BYTES);
// 遍歷所有資料庫
for (j = 0; j < server.dbnum; j++) {
char selectcmd[] = "*2\r\n$6\r\nSELECT\r\n";
redisDb *db = server.db+j;
// 指向鍵空間
dict *d = db->dict;
if (dictSize(d) == 0) continue;
// 建立鍵空間迭代器
di = dictGetSafeIterator(d);
if (!di) {
fclose(fp);
return REDIS_ERR;
}
/* SELECT the new DB
*
* 首先寫入 SELECT 命令,確保之後的資料會被插入到正確的資料庫上
*/
if (rioWrite(&aof,selectcmd,sizeof(selectcmd)-1) == 0) goto werr;
if (rioWriteBulkLongLong(&aof,j) == 0) goto werr;
/* Iterate this DB writing every entry
*
* 遍歷資料庫所有鍵,並通過命令將它們的當前狀態(值)記錄到新 AOF 檔案中
*/
while((de = dictNext(di)) != NULL) {
sds keystr;
robj key, *o;
long long expiretime;
// 取出鍵
keystr = dictGetKey(de);
// 取出值
o = dictGetVal(de);
initStaticStringObject(key,keystr);
// 取出過期時間
expiretime = getExpire(db,&key);
/* If this key is already expired skip it
*
* 如果鍵已經過期,那麼跳過它,不儲存
*/
if (expiretime != -1 && expiretime < now) continue;
/* Save the key and associated value
*
* 根據值的型別,選擇適當的命令來儲存值
*/
if (o->type == REDIS_STRING) {
/* Emit a SET command */
char cmd[]="*3\r\n$3\r\nSET\r\n";
if (rioWrite(&aof,cmd,sizeof(cmd)-1) == 0) goto werr;
/* Key and value */
if (rioWriteBulkObject(&aof,&key) == 0) goto werr;
if (rioWriteBulkObject(&aof,o) == 0) goto werr;
} else if (o->type == REDIS_LIST) {
if (rewriteListObject(&aof,&key,o) == 0) goto werr;
} else if (o->type == REDIS_SET) {
if (rewriteSetObject(&aof,&key,o) == 0) goto werr;
} else if (o->type == REDIS_ZSET) {
if (rewriteSortedSetObject(&aof,&key,o) == 0) goto werr;
} else if (o->type == REDIS_HASH) {
if (rewriteHashObject(&aof,&key,o) == 0) goto werr;
} else {
redisPanic("Unknown object type");
}
/* Save the expire time
*
* 儲存鍵的過期時間
*/
if (expiretime != -1) {
char cmd[]="*3\r\n$9\r\nPEXPIREAT\r\n";
// 寫入 PEXPIREAT expiretime 命令
if (rioWrite(&aof,cmd,sizeof(cmd)-1) == 0) goto werr;
if (rioWriteBulkObject(&aof,&key) == 0) goto werr;
if (rioWriteBulkLongLong(&aof,expiretime) == 0) goto werr;
}
}
// 釋放迭代器
dictReleaseIterator(di);
}
/* Make sure data will not remain on the OS's output buffers */
// 沖洗並關閉新 AOF 檔案
if (fflush(fp) == EOF) goto werr;
if (aof_fsync(fileno(fp)) == -1) goto werr;
if (fclose(fp) == EOF) goto werr;
/* Use RENAME to make sure the DB file is changed atomically only
* if the generate DB file is ok.
*
* 原子地改名,用重寫後的新 AOF 檔案覆蓋舊 AOF 檔案
*/
if (rename(tmpfile,filename) == -1) {
redisLog(REDIS_WARNING,"Error moving temp append only file on the final destination: %s", strerror(errno));
unlink(tmpfile);
return REDIS_ERR;
}
redisLog(REDIS_NOTICE,"SYNC append only file rewrite performed");
return REDIS_OK;
werr:
fclose(fp);
unlink(tmpfile);
redisLog(REDIS_WARNING,"Write error writing append only file on disk: %s", strerror(errno));
if (di) dictReleaseIterator(di);
return REDIS_ERR;
}
AOF後臺重寫
將AOF重寫程式放到子程序裡執行,可以達到兩個目的:
- 子程序進行AOF重寫期間,伺服器程序(父程序)可以繼續處理命令請求
- 子程序帶有伺服器程序的資料副本,使用子程序而不是執行緒,可以在避免使用鎖的情況下,保證資料的安全性
不過,子程序在進行AOF重寫期間,伺服器程序還需要繼續處理命令請求,而新的命令可能會對所有的資料庫狀態進行修改,從而使得伺服器當前的資料庫狀態和重寫後的AOF檔案所儲存的資料庫狀態不一致。
為了解決這種資料不一致問題,redis伺服器設定了一個AOF重寫緩衝區,這個緩衝區在伺服器建立子程序之後開始使用,當redis伺服器執行完一個寫命令之後,它會同時將這個寫命令傳送給AOF緩衝區和AOF重寫緩衝區。
這也就是說,在子程序執行AOF重寫期間,伺服器程序需要執行以下三個工作:
- 執行客戶端發來的命令
- 將執行後的寫命令追加到AOF緩衝區
- 將執行後的寫命令追加到AOF重寫緩衝區
這樣一來,可以保證:
- AOF緩衝區帶內容會定期被寫入和同步到AOF檔案,對現有AOF檔案的處理工作會如常進行
- 從建立子程序開始,伺服器執行的所有寫命令都被記錄到AOF重寫緩衝區裡
當子程序完成AOF重寫工作之後,它會向父程序傳送一個訊號,父程序在接到該訊號之後,會呼叫一個 訊號處理函式 ,並執行以下工作:
- 將AOF重寫緩衝區中的所有內容寫入到新AOF檔案中,這時新AOF檔案所儲存的資料庫狀態和伺服器當前的資料庫狀態一致
- 對新的AOF檔案進行改名,原子地(atomic)覆蓋現有的AOF檔案,完成新舊兩個AOF檔案的替換
訊號處理函式 執行完畢之後,父程序就可以繼續像往常一樣接受命令請求了。
在整個AOF後臺重寫過程中,只有 訊號處理函式 執行時會對伺服器程序(父程序)造成阻塞,其他時候,AOF後臺重寫都不會阻塞父程序,這將AOF重寫對伺服器效能造成的影響降到了最低。
rewriteAppendOnlyFileBackground函式的實現:
/* This is how rewriting of the append only file in background works:
*
* 以下是後臺重寫 AOF 檔案(BGREWRITEAOF)的工作步驟:
*
* 1) The user calls BGREWRITEAOF
* 使用者呼叫 BGREWRITEAOF
*
* 2) Redis calls this function, that forks():
* Redis 呼叫這個函式,它執行 fork() :
*
* 2a) the child rewrite the append only file in a temp file.
* 子程序在臨時檔案中對 AOF 檔案進行重寫
*
* 2b) the parent accumulates differences in server.aof_rewrite_buf.
* 父程序將新輸入的寫命令追加到 server.aof_rewrite_buf 中
*
* 3) When the child finished '2a' exists.
* 當步驟 2a 執行完之後,子程序結束
*
* 4) The parent will trap the exit code, if it's OK, will append the
* data accumulated into server.aof_rewrite_buf into the temp file, and
* finally will rename(2) the temp file in the actual file name.
* The the new file is reopened as the new append only file. Profit!
*
* 父程序會捕捉子程序的退出訊號,
* 如果子程序的退出狀態是 OK 的話,
* 那麼父程序將新輸入命令的快取追加到臨時檔案,
* 然後使用 rename(2) 對臨時檔案改名,用它代替舊的 AOF 檔案,
* 至此,後臺 AOF 重寫完成。
*/
int rewriteAppendOnlyFileBackground(void) {
pid_t childpid;
long long start;
// 已經有程序在進行 AOF 重寫了
if (server.aof_child_pid != -1) return REDIS_ERR;
// 記錄 fork 開始前的時間,計算 fork 耗時用
start = ustime();
if ((childpid = fork()) == 0) {
char tmpfile[256];
/* Child */
// 關閉網路連線 fd
closeListeningSockets(0);
// 為程序設定名字,方便記認
redisSetProcTitle("redis-aof-rewrite");
// 建立臨時檔案,並進行 AOF 重寫
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) getpid());
if (rewriteAppendOnlyFile(tmpfile) == REDIS_OK) {
size_t private_dirty = zmalloc_get_private_dirty();
if (private_dirty) {
redisLog(REDIS_NOTICE,
"AOF rewrite: %zu MB of memory used by copy-on-write",
private_dirty/(1024*1024));
}
// 傳送重寫成功訊號
exitFromChild(0);
} else {
// 傳送重寫失敗訊號
exitFromChild(1);
}
} else {
/* Parent */
// 記錄執行 fork 所消耗的時間
server.stat_fork_time = ustime()-start;
if (childpid == -1) {
redisLog(REDIS_WARNING,
"Can't rewrite append only file in background: fork: %s",
strerror(errno));
return REDIS_ERR;
}
redisLog(REDIS_NOTICE,
"Background append only file rewriting started by pid %d",childpid);
// 記錄 AOF 重寫的資訊
server.aof_rewrite_scheduled = 0;
server.aof_rewrite_time_start = time(NULL);
server.aof_child_pid = childpid;
// 關閉字典自動 rehash
updateDictResizePolicy();
/* We set appendseldb to -1 in order to force the next call to the
* feedAppendOnlyFile() to issue a SELECT command, so the differences
* accumulated by the parent into server.aof_rewrite_buf will start
* with a SELECT statement and it will be safe to merge.
*
* 將 aof_selected_db 設為 -1 ,
* 強制讓 feedAppendOnlyFile() 下次執行時引發一個 SELECT 命令,
* 從而確保之後新新增的命令會設定到正確的資料庫中
*/
server.aof_selected_db = -1;
replicationScriptCacheFlush();
return REDIS_OK;
}
return REDIS_OK; /* unreached */
}