春招必問的redis持久化(RDB AOF),你能答上來麼?
春招面試模擬,如同雷同,純屬巧合!!!
面試的大體流程:
第一步:一般會有筆試題,也可能沒有。有筆試題就要好好答了,因為會重視筆試結果,為了節約面試官時間,HR可能先會看,不合格直接讓你走人了。
第二步:開始面試,面試官會讓你先來個自我介紹,他在看你的簡歷。可能有人會有疑問?為什麼簡歷上都寫的很清楚還讓我做自我介紹?因為面試官不會提前看你的簡歷,他要有時間去看簡歷。所以自我介紹,不宜過長或過短,要重點突出,怎麼重點突出?提前看好崗位要求,要求都是入職之後用到的技術,所以面試官會看重那些!!!自我介紹結束,步入正題。
面試官會問:之前的公司用過redis麼?
面試者有兩類回答:
面試者甲:沒用過,但自己學過,下載過原始碼,自己部署安裝過,基本的命令像string/hash/lsit/set/zset,都熟悉;
面試官(心想雖然沒用,但動手能力很強,也很好學,不錯):redis是記憶體資料庫,那它怎麼進行持久化的?
面試官甲(心想沒注意看啊,不知道啊):...
面試者乙:用過,比較熟悉
面試官會接著問:持久化方式有哪些?
面試者乙:RDB和AOF
面試官:原理?區別?優缺點說一下吧?
面試官乙:...
要是是你去面試,你能回答上來麼?
下面讓我為你一一揭曉答案!!!
RDB持久化(Redis DataBase)
redis是記憶體資料庫,一旦伺服器程序退出,伺服器中的資料庫狀態也會消失不見。重點介紹save和bgsave命令。
RDB檔案的建立與載入
save命令會阻塞Redis伺服器程序,直到RDB檔案建立為止,在伺服器程序阻塞期間,伺服器不能處理任何命令請求。
bgsave命令派生出子程序,,然後由子程序建立RDB檔案,父程序繼續處理請求。
建立RDB檔案實際rdb.c/rdbLoad函式完成的
自動間隔性儲存
redis伺服器會通過使用者配置save選項,每隔一段時間去執行一下bgsave命令;預設的配置檔案redis.conf,關於rdb部分的配置如下:
1 ################################ SNAPSHOTTING ################################ 2 # 3 # Save the DB on disk: 4 # 5 # save <seconds> <changes> 6 # 7 # Will save the DB if both the given number of seconds and the given 8 # number of write operations against the DB occurred. 9 # 10 # In the example below the behaviour will be to save: 11 # after 900 sec (15 min) if at least 1 key changed 12 # after 300 sec (5 min) if at least 10 keys changed 13 # after 60 sec if at least 10000 keys changed 14 # 15 # Note: you can disable saving completely by commenting out all "save" lines. 16 # 17 # It is also possible to remove all the previously configured save 18 # points by adding a save directive with a single empty string argument 19 # like in the following example: 20 # 21 # save "" 22 23 save 900 1 24 save 300 10 25 save 60 10000 26 27 # By default Redis will stop accepting writes if RDB snapshots are enabled 28 # (at least one save point) and the latest background save failed. 29 # This will make the user aware (in a hard way) that data is not persisting 30 # on disk properly, otherwise chances are that no one will notice and some 31 # disaster will happen. 32 # 33 # If the background saving process will start working again Redis will 34 # automatically allow writes again. 35 # 36 # However if you have setup your proper monitoring of the Redis server 37 # and persistence, you may want to disable this feature so that Redis will 38 # continue to work as usual even if there are problems with disk, 39 # permissions, and so forth. 40 stop-writes-on-bgsave-error yes 41 42 # Compress string objects using LZF when dump .rdb databases? 43 # For default that's set to 'yes' as it's almost always a win. 44 # If you want to save some CPU in the saving child set it to 'no' but 45 # the dataset will likely be bigger if you have compressible values or keys. 46 rdbcompression yes 47 48 # Since version 5 of RDB a CRC64 checksum is placed at the end of the file. 49 # This makes the format more resistant to corruption but there is a performance 50 # hit to pay (around 10%) when saving and loading RDB files, so you can disable it 51 # for maximum performances. 52 # 53 # RDB files created with checksum disabled have a checksum of zero that will 54 # tell the loading code to skip the check. 55 rdbchecksum yes 56 57 # The filename where to dump the DB 58 dbfilename dump.rdb 59 60 # The working directory. 61 # 62 # The DB will be written inside this directory, with the filename specified 63 # above using the 'dbfilename' configuration directive. 64 # 65 # The Append Only File will also be created inside this directory. 66 # 67 # Note that you must specify a directory here, not a file name. 68 dir ./
主要有三條策略,滿足任意一個,就會執行bgave命令:
伺服器在900秒之內,對資料庫至少修改了1次
伺服器在300秒之內,對資料庫至少修改了10次
伺服器在60秒之內,對資料庫至少修改了10000次
優缺點
優點:
適合大規模的資料恢復
對資料完整性和一致性要求不高
缺點:
會丟失最後一次修改的資料
fork會產生額外消耗
AOF持久化(Append Only File)
與RDB通過鍵值對來記錄資料庫狀態不同,AOF是通過Redis伺服器所執行的寫命令來記錄資料庫狀態的。
AOF持久化的實現
AOF持久化功能的實現可以分為命令追加(append)、檔案寫入、檔案同步(sync)三個步驟。
aof在redis.conf配置檔案的:
1 ############################## APPEND ONLY MODE ############################### 2 3 # By default Redis asynchronously dumps the dataset on disk. This mode is 4 # good enough in many applications, but an issue with the Redis process or 5 # a power outage may result into a few minutes of writes lost (depending on 6 # the configured save points). 7 # 8 # The Append Only File is an alternative persistence mode that provides 9 # much better durability. For instance using the default data fsync policy 10 # (see later in the config file) Redis can lose just one second of writes in a 11 # dramatic event like a server power outage, or a single write if something 12 # wrong with the Redis process itself happens, but the operating system is 13 # still running correctly. 14 # 15 # AOF and RDB persistence can be enabled at the same time without problems. 16 # If the AOF is enabled on startup Redis will load the AOF, that is the file 17 # with the better durability guarantees. 18 # 19 # Please check http://redis.io/topics/persistence for more information. 20 21 appendonly no 22 23 # The name of the append only file (default: "appendonly.aof") 24 25 appendfilename "appendonly.aof" 26 27 # The fsync() call tells the Operating System to actually write data on disk 28 # instead of waiting for more data in the output buffer. Some OS will really flush 29 # data on disk, some other OS will just try to do it ASAP. 30 # 31 # Redis supports three different modes: 32 # 33 # no: don't fsync, just let the OS flush the data when it wants. Faster. 34 # always: fsync after every write to the append only log. Slow, Safest. 35 # everysec: fsync only one time every second. Compromise. 36 # 37 # The default is "everysec", as that's usually the right compromise between 38 # speed and data safety. It's up to you to understand if you can relax this to 39 # "no" that will let the operating system flush the output buffer when 40 # it wants, for better performances (but if you can live with the idea of 41 # some data loss consider the default persistence mode that's snapshotting), 42 # or on the contrary, use "always" that's very slow but a bit safer than 43 # everysec. 44 45 # More details please check the following article: 46 # http://antirez.com/post/redis-persistence-demystified.html 47 # 48 # If unsure, use "everysec". 49 50 # appendfsync always 51 appendfsync everysec 52 # appendfsync no 53 54 # When the AOF fsync policy is set to always or everysec, and a background 55 # saving process (a background save or AOF log background rewriting) is 56 # performing a lot of I/O against the disk, in some Linux configurations 57 # Redis may block too long on the fsync() call. Note that there is no fix for 58 # this currently, as even performing fsync in a different thread will block 59 # our synchronous write(2) call. 60 # 61 # In order to mitigate this problem it's possible to use the following option 62 # that will prevent fsync() from being called in the main process while a 63 # BGSAVE or BGREWRITEAOF is in progress. 64 # 65 # This means that while another child is saving, the durability of Redis is 66 # the same as "appendfsync none". In practical terms, this means that it is 67 # possible to lose up to 30 seconds of log in the worst scenario (with the 68 # default Linux settings). 69 # 70 # If you have latency problems turn this to "yes". Otherwise leave it as 71 # "no" that is the safest pick from the point of view of durability. 72 73 no-appendfsync-on-rewrite no 74 75 # Automatic rewrite of the append only file. 76 # Redis is able to automatically rewrite the log file implicitly calling 77 # BGREWRITEAOF when the AOF log size grows by the specified percentage. 78 # 79 # This is how it works: Redis remembers the size of the AOF file after the 80 # latest rewrite (if no rewrite has happened since the restart, the size of 81 # the AOF at startup is used). 82 # 83 # This base size is compared to the current size. If the current size is 84 # bigger than the specified percentage, the rewrite is triggered. Also 85 # you need to specify a minimal size for the AOF file to be rewritten, this 86 # is useful to avoid rewriting the AOF file even if the percentage increase 87 88 # Specify a percentage of zero in order to disable the automatic AOF 89 # rewrite feature. 90 91 auto-aof-rewrite-percentage 100 92 auto-aof-rewrite-min-size 64mb 93 94 # An AOF file may be found to be truncated at the end during the Redis 95 # startup process, when the AOF data gets loaded back into memory. 96 # This may happen when the system where Redis is running 97 # crashes, especially when an ext4 filesystem is mounted without the 98 # data=ordered option (however this can't happen when Redis itself 99 # crashes or aborts but the operating system still works correctly). 100 # 101 # Redis can either exit with an error when this happens, or load as much 102 # data as possible (the default now) and start if the AOF file is found 103 # to be truncated at the end. The following option controls this behavior. 104 # 105 # If aof-load-truncated is set to yes, a truncated AOF file is loaded and 106 # the Redis server starts emitting a log to inform the user of the event. 107 # Otherwise if the option is set to no, the server aborts with an error 108 # and refuses to start. When the option is set to no, the user requires 109 # to fix the AOF file using the "redis-check-aof" utility before to restart 110 # the server. 111 # 112 # Note that if the AOF file will be found to be corrupted in the middle 113 # the server will still exit with an error. This option only applies when 114 # Redis will try to read more data from the AOF file but not enough bytes 115 # will be found. 116 aof-load-truncated yesaof
AOF重寫
隨著伺服器時間的流逝,檔案的體積越來越大,體積過大的AOF檔案對redis伺服器、甚至整個宿主計算機造成影響。並且AOF檔案的體積越大,使用AOF檔案進行資料還原所需的時間越多。
為了解決AOF檔案體重膨脹的問題,redis提供了AOF檔案重寫(rewrite)的功能。
觸發機制
redis會記錄上次重寫時AOF的大小,預設配置是當AOF檔案大小是上次rewrite後大小的一倍且大於64M;預設配置如下:
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
悄悄告訴你們個小祕密:可能在面試時,面試官會吹噓公司很牛,redis用的出神入化,當你入職之後,可以悄悄看看“auto-aof-rewrite-min-size 64mb”,預設64M大小,根本不夠用,告訴發展的公司起碼要3G起。
優缺點
優點:
配置靈活,可以選擇多種方式進行持久化
缺點:
相同資料集的資料而言,aof檔案要遠大於rdb檔案,恢復速度慢於rdb