linux之實時備份rsync+inotify
阿新 • • 發佈:2018-11-06
rsync缺點/不足:
1.rsync在同步資料時,需要掃描所有檔案後進行比對,進行差量傳輸。如果檔案數量達到了百萬甚至千萬量級,掃描所有檔案將是非常耗時的,並且正在發生變化的往往是其中很少的一部分,這是非常低效的方式。
2.rsync不能實時的去監測、同步資料,雖然它可以通過linux守護程序的方式進行觸發同步,但是兩次觸發動作一定會有時間差,這樣就導致了服務端和客戶端資料可能出現不一致,無法在應用故障時完全的恢復資料。
在使用rsync首次全量同步後,結合inotify對源目錄進行實時監控,只有有檔案變動或新檔案產生,就會立刻同步到目標目錄下,非常高效使用!
inotify :建立一個檔案描述符,附加一個或多個監視器(一個監視器是一個路徑和一組事件),然後用read方法從描述符獲取事件,read並不會用完整個週期,它是事件發生之前是被阻塞的。
檔案描述符:linux核心為了更優秀的管理被開啟的檔案建立的索引,是一個非負整數。用於指代被開啟的檔案,所有執行io的操作 系統呼叫都是通過檔案描述符。0表示標準輸入,1表示標準輸出,2表示標準錯誤輸出。
檢視linux核心版本
需要看當前linux是否支援inotify
[[email protected] ~]# uname -a Linux Rsync-139 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux [[email protected] ~]# ls -lsart /proc/sys sys/ sysrq-trigger sysvipc/ [[email protected] ~]# ls -lsart /proc/sys/fs/inotify/ 總用量 0 0 dr-xr-xr-x 0 root root 0 11月 5 15:31 .. 0 dr-xr-xr-x 0 root root 0 11月 6 16:12 . 0 -rw-r--r-- 1 root root 0 11月 6 16:12 max_user_watches 0 -rw-r--r-- 1 root root 0 11月 6 16:12 max_user_instances 0 -rw-r--r-- 1 root root 0 11月 6 16:12 max_queued_events
安裝
[[email protected] ~]# tar -zxvf inotify-tools-3.14.tar.gz -C /usr/local/src/ [[email protected] ~]# ./configrue --prefix=/usr/local/inotify [[email protected] ~]# make &&make install
需要加環境變數
vim /etc/profile 最低行加:/usr/local/inotify/bin/
重新載入配置檔案
. /etc/profile 或者 source /etc/profile
檢視幫助資訊
inotifywait --help
[[email protected] script]# inotifywait --help inotifywait 3.14 Wait for a particular event on a file or set of files. Usage: inotifywait [ options ] file1 [ file2 ] [ file3 ] [ ... ] Options:可用選項 -h|--help Show this help text. @<file> Exclude the specified file from being watched. --exclude <pattern> Exclude all events on files matching the extended regular expression <pattern>. --excludei <pattern> Like --exclude but case insensitive. -m|--monitor Keep listening for events forever. Without this option, inotifywait will exit after one event is received. -d|--daemon Same as --monitor, except run in the background logging events to a file specified by --outfile. Implies --syslog. -r|--recursive Watch directories recursively. --fromfile <file> Read files to watch from <file> or `-' for stdin. -o|--outfile <file> Print events to <file> rather than stdout. -s|--syslog Send errors to syslog rather than stderr. -q|--quiet Print less (only print events). -qq Print nothing (not even events). --format <fmt> Print using a specified printf-like format string; read the man page for more details. --timefmt <fmt> strftime-compatible format string for use with %T in --format string. -c|--csv Print events in CSV format. -t|--timeout <seconds> When listening for a single event, time out after waiting for an event for <seconds> seconds. If <seconds> is 0, inotifywait will never time out. -e|--event <event1> [ -e|--event <event2> ... ] Listen for specific event(s). If omitted, all events are listened for. Exit status:可以監控的事件 0 - An event you asked to watch for was received. 1 - An event you did not ask to watch for was received (usually delete_self or unmount), or some error occurred. 2 - The --timeout option was given and no events occurred in the specified interval of time. Events: access file or directory contents were read modify file or directory contents were written attrib file or directory attributes changed close_write file or directory closed, after being opened in writeable mode close_nowrite file or directory closed, after being opened in read-only mode close file or directory closed, regardless of read/write mode open file or directory opened moved_to file or directory moved to watched directory moved_from file or directory moved from watched directory move file or directory moved to or from watched directory create file or directory created within watched directory delete file or directory deleted within watched directory delete_self file or directory was deleted unmount file system containing file or directory unmounted
常用
-r 遞迴(目錄)
-m 永久監聽
-d 後臺執行
-q 靜默,只輸出較少的資訊
編寫指令碼進行實時備份
vim inotify.sh inotifywait -mrq --timefmt '%d%m%y %H:%M' --format '%T %w%f%e' -e close_write,modify,delete,create,attrib,move /var/log/ |while read file do rsync -az --delete-before /var/log/ [email protected]::WebA/ --password-file=/etc/WebA.pass done ./inotify &放入後臺執行即可
注:
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at
main.c(1505)是因為在同步的時候,源目錄下有軟連結檔案!
rsync同步軟連結檔案,應該加引數-l
參考文件:https://www.cnblogs.com/kevingrace/p/6001252.html 此文件非常詳細。