nginx open files limits 導致大量錯誤信息
nginx error.log 中出現大量如下錯誤信息:
[root@localhost nginx]# grep -aP ‘^20.* \[crit\]‘ error.log
2017/03/14 12:06:31 [crit] 3549#0: accept4() failed (24: Too many open files)
[root@localhost nginx]# grep -aP ‘^20.* \[alert\]‘ error.log
2017/03/14 16:04:27 [alert] 3551#0: *84168270 socket() failed (24: Too many open files) while connecting to upstream, client: 1.1.1.1, server:...
由於系統limits open files 限制導致以上錯誤,所以:
root@open-mk-push-1:~# cat /etc/security/limits.conf * soft nofile 655360 * hard nofile 655360 root soft nofile 655360 root hard nofile 655360 * soft core unlimited * hard core unlimited root soft core unlimited root@open-mk-push-1:~# ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 512240 max locked memory (kbytes, -l) 64 max memory size (kbytes,-m) unlimited open files (-n) 655360 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 512240 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
但以上只是系統的open files 限制設置為655350;而nginx的open files 是否繼承系統open files 設置還需要重新啟動nginx 進程。可以通過以下命令查看:
root@open-mk-push-1:~# cat /proc/`ps -ef | grep nginx|grep -v grep|head -1|awk ‘{print $2}‘`/limits Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 512240 512240 processes Max open files 1024 4096 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 512240 512240 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us
可以看到nginx進程的Max open files的Soft Limit和Hard Limit分別只有1024和4096,這是一個初始的值可以說是很低了,故之前出現的Too many open files就可以理解了。
總結:ulimit -a && cat /proc/`ps -ef | grep nginx|grep -v grep| head -1 | awk ‘{print $2}‘`/limits 必須保持一致,否則會導致大量連接失敗,此時需要重啟nginx 進程 。
由於我們nginx是通過supervisor納管的,所以得重啟supervisor這個服務才能使nginx的max open files與系統值保持一致。
nginx open files limits 導致大量錯誤信息