最近有臺雲上的伺服器需要釋放，然後上面跑的 gitlab 也要挪個地方，如在 docker 內執行，gitlab 映象大約 1.56G，需佔用 4G 以上的記憶體，因資源有限，於是借在其他的伺服器上搭建環境（可用記憶體小於4G），然鵝啟動的時候莫名出現 502，Excuse me？接著搜了一些 issue 部落格上的解決方案（如修改埠、重啟或 hup 某個服務）無果，後來在調整的過程中從日誌裡發現了一些資訊。

這次排錯過程主要是思路，視野開啟後會覺得豁然開朗，原來這其實是個小問題[尷尬]。

1、不清楚應用啟動的各服務以及用途，只會簡單檢視 status；

2、看到錯誤第一時間想到的是 Baidu（沒其他意思），找找 logpath 先看日誌不好嗎？

3、未認識到服務之間的關聯關係（比如 postgresql 與 unicorn 之間），前面一直知道 unicorn 啟動後沒正常監聽到埠，但是日誌並沒啥特別資訊（嗯，可能是因為看錯了檔案）[苦笑]

一、錯誤資訊

二、排錯過程

1、啟動 unicorn 未監聽埠

日誌路徑 :   /var/log/gitlab/unicorn/unicorn_stderr.log

PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?

  /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `initialize'
  /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `new'
  /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `connect'
  /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:242:in `initialize'
  /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `new'
  /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `postgresql_connection'

··· ···
··· ···
 

資訊顯示是因為連不上PG，所以啟動 postgresql 後重啟即可正常（嗯，是這樣的）。

2、postgresql down

down: postgresql: 0s, normally up, want up; run: log: (pid 623) 15816094s

通過 PG 的日誌路徑 ： /var/log/gitlab/postgresql/current 可以檢視到如下資訊

2018-11-01_08:18:09.49669 FATAL:  could not map anonymous shared memory: Cannot allocate memory
2018-11-01_08:18:09.49671 HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 4292984832 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
2018-11-01_08:18:09.49671 LOG:  database system is shut down
 

也可以通過命令 `gitlab-ctl tail postgresql`，得到一樣的資訊，於是可以確定問題。。

This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 4292984832 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
2018-11-01_07:52:06.63024 LOG:  database system is shut down

於是在配置檔案中對 postgresql 的 shared_buffers 和 max_connections 兩項進行了限制

[[email protected] ~]# cat /etc/gitlab/gitlab.rb |grep -v ^$ |grep -v ^#
external_url 'http://xxx.xxx.xxx.xxx.xxx:8090'
unicorn['worker_timeout'] = 60
unicorn['worker_processes'] = 3
unicorn['listen'] = 'xxx.xxx.xxx.xxx.xxx'
unicorn['port'] = 8870
postgresql['enable'] = true
postgresql['data_dir'] = "/var/opt/gitlab/postgresql/data"
postgresql['shared_buffers'] = "256MB"   # ! **recommend value is 1/4 of total RAM, up to 14GB.**      
postgresql['max_connections'] = 200
nginx['listen_addresses'] = ['*']
nginx['listen_port'] = 8090

配置完成儲存，之後更新配置，重啟應用即可。

gitlab-ctl reconfigure   # 更新配置
gitlab-ctl restart       # 重啟應用
gitlab-ctl status        # 檢視服務狀態

參考資料：

1. 502-Whoops, GitLab is taking too much time to respond

2. gitlab安裝以及安裝過程中遇到的問題

3. Postgresql down

4. centos7安裝部署gitlab伺服器

5. 我所遇到的GitLab 502問題的解決

6. HTTP 502: Whoops, GitLab is taking too much time to respond. Even after Server restart