運維監控系統之Open-Falcon

阿新 • • 發佈：2019-02-18

一、Open-Falcon介紹

1、監控系統，可以從運營級別（基本配置即可），以及應用級別（二次開發，通過埠進行日誌上報），對伺服器、作業系統、中介軟體、應用進行全面的監控，及報警，對我們的系統正常執行的作用非常重要。

2、基礎監控

CPU、Load、記憶體、磁碟、IO、網路相關、核心引數、ss 統計輸出、埠採集、核心服務的程序存活資訊採集、關鍵業務程序資源消耗、NTP offset採集、DNS解析採集，這些指標，都是open-falcon的agent元件直接支援的。

對於這些基礎監控選項全部理解透徹的時刻，也就是對Linux執行原理及命令進階的時刻。

3、第三方監控

術業有專攻，執行在OS上的應用甚多，Open-Falcon的開發團隊不可能把所有的第三方應用的監控全部做完，這個就需要開源社群提供更多的外掛，當前對於很多常用的第三方應用都有相關外掛了。

4、JVM監控

對於Java作為主要開發語言的大多數公司，對於JVM的監控不可或缺。

每個JVM應用的引數，比如GC、類載入、JVM記憶體、程序、執行緒，都可以上報給Falcon，而這些引數的獲得，都可以通過MxBeans實現。

5、業務應用監控

對於業務需要監控的介面，比如響應時間等。可以根據業務的需要，上報相關資料到Falcon，並通過Falcon檢視結果。

二、Zabbix、Nagios、Open-Falcon這3大開源運維監控工具的比較

三、Open-Falcon編寫的整個腦洞歷程

官網：

說明：Open-Falcon部署分為前端（客戶端）和後端（服務端）

四、後端部署（服務端）

1、系統環境

[root@open-falcon-server ~]# cat /etc/redhat-release

CentOS Linux release 7.2.1511 (Core)

2、系統優化

#安裝下載軟體

yum install wget -y

#更換aliyun源

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup

wget -O /etc/yum.repos.d/CentOS-Base

.repo http://mirrors.aliyun.com/repo/Centos-7.repo

#下載epel源

yum install epel-release.noarch -y

rpm -Uvh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm

yum clean all

yum makecache

#下載常用軟體

yum install git telnet net-tools tree nmap sysstat lrzsz dos2unix tcpdump ntpdate -y

#配置時間同步

ntpdate cn.pool.ntp.org

#更改主機名

hostnamectl set-hostname open-falcon-server

hostname open-falcon-server

#開啟快取

sed -i 's#keepcache=0#keepcache=1#g' /etc/yum.conf

grep keepcache /etc/yum.conf

#關閉selinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

setenforce 0

#關閉防火牆

systemctl stop firewalld.service

systemctl disable firewalld.service

3、軟體環境準備

#安裝 redis

1	`yum` `install` `redis -y`

#redis常用命令

redis-server redis 服務端

redis-cli　　　　redis 命令列客戶端

redis-benchmark redis 效能測試工具

redis-check-aof 　　AOF檔案修復工具

redis-check-dump RDB檔案修復工具

redis-sentinel 　　 Sentinel 服務端

#啟動redis

#啟動redis

[root@open-falcon-server ~]# redis-server &

[1] 3308

[root@open-falcon-server ~]# 3308:C 27 Oct 08:37:26.435 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf

3308:M 27 Oct 08:37:26.436 * Increased maximum number of open files to 10032 (it was originally set to 1024).

_._

_.-``__ ''-._

_.-`` `. `_. ''-._ Redis 3.2.10 (00000000/0) 64 bit

.-`` .-```. ```\/ _.,_ ''-._

( ' , .-` | `, ) Running in standalone mode

|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379

| `-._ `._ / _.-' | PID: 3308

`-._ `-._ `-./ _.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' | http://redis.io

`-._ `-._`-.__.-'_.-' _.-'

|`-._`-._ `-.__.-' _.-'_.-'|

| `-._`-._ _.-'_.-' |

`-._ `-._`-.__.-'_.-' _.-'

`-._ `-.__.-' _.-'

`-._ _.-'

`-.__.-'

3308:M 27 Oct 08:37:26.437 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

3308:M 27 Oct 08:37:26.437 # Server started, Redis version 3.2.10

3308:M 27 Oct 08:37:26.437 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

3308:M 27 Oct 08:37:26.438 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

3308:M 27 Oct 08:37:26.438 * The server is now ready to accept connections on port 6379

#安裝mysql

[root@open-falcon-server ~]#wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm

[root@open-falcon-server ~]# ll

total 12

-rw-------. 1 root root 1262 Oct 27 2017 anaconda-ks.cfg

-rw-r--r--. 1 root root 6140 Nov 12 2015 mysql-community-release-el7-5.noarch.rpm

[root@open-falcon-server ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm

Preparing... ################################# [100%]

Updating / installing...

1:mysql-community-release-el7-5 ################################# [100%]

[root@open-falcon-server ~]# ll /etc/yum.repos.d/

total 48

-rw-r--r--. 1 root root 2573 Nov 21 2014 CentOS-Base.repo

-rw-r--r--. 1 root root 1664 Dec 9 2015 CentOS-Base.repo.backup

-rw-r--r--. 1 root root 1309 Dec 9 2015 CentOS-CR.repo

-rw-r--r--. 1 root root 649 Dec 9 2015 CentOS-Debuginfo.repo

-rw-r--r--. 1 root root 290 Dec 9 2015 CentOS-fasttrack.repo

-rw-r--r--. 1 root root 630 Dec 9 2015 CentOS-Media.repo

-rw-r--r--. 1 root root 1331 Dec 9 2015 CentOS-Sources.repo

-rw-r--r--. 1 root root 1952 Dec 9 2015 CentOS-Vault.repo

-rw-r--r--. 1 root root 951 Jun 24 23:04 epel.repo

-rw-r--r--. 1 root root 1050 Jun 24 23:04 epel-testing.repo

-rw-r--r--. 1 root root 1209 Jan 29 2014 mysql-community.repo

-rw-r--r--. 1 root root 1060 Jan 29 2014 mysql-community-source.repo

[root@open-falcon-server ~]#yum install -y mysql-server<br>

#啟動mysql並設定開機自啟動

[root@open-falcon-server ~]# systemctl start mysqld.service

[root@open-falcon-server ~]# systemctl enable mysqld.service

[root@open-falcon-server ~]# mysql -uroot -p

Enter password:

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 2

Server version: 5.6.38 MySQL Community Server (GPL)

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit

Bye

#檢查服務

[root@open-falcon-server ~]# netstat -lntup

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name

tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 3308/redis-server *

tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1443/sshd

tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1634/master

tcp6 0 0 :::3306 :::* LISTEN 3283/mysqld

tcp6 0 0 :::6379 :::* LISTEN 3308/redis-server *

tcp6 0 0 :::22 :::* LISTEN 1443/sshd

tcp6 0 0 ::1:25 :::* LISTEN 1634/master

udp 0 0 0.0.0.0:68 0.0.0.0:* 2042/dhclient

udp 0 0 0.0.0.0:4817 0.0.0.0:* 2042/dhclient

udp6 0 0 :::20380 :::* 2042/dhclient

4、初始化MySQL表結構（備註：匯入表結構後，注意檢查）

cd /tmp/ && git clone https://github.com/open-falcon/falcon-plus.git

cd /tmp/falcon-plus/scripts/mysql/db_schema/

mysql -h 127.0.0.1 -u root -p < 1_uic-db-schema.sql

mysql -h 127.0.0.1 -u root -p < 2_portal-db-schema.sql

mysql -h 127.0.0.1 -u root -p < 3_dashboard-db-schema.sql

mysql -h 127.0.0.1 -u root -p < 4_graph-db-schema.sql

mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql

rm -rf /tmp/falcon-plus/

#設定資料庫密碼 (備註：使用者名稱：root ,密碼: password )

[root@open-falcon-server open-falcon]# mysqladmin -uroot password "password"

Warning: Using a password on the command line interface can be insecure.

#檢查匯入的資料庫表

[root@open-falcon-server open-falcon]# mysql -uroot -p

Enter password:

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 35

Server version: 5.6.38 MySQL Community Server (GPL)

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;

+--------------------+

| Database |

+--------------------+

| information_schema |

| alarms |

| dashboard |

| falcon_portal |

| graph |

| mysql |

| performance_schema |

| uic |

+--------------------+

8 rows in set (0.00 sec)

mysql> exit

Bye

5、配置Go語言開發環境

#安裝go語言開發包（備註：安裝需要epel源）

1	`yum` `install` `golang -y`

#檢查版本

[root@open-falcon-server ~]# go version

go version go1.8.3 linux/amd64

#檢視go的安裝路徑

[root@open-falcon-server ~]# find / -name go

/etc/alternatives/go

/var/lib/alternatives/go

/usr/bin/go

/usr/lib/golang/src/cmd/go #需要的是這個路徑

/usr/lib/golang/src/go

/usr/lib/golang/bin/go

/usr/lib/golang/pkg/linux_amd64/go

#設定環境變數GOROOT和GOPATH

export GOROOT=/usr/lib/golang

export GOPATH=/home

6、下載編譯好的二進位制版本

https://github.com/open-falcon/falcon-plus/releases

#開始安裝open-falcon

[root@open-falcon-server ~]# export FALCON_HOME=/home/work

[root@open-falcon-server ~]# export WORKSPACE=$FALCON_HOME/open-falcon

[root@open-falcon-server ~]# mkdir -p $WORKSPACE

#上傳軟體

[root@open-falcon-server ~]# ll

-rw-r--r--. 1 root root 40031472 Oct 29 2017 open-falcon-v0.2.1.tar.gz

#解壓

[root@open-falcon-server ~]# tar -xf open-falcon-v0.2.1.tar.gz -C $WORKSPACE

[root@open-falcon-server ~]# cd $WORKSPACE

<br>#檢視解壓結果

[root@open-falcon-server open-falcon]# ll

total 3896

drwxrwxr-x. 7 501 501 67 Aug 15 19:16 agent

drwxrwxr-x. 5 501 501 40 Aug 15 19:16 aggregator

drwxrwxr-x. 5 501 501 40 Aug 15 19:16 alarm

drwxrwxr-x. 6 501 501 51 Aug 15 19:16 api

drwxrwxr-x. 5 501 501 40 Aug 15 19:16 gateway

drwxrwxr-x. 6 501 501 51 Aug 15 19:16 graph

drwxrwxr-x. 5 501 501 40 Aug 15 19:16 hbs

drwxrwxr-x. 5 501 501 40 Aug 15 19:16 judge

drwxrwxr-x. 5 501 501 40 Aug 15 19:16 nodata

-rwxrwxr-x. 1 501 501 3987469 Aug 15 19:16 open-falcon

lrwxrwxrwx. 1 501 501 16 Aug 15 19:16 plugins -> ./agent/plugins/

lrwxrwxrwx. 1 501 501 15 Aug 15 19:16 public -> ./agent/public/

drwxrwxr-x. 5 501 501 40 Aug 15 19:16 transfer

7、修改配置檔案cfg.json

坑：

部分模組依賴連線資料庫，因為如果不修改配置檔案，aggregator模組會出現無法啟動，graph、hbs、nodata、api、alarm模組會出現開啟不報錯但是狀態為開啟失敗的情況。

如果需要每個模組都能正常啟動，需要將上面模組的cfg.json的資料庫資訊進行修改。根據本教程的配置，需要修改配置檔案所在的目錄：

模組	配置檔案所在路徑
aggregator	/home/work/aggregator/config/cfg.json
graph	/home/work/graph/config/cfg.json
hbs	/home/work/hbs/config/cfg.json
nodata	/home/work/nodata/config/cfg.json
api	/home/work/api/config/cfg.json
alarm	/home/work/alarm/config/cfg.json

1）修改aggregator的配置檔案

vim /home/work/aggregator/config/cfg.json