1 概述

服務器上的日誌，處理記錄在本機上，也可以搭建專門收集log的服務器，方便分析。本文將介紹如何啟用rsyslog的TCP和UDP的514端口進行網絡日誌收集，並通過rsyslog-mysql這個工具將日誌記錄到mysql數據庫中， 再通過loganalyzer工具將日誌用頁面的形式展示出來

最後，本文將附上一鍵化部署安裝log服務器的腳本

2 配置rsyslog成為日誌服務器

要使得rsyslog成功日誌服務器，從而收集其他主機的日誌，要將TCP和UDP的514端口打開，通過修改配置文件/etc/rsyslog實現

配置如下

vim  /etc/rsyslog
#### MODULES ####
# Provides UDP syslog reception
$ModLoadim  udp
$UDPServerRun  514
# Provides TCP syslog reception
$ModLoadim  tcp
$InputTCPServerRun  514

例子

設置一臺機器為serverlog,專門用來收集日誌，其他機器為客戶端，不記錄日誌

假設log server為172.18.50.75這臺，客戶端上定義一個facility為local2，配置如下

服務器端

vim  /etc/rsyslog.conf
$ModLoad imudp#啟用imudp模塊
$UDPServerRun 514#開啟UDP 514端口用來收集日誌
$ModLoad imtcp #啟用imtcp模塊
$InputTCPServerRun 514 #開啟TCP514端口用來收集日誌
local2.*   /var/serverlog/6Alocal2.log

#定義facility為local2的所有基本的日誌都記錄到/var/serverlog/6Alocal2.log，註意，這裏6Alocal2.log這個log文件可以不用創建，只需重啟rsyslog的服務，下次當滿足條件的日誌要記錄的時候，系統就會自動生成。註意如果沒有生成，可能是服務器端沒有重啟服務導致。

重啟服務

service rsyslog restart

客戶端

vim  /etc/rsyslog.conf
local2.*   @172.18.50.75

#表示將facility為2的log記錄到172.18.50.75這臺機器上，其中，@表示UDP協議，@@表示UDP協議

重啟服務

service rsyslog restart

3 rsyslog-mysql將日誌記錄於MySQL中

安裝rsyslog-mysql（epel源中），將日誌記錄到mysql數據庫中，安裝過程會有腳本mysql-createDB.sql生成，需要在mysql服務器上執行該腳本。生成相關的庫和表

.(1)準備MySQLServer

.(2)在rsyslog服務器上安裝mysql模塊相關的程序包（epel源中）

yum install rsyslog-mysql

.(3)為rsyslog創建數據庫及表；安裝rsyslog-mysql（epel源中），會有腳本/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql生成，將這個腳本導入到數據庫中，用來生成表和數據庫，

mysql  -uUSERNAME  -hHOST  -pPASSWORD D < /usr/share/doc/rsyslog-7.4.7/mysql-createDB.sql

#執行數據庫的腳本，表示用數據庫賬號直接執行mysql-createDB.sql這個腳本，註意用<小於號表示將mysql-createDB.sql這個腳本導入數據庫

.(4)在mysqlserver上授權rsyslog能連接至當前服務器

mysql> GRANT ALL ON Syslog.*  TO ‘USER‘@‘HOST‘  IDENTIFIED BY ‘PASSWORD‘;

.(5)配置rsyslog，將日誌保存到mysql中

#### MODULES ####
$ModLoadom  mysql
#### RULES ####
facility.priority    :ommysql:DBHOST,DBNAME,DBUSER, PASSWORD

:ommysql:這個是模塊名，將來local2這個模式的日誌將會安裝以下的定義將日誌記錄到106的服務器上

4 通過loganalyzer展示數據庫中的日誌

loganalyzer基於php開發的，可以讀取數據庫的內容，以報表形式展現出來

.(1)在rsyslog服務器上準備amp或nmp組合

yum install httpd php  php-mysql  php-gd

php-mysql連接數據庫用的

php-gd畫圖用的，epel源中

.(2)安裝LogAnalyzer

tar xf  loganalyzer-4.1.5.tar.gz

cp -a loganalyzer-4.1.5/src /var/www/html/log

#只拷貝該軟件loganalyzer-4.1.5下src的目錄就可以了

cd /var/www/html/log
touch config.php
chmod  666 config.php

#這個config.php文件只需要創建並賦予寫的權限即可，不需要添加內容，該配置文件的內容將在重啟服務後，在網頁進行配置的時候寫入。

.(3)配置loganalyzer

#重啟httpd服務

systemctl start httpd.service

輸入http://websrv/log進行配置，寫入的內容將入記錄到config.php這個文件裏，註意MySQL Native, Syslog Fields,Monitorware的選擇

打開網頁，點擊下一步進行配置

Source Type選擇MYSQL Native，出現出現的配置。以下的信息中，默認的信息有問題，如表的大小寫，如果這裏寫錯信息，需要更改config.php這個文件，不修改的話，可以直接刪掉該配置文件，重新創建並寫入。

.(4)安全加強

cd  /var/www/html/loganalyzer
chmod 644 config.php

#將config.php改成只讀模式，防止被其他人修改了相關配置。這一步建議操作

5 一鍵安裝腳本

腳本使用前提

註意要配置好本地yum源和epel源，其中epel源建議使用sohu的epel源

loganalyzer這個軟件包由於是解壓安裝，所以要提前準備好，或者有可以下載該服務包的路徑。腳本中作者已經提前將這個服務包放到了自制的http服務器中http://172.18.50.75上。

有兩個腳本

腳本一，用於一鍵化安裝log服務器

腳本二，用於修改客戶端的/etc/rsyslog.conf這個配置文件，使得log直接記錄到對應的log服務器端

腳本一：一鍵安裝log服務器

#!/bin/bash
#
#*****************************************************************************************
#Author:               Sunny
#Date:                 2017-10-16
#FileName:             auto_install_log_server.sh
#version:              1.0
#Your change info:      
#Description:          For auto install log server by rsyslog-mysql and LogAnalyzer
#DOC URL:               
#Copyright(C):         2017  All rihts reserved
#*****************************************************************************************

os_version=`cat /etc/system-release | grep -o " [0-9]"| cut -d " " -f2`
time=`date +%Y%m%d%H%M`
ip=$(ifconfig  | awk ‘/inet /{print $2}‘| awk -F : ‘{print $NF}‘| head -1)
package=‘loganalyzer-4.1.5.tar.gz‘
  [ -e /root/package/package."$time" ] || mkdir -p /root/package/package."$time";
  echo "$package" | tr -s " " "\n" &>/root/package/package.file
  echo

install_rsyslog_mysq(){

rpm -q rsyslog-mysql &>/dev/null || { yum -y install rsyslog-mysql &>/dev/null && echo "rsyslog-mysql is install complete" || { echo "rsyslog-mysql is not install,check yum source";exit; }; }
read -p "Input your sql admin user(default:root): " mysqladmin
mysqladmin=${mysqladmin:-root}
read -p "Input your sql admin user password: " adminpass
createdb=$(rpm -ql rsyslog-mysql | grep createDB.sql)
mysql -u$mysqladmin -p$adminpass < $createdb
/usr/bin/mysql -u$mysqladmin -p$adminpass <<EOF
grant all on Syslog.* to [email protected]‘%‘ identified by ‘Pass123456‘;
EOF
}
config_rsyslog(){
rpm -q rsyslog &>/dev/null || { yum -y install rsyslog &>/dev/null && echo "rsyslog is install complete" || { echo "rsyslog is not install,check yum source";exit; }; }
cat >>/etc/rsyslog.conf <<EOF
\$ModLoad imudp
\$UDPServerRun 514 
\$ModLoad imtcp
\$InputTCPServerRun 514 
\$ModLoad ommysql
local2.*                                                :ommysql:$ip,Syslog,logadmin,Pass123456
EOF
echo "rsyslog has been complete config,you can test if facility local2 can be log now."
echo "You can add facility.loglevel  :ommysql:$ip,Syslog,logadmin,Pass123456 to /etc/rsyslog.conf to log more log in the log server"
}

install_LogAnalyzer(){
echo "Now install loganalyzer"
tar xf /root/package/package."$time"/$package  -C  /usr/local/
cp -a /usr/local/loganalyzer-4.1.5/src  /var/www/html/log
touch /var/www/html/log/config.php
chmod  666 /var/www/html/log/config.php
echo -e "LogAnalyzer has been release,please run http://$ip/blog to config your log admin,defautl config is below\n\n
DBServer = "$ip";\n
DBName = ‘Syslog‘;\n
DBUser = ‘logadmin‘;\n
DBPassword = ‘Pass123456‘;\n
DBTableName = ‘SystemEvents‘;\n
"
}
restart_service(){
service rsyslog restart &>/dev/null && echo "rsyslog has been restart" || echo "Something wrong when restart rsyslog,please check"
service httpd restart &>/dev/null && echo "httpd has been restart" || echo "Something wrong when restart httpd,please check"
case $os_version in
6)
service mysqld restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check"
;;
7)
service mariadb restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check"
;;
*)
echo "Something wrong when restart mysql,please check"
exit
;;
esac
}


install_pack(){
rpm -q httpd &>/dev/null || { yum -y install httpd &>/dev/null && echo "httpd is install complete" || { echo "httpd is not install,check yum source";exit; }; }
rpm -q php &>/dev/null || { yum -y install php &>/dev/null && echo "php is install complete" || { echo "php is not install,check yum source";exit; }; }
rpm -q php-mysql &>/dev/null || { yum -y install php-mysql &>/dev/null && echo "php-mysql is install complete" || { echo "php-mysql is not install,check yum source";exit; }; }
rpm -q php-gd &>/dev/null || { yum -y install php-gd &>/dev/null && echo "php-gd is install complete" || { echo "php-gd is not install,check yum source";exit; }; }
}
download_LogAnalyzer(){

echo "You have two ways to get packages you want:"
echo "remote: You will download from remote server,default url is  http://192.168.32.75/source"
echo "local:  You have already prepare package in the local host"
echo
[ -e /root/package ] || mkdir -p /root/package;
read -p  "Your package in l(local) or r(remote)( r or l ): " choice
case $choice in
r)
  read -p "Please input the url where you want to download package(default:http://172.18.50.75/source): " url 
  url=${url:-http://172.18.50.75/source}
  wget -nv --spider $url 2>&1 | grep -o "200 OK" &>/dev/null || { echo "The url is wrong or could not be connect,the scirpt will exit,please check";exit; }
  echo "Now start to download pack,please wait a minute"
  cd  /root/package/package."$time"
  while read pack;
  do
  [ -e /root/package/package."$time"/$pack ] || wget -q "$url/$pack"
  [ -e /root/package/package."$time"/$pack ] && echo  "$pack had been success download !" || { echo "$pack did not been downloaded,it will exist,please check...";exit; }
  done</root/package/package.file;
 # rm -f /root/package/package.file;
;;
l)
   read -p "Please input the package directory(eg: /root/mariadb ): " localdir
  echo "Now start to copy pack to /root/package/package."$time",please wait a minute"
  cd  /root/package/package."$time"
  while read pack
  do
  [ -e /root/package/package."$time"/$pack ] || cp $localdir/$pack /root/package/package."$time" &>/dev/null;
  [ -e /root/package/package."$time"/$pack ] && echo  "$pack had been  success copy to /root/package/package.$time " || { echo "$pack did not copy to /root/package/package.$time,it will exist,please check...";exit; }
  done</root/package/package.file
 # rm -f /root/package/package.file;
;;
*)
  echo "Your input is not r or l ,and it is wrong input,the script will exit,please check"
  exit
;;
esac
}

echo 
echo "First of all,make your basic and epel source is ok,it is better sohu epel,you can run cmd  yum repolist to check your yum source"
echo
read -p "Is your epel ok?,answer y to continue,other to check your epel: " isepel
case $isepel in
y)
echo "Since your answer is y,I know your epel is ok,the script will continue..."
;;
*)
echo "For your answer is not y,it will exit,please check your epel"
echo -e "Below is how to config sohu epel,you can write it to /etc/yum.repo.d/sunny.repo\n\n
[sohu]\n
name=sohu-source\n
baseurl=http://mirrors.sohu.com/centos/\$releasever/os/\$basearch/\n
gpgcheck=1\n
enabled=0\n
gpgkey=http://mirrors.sohu.com/centos/\$releasever/os/\$basearch/RPM-GPG-KEY-CentOS-\$releasever\n"
echo
exit
;;
esac

case $os_version in 
6)
if rpm -q mysql-server &>/dev/null;then
echo "The  mysql-server is already install before"
else
rpm -q mysql &>/dev/null || { yum -y install mysql &>/dev/null && echo "mysql is install complete" || { echo "mysql is not install,check yum source";exit; }; }
rpm -q mysql-server &>/dev/null || { yum -y install mysql-server &>/dev/null && echo "mysql-server is install complete" || { echo "mysql is not install,check yum source";exit; }; }
service mysqld restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check"
/usr/bin/mysql_secure_installation;
fi
echo "Now install some relative package"
install_pack
install_rsyslog_mysq
config_rsyslog
download_LogAnalyzer
install_LogAnalyzer

;;

7)
if rpm -q mysql-server &>/dev/null;then
echo "The  mysql-server is already install before"
else
rpm -q mariadb &>/dev/null || { yum -y install mariadb &>/dev/null && echo "mysql is install complete" || { echo "mysql is not install,check yum source";exit; }; }
rpm -q mariadb-server &>/dev/null || { yum -y install mariadb-server &>/dev/null && echo "mysql-server is install complete" || { echo "mysql-server is not install,check yum source";exit; }; }
service mariadb restart &>/dev/null && echo "mysql has been restart" || echo "Something wrong when restart mysql,please check"
/usr/bin/mysql_secure_installation
fi
echo "Now install some relative package"
install_pack
install_rsyslog_mysq
config_rsyslog
download_LogAnalyzer
install_LogAnalyzer
;;

*)
echo "Your system is not centos6 or 7,please check"
exit
;;
esac

echo 
echo "All config is done now,Now restart service"

restart_service

echo "If all service is restart ok,you can test now,otherwise,you just to solve the restart problem,the test"
echo "test url is http://$ip/blog"

腳本二：一鍵配置客戶端/etc/rsyslog.conf

#!/bin/bash
#
#******************************************************************************
#Author:               Sunny
#Date:                 2017-10-15
#FileName:             auto_set_rsyslog_conf.sh
#version:              1.0
#Your change info:      
#Description:          For auto set rsylog_conf in client
#DOC URL:               
#Copyright(C):         2017  All rihts reserved
#*****************************************************************************

time=`date +%Y%m%d%H%M`
os_version=`cat /etc/system-release | grep -o " [0-9]"| cut -d " " -f2`

mv /etc/rsyslog.conf /etc/rsyslog.conf.$time.bak
read -p "Please input your log server ip(default:172.18.50.75): " ip
ip=${ip:-172.18.50.75}
echo ip is $ip
case $os_version in 

6)
cat >/etc/rsyslog.conf<<eof
\$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
\$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
\$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
\$IncludeConfig /etc/rsyslog.d/*.conf
*.info;mail.none;authpriv.none;cron.none               @$ip
authpriv.*                                             @$ip
mail.*                                                 @$ip
cron.*                                                 @$ip
*.emerg                                                 *
uucp,news.crit                                         @$ip
local7.*                                               @$ip 
eof
;;
7)
cat >/etc/rsyslog.conf<<eof
\$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
\$ModLoad imjournal # provides access to the systemd journal
\$WorkDirectory /var/lib/rsyslog
\$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
\$IncludeConfig /etc/rsyslog.d/*.conf
\$OmitLocalLogging on
\$IMJournalStateFile imjournal.state
*.info;mail.none;authpriv.none;cron.none                @$ip
authpriv.*                                              @$ip
mail.*                                                  @$ip
cron.*                                                  @$ip
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          @$ip
local7.*                                                @$ip 
eof
;;
*)
echo "The host is not centos6 or 7,it will exit now"
mv  /etc/rsyslog.conf.$time.bak /etc/rsyslog.conf 
exit
;;
esac

service rsyslog restart && echo "rsyslog has been restart" || echo "something wrong when restart rsyslog,please check"

6 總結

本文實現了一鍵安裝安裝log服務器，關於日誌級別的定義，以及日誌的存儲規則，可以參見博客服務器日誌 之 rsyslog和logrotate 概念介紹 http://ghbsunny.blog.51cto.com/7759574/1972977進行按需配置。

本文出自 “陽光運維” 博客，請務必保留此出處http://ghbsunny.blog.51cto.com/7759574/1973012

服務器日誌 之 一鍵部署log服務器