1. 程式人生 > >分析網站監測log,得到所需資料

分析網站監測log,得到所需資料

 log:

2018/11/04 02:09:58 [11212] connect from joomla-site-yuanxi (19              2018/11/04 02:09:58 [11212] rsync to clxy_web/web/Sunday/ from [email protected] (1              2018/11/04 02:40:06 [12864] sent 10185 bytes  received 2895278 bytes  total size 2939121                 2018/11/04 02:09:58 [11212] sent 23 bytes  received 49 bytes  total size 0                 2018/11/04 02:09:58 [11214] connect from joomla-site-yuanxi (192.1             2018/11/04 02:09:58 [11214] rsync to clxy_web/web/current from

[email protected] (19              2018/11/04 02:09:58 [11214] receiving file list                 2018/11/04 02:09:59 [11214] sent 3719 bytes  received 329620 bytes  total size 217691468                 2018/11/04 02:14:58 [11482] connect from joomla-site-yuanxi (192          2018/11/04 02:14:58 [11482] rsync to office_web/web/Sunday/ from
[email protected]
(192.1               2018/11/04 02:14:58 [11482] receiving file list                 2018/11/04 02:14:58 [11482] sent 23 bytes  received 49 bytes  total size 0                 2018/11/04 02:14:58 [11484] connect from joomla-site-yuanxi (192.168.               2018/11/04 02:14:58 [11484] rsync to office_web/web/current from
[email protected]
(192.16              2018/11/04 02:14:58 [11484] receiving file list                 2018/11/04 02:14:58 [11484] sent 2620 bytes  received 254522 bytes  total size 69138548                 2018/11/04 02:24:58 [12031] connect from joomla-site-yuanxi (192.16             2018/11/04 02:24:58 [12032] connect from joomla-site-yuanxi (192.1        2018/11/04 02:24:58 [12033] connect from campus-wwww (192.16                2018/11/04 02:24:58 [12031] rsync to jgxy_web/web/Sunday/ from [email protected] (              2018/11/04 02:24:58 [12032] rsync to wfxy_web/web/Sunday/ from [email protected] (            2018/11/04 02:24:58 [12033] rsync to www_web/web/Sunday/ from [email protected] (192             2018/11/04 02:24:58 [12031] receiving file list                 2018/11/04 02:24:58 [12032] receiving file list                 2018/11/04 02:24:58 [12033] receiving file list                 2018/11/04 02:24:58 [12031] sent 23 bytes  received 49 bytes  total size 0                 2018/11/04 02:24:58 [12032] sent 23 bytes  received 49 bytes  total size 0                 2018/11/04 02:24:58 [12037] connect from joomla-site-yuanxi (1)                 2018/11/04 02:24:58 [12038] connect from joomla-site-yuanxi ()                 2018/11/04 02:24:58 [12033] sent 23 bytes  received 49 bytes  total size 0                 2018/11/04 02:24:58 [12039] connect from campus-wwww )                 2018/11/04 02:24:58 [12037] rsync to jgxy_web/web/current from [email protected] (.............)                 2018/11/04 02:24:58 [12038] rsync to wfxy_web/web/current from [email protected] ()                 2018/11/04 02:24:58 [12039] rsync to www_web/web/current from [email protected] ()                 2018/11/04 02:24:58 [12037] receiving file list                 2018/11/04 02:24:58 [12038] receiving file list                 2018/11/04 02:24:58 [12039] receiving file list                 2018/11/04 02:24:59 [12038] sent 4116 bytes  received 361511 bytes  total size 249585747                 2018/11/04 02:24:59 [12037] sent 4100 bytes  received 364106 bytes  total size 219381539                 2018/11/04 02:24:59 [12039] sent 42690 bytes  received 668352 bytes  total size 237763956                 2018/11/04 02:40:00 [12851] connect from joomla-site-yuanxi (..........................)                 2018/11/04 02:40:00 [12851] rsync to clxy_web/db/ from [email protected] (.......................)                 2018/11/04 02:40:00 [12851] receiving file list                 2018/11/04 02:40:00 [12851] sent 12861 bytes  received 4317598 bytes  total size 4544221                 2018/11/04 02:40:02 [12858] connect from joomla-site-yuanxi (192            2018/11/04 02:40:02 [12858] rsync to office_web/db/ from [email protected] (................)                 2018/11/04 02:40:02 [12858] receiving file list                 2018/11/04 02:40:02 [12858] sent 7065 bytes  received 48468 bytes  total size 1367105              2018/11/04 02:40:02 [12860] connect from campus-wwww (......................                2018/11/04 02:40:02 [12860] rsync to www_web/db/ from [email protected] (.................)                 2018/11/04 02:40:02 [12860] receiving file list                 2018/11/04 02:40:03 [12860] sent 18381 bytes  received 8569894 bytes  total size 9427853                 2018/11/04 02:40:04 [12862] connect from joomla-site-yuanxi (.....................)                 2018/11/04 02:40:04 [12862] rsync to jgxy_web/db/ from [email protected] (.............)                 2018/11/04 02:40:04 [12862] receiving file list                 2018/11/04 02:40:05 [12862] sent 14463 bytes  received 3321543 bytes  total size 5763078                 2018/11/04 02:40:06 [12864] connect from joomla-site-yuanxi (..........1)                 2018/11/04 02:40:06 [12864] rsync to wfxy_web/db/ from [email protected] (........)                 2018/11/04 02:40:06 [12864] receiving file list                 2018/11/04 02:40:06 [12864] sent 10185 bytes  received 2895278 bytes  total size 2939121        

結果:

      

import csv
import re
import sys
import os

ZABBIX_CONFIG = 'zabbix_sender -c /etc/zabbix/zabbix_agentd.conf -k'

#全域性變數宣告:主要是正則表示式匹配格式:
# pattern1的作用是匹配網站名稱,如果害怕此處不精確可以\w*_web/進行定位
pattern1 = re.compile(r' \w*/')
# pattern2的作用是將網站名稱進行優化去除空格和'/'
pattern2 = re.compile(r'\w+')
# 找出執行緒名稱
pattern3 = re.compile(r'\[\d+\]')
#得到傳輸數字,去除字母與空格,考慮輸入值五匹配情況用-1表示
pattern4 = re.compile(r'\d+|-\d+')


# 宣告兩個list分別放web db 三種資料
quantity_web = []
quantity_db = []
webAndDb = []


# 用csv讀檔案
def readCsv():
    csvFile = open('rsyncd.log', 'r')
    csvData = csv.reader(csvFile)
    return csvData

# 找到所需資料的執行緒名稱,放在list中
def findThread(input_value):
    data = readCsv()
    thread = []
    for item in data:
        item1 = ''.join(item)
        result = pattern1.findall(item1)
        web_name = pattern2.findall(''.join(result))
        if result and ''.join(web_name) == input_value:
            # print(item1)
            thread.append(pattern3.findall(item1))
    return thread

# 匹配執行緒,根據條件將相同的執行緒log句子放在一起(thread[1]代表web,thread[2]代表db)
# 分別放至全域性變數quantity_web,quantity_db中
def dataQuantity(thread):
    data = readCsv()
    num = 0
    if len(thread):
        for item in data:
            item1 = ''.join(item)
            if thread[0] == pattern3.findall(item1):
                continue
            # 若擔心隱患,則用-1,-2表示
            elif thread[1] == pattern3.findall(item1):
                quantity_web.append(item)
            elif thread[2] == pattern3.findall(item1):
                quantity_db.append(item)
    return 1

# name[-1]指的是,每個執行緒最後一句對應的是傳輸資料情況
def getResult(name):
    if name:
        data_SR = name[-1]
        # print((data_SR))
        result = re.findall("[a-z]+ \d+",''.join(data_SR))
    else:
        result = ['-1','-1','-1']
    return result

# 顯示list,如需要返回值,則直接return
def show(list):
    for item in list:
        print(item)

# 拼接基礎字串格式:send -o "5689"  web拼接
def joinStr1(result):
    # 需要字串的共同串,定義後進行拼接
    str = ZABBIX_CONFIG + ' rsync_statu.web.'
    str3 = ['send.','receive.','total.']
    #拼接
    for index in range(len(result)):
        webAndDb.append(str + str3[index]  + '[' + input_value + '] -o '+ '"' + result[index] + '"')

# 拼接基礎字串格式: send -o "5689"  db拼接
def joinStr2(result):
    # 需要字串的共同串,定義後進行拼接
    str = ZABBIX_CONFIG + ' rsync_statu.db.'
    str3 = ['send.', 'receive.', 'total.']
    #拼接
    for index in range(len(result)):
        webAndDb.append(str + str3[index] + '[' + input_value + '] -o ' + '"' + result[index] + '"')


if __name__ == '__main__':

    # 命令列引數
    # input_value = input("輸入網站名稱(如:clxy_web):")
    if len(sys.argv)>=2:
        input_value = str(sys.argv[1])
    else:
        exit(0)

    dataQuantity(findThread(input_value))

    web_result = pattern4.findall(''.join(getResult(quantity_web)))
    db_result = pattern4.findall(''.join(getResult(quantity_db)))

    joinStr1(web_result)
    joinStr2(db_result)

    show(webAndDb)

    for item in webAndDb:
        os.system(item)