示例

日誌格式

第一種日誌格式【INFO前面是空格】
    INFO 2020-08-05 10:01:39,060 1 --- [cache-pool-13] c.w.c.w.u.RequestLoggerUtils RequestLoggerUtils.java:96 - <log> - {"appName":"test info","data":{"result":{"flag":-2,"id":"255","potentialFlag":0,"school":0,"status":0,"username":"1234890632144319874"},"status":200}}

第二種日誌格式
ERROR 
 
2020-08-05 11:05:27,631 1 --- [com.alibaba.nacos.client.Worker.longPollingfixed-10.0.0.189_8848] c.a.n.c.c.h.ServerHttpAgent ServerHttpAgent.java:89 - [NACOS ConnectException] currentServerAddr:10.0.0.189:8848

logstash 配置檔案 [通過正則匹配兩種日誌]

[root@ope-elk ~]# cat /home/wx/logstash-6.2.4/config/beats.conf 
input {
  beats {
    port 
 
=> 5044
  }
}

filter {
        grok{
             match => [
                        "message" , "(^[ ](?<Level>[A-Z]{0,})\s(?<Date>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{0,})\s\d{0,}\s.*).*",
                        "message" , "(?<Level>^[A-Z]{0,})\s(?<Date>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{0,})\s\d{0,}\s.*
 
"
                      ]             
             overwrite =>["message"]
        }
        date {
        match => [ "Date", "yyyy-MM-dd HH:mm:ss,SSS" ]
        target => [ "@timestamp" ]
    }
}

output{
  #if [fields][service] == "es-test"{
  #  輸出到桌面
  #  stdout {
  #    codec => rubydebug
  #  }
  #  輸出到elasticsearch中
  #  elasticsearch {
  #      hosts => ["192.168.56.30:9200"]
  #      index => "test-%{+YYYY.MM.dd}"
  #  }}
  if [fields][service] == "es-test"{
    elasticsearch {
        hosts => ["192.168.56.30:9200"]
        index => "es-test-%{+YYYY.MM.dd}"
    }}
}