ELK分析MySQL慢查詢日誌並生成影象
阿新 • • 發佈:2018-11-09
一、背景
1.MySQL慢查詢日誌格式:
# Time: 181109 15:04:08
# [email protected]: tvpayrcdev[tvpayrcdev] @ [172.16.14.51] Id: 8960747
# Query_time: 35.918265 Lock_time: 0.000141 Rows_sent: 1 Rows_examined: 11699162
SET timestamp=1541747048;
select count(*) from trade_risk_control_record
2.MySQL慢查詢日誌已通過rsyslog實時傳輸到logstash作為Indexer的節點。
二、logstash配置檔案
input部分
input { file { type => "logstash-rc-mysql-slow" path => "/opt/data/logs/localhost-172.16.14.35/db1-slow.log" codec => multiline { pattern => "^# Time:" negate => true what => "previous" } stat_interval => 1 discover_interval => 1 start_position=>"end" sincedb_path => "/dev/null" } }
filter部分
if [type] == "logstash-rc-mysql-slow" { grok { patterns_dir => ["/usr/local/logstash/etc/conf.d/patterns/mysql"] match => { "message" => "%{LONGQUERYLOG}" } } date { match => ["timestamp","UNIX"] } mutate { convert => [ "query_time", "float" ] convert => [ " lock_time", "float" ] remove_field => "message" remove_field => "timestamp" } }
output部分
if [type] == "logstash-rc-mysql-slow" {
elasticsearch {
hosts => ["172.16.1.25","172.16.1.26","172.16.1.27"]
index => 'logstash-mysql_slow_log-%{+YYYY-MM-dd}'
codec=>plain{charset=>"UTF-8"}
}
}
三、kibana展示
1.建立索引
2.發現數據
包括欄位:
3.繪製visualize
例1:統計數量排名前10的sql語句及對應的查詢時間