|NO.Z.00081|——————————|^^ 部署 ^^|——|Linux&ELK日誌分析&.V02|--------------------------------------------|JDK|ELK|Logstash|

一、實驗部署

### --- 實驗架構

~~~		本次部署的是filebeats（客戶端），logstash+elasticsearch+kibana（服務端）組成的架構
~~~		業務請求到達nginx-server機器上的nginx；nginx的響應請求，
~~~		並在access.log檔案中增加訪問記錄，FileBeat蒐集新增的日誌，

~~~		#通過LogStash的5044埠上傳日誌：
~~~		LogStash將日誌資訊通過本機的9200埠傳入到ElasticSerach；
~~~		蒐集日誌的使用者通過瀏覽器訪問Kibana，伺服器埠是5601；
~~~		Kibana通過9200埠訪問ElasticSerach；

一、實驗環境

### --- 本次部署的單點ELK用了兩臺機器（Centos-7.5）

~~~     ELK服務端： centos7.x       10.10.10.11 ELK-server
~~~     Nginx客戶端：centos7.x      10.10.10.12 Nginx-server

### --- 配置好網路yum源

[root@server11 yum.repos.d]# wget http://mirrors.aliyun.com/repo/Centos-7.repo
[root@server11 yum.repos.d]# wget http://mirrors.aliyun.com/repo/epel-7.repo

### --- 關閉防火牆：systemctl stop(disable)firewalld
[root@server11 ~]# systemctl status firewalld.service
 
### --- 關閉selinux：SELINUX=disabled
[root@server11 ~]# getenforce
Disabled

二、下載並安裝軟體包

### --- 下載並安裝軟體包

[root@server11 ~]# mkdir /elk
[root@server11 ~]# cd /elk
[root@server11 elk]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.tar.gz
[root@server11 elk]# wget https://artifacts.elastic.co^Cownloads/logstash/logstash-6.2.3.tar.gz
[root@server11 elk]# wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.3-linux-x86_64.tar.gz

### --- 全部解壓縮，並複製到/usr/local/目錄下

[root@server11 elk]# tar -xf elasticsearch-6.2.3.tar.gz 
[root@server11 elk]# tar -xf kibana-6.2.3-linux-x86_64.tar.gz
[root@server11 elk]# tar -xf logstash-6.2.3.tar.gz 
[root@server11 elk]# cp -a elasticsearch-6.2.3 kibana-6.2.3-linux-x86_64 logstash-6.2.3 /usr/local/

三、安裝JDK（java）環境工具

### --- 安裝JDK（java）環境工具

[root@server11 ~]# yum install -y java-1.8*

四、配置elasticsearch：

### --- 配置elasticsearch：
~~~     新建elasticsearch使用者並啟動（elasticsearch普通使用者啟動）

[root@server11 ~]# useradd  elasticsearch
[root@server11 ~]# chown -R elasticsearch.elasticsearch /usr/local/elasticsearch-6.2.3/     #將這個使用者的所有者改為elasticsearch目錄
[root@server11 ~]# su - elasticsearch                               // 以這個使用者的身份進行啟動部署
[elasticsearch@server11 ~]$ cd /usr/local/elasticsearch-6.2.3
[elasticsearch@server11 elasticsearch-6.2.3]$ ./bin/elasticsearch -d

### --- 檢視程序是否啟動成功（等待一下，不是立刻啟動）

[elasticsearch@server11 elasticsearch-6.2.3]$ netstat -antp             
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      12133/java

### --- 若出現錯誤可以檢視日誌

[elasticsearch@server11 elasticsearch-6.2.3]$ cat /usr/local/elasticsearch-6.2.3/logs/elasticsearch.log

### --- 測試是否可以正常訪問

[elasticsearch@server11 elasticsearch-6.2.3]$ curl localhost:9200   // 只要看到這個介面說明部署成功
{
  "name" : "XRgtD77",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "jPl_H0_FRIWo0CY-ureAuA",
  "version" : {
    "number" : "6.2.3",
    "build_hash" : "c59ff00",
    "build_date" : "2018-03-13T10:06:29.741383Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

五、配置logstash

### --- 配置logstash
~~~     Logstash收集nginx日誌之使用grok過濾外掛解析日誌，grok作為一個logstash的過濾外掛，
~~~     支援根據模式解析文字日誌行，拆成欄位
### --- logstash中grok的正則匹配

[elasticsearch@server11 elasticsearch-6.2.3]$ exit
logout
[root@server11 ~]# cd /usr/local/logstash-6.2.3/
[root@server11 logstash-6.2.3]# vim vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/grok-patterns atterns
#Nginx log
WZ ([^ ]*)
NGINXACCESS %{IP:remote_ip} \- \- \[%{HTTPDATE:timestamp}\] "%{WORD:method} %{WZ:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:status} %{NUMBER:bytes} %{QS:referer} %{QS:agent} %{QS:xforward}

### --- 建立logstash配置檔案

[root@server11 logstash-6.2.3]# vim /usr/local/logstash-6.2.3/default.conf
input {
    beats {
        port=> "5044"
    }
}
#資料過濾
filter {
    grok {
        match => { "message" => "%/{NGINXACCESS}" }
    }
    geoip {
#nginx客戶端IP
        source => "10.10.10.12"
    }
}
#輸出配置為本機的9200埠，這是ElasticSerach服務的監聽埠
output{
    elasticsearch {
        hosts => ["127.0.0.1:9200"]
    }
}

### --- 進入到/usr/local/logstash-6.2.3目錄下，並執行下列命令
~~~     後臺啟動logstash：nohup bin/logstash -f default.conf &
~~~     檢視啟動日誌：tailf nohup.out
~~~     檢視埠是否啟動：netstat -napt|grep 5044

[root@server11 logstash-6.2.3]# nohup bin/logstash -f default.conf &
[1] 12384
[root@server11 logstash-6.2.3]# nohup: ignoring input and appending output to ‘nohup.out’       #自動執行
[root@server11 logstash-6.2.3]# tailf nohup.out
[2021-02-15T23:27:50,599][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2021-02-15T23:27:50,699][INFO ][logstash.pipeline        ] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x35e626f run>"}
[2021-02-15T23:27:50,864][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2021-02-15T23:27:51,054][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[root@server11 logstash-6.2.3]# netstat -napt|grep 5044
tcp6       0      0 :::5044                 :::*                    LISTEN      12384/java

===============================END===============================

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart ——W.S.Landor

來自為知筆記(Wiz)

|NO.Z.00081|——————————|^^ 部署 ^^|——|Linux&ELK日誌分析&.V02|--------------------------------------------|JDK|ELK|Logstash|

相關推薦