四、部署filebeat

我這裡用的是filebeat-7.3.2

1、下載rpm包

wget https://mirrors.huaweicloud.com/filebeat/7.3.2/filebeat-7.3.2-x86_64.rpm
yum -y install filebeat-7.3.2-x86_64.rpm

2、修改配置檔案

我這裡採集的是nginx的日誌，都在/data0/logs下，以xxx.access.log命名的日誌檔案

# cat /etc/filebeat/filebeat.yml  | grep -v -E "^#|^$|^  #"
filebeat.inputs:
- type: log
  paths:
    
 
- /data0/logs/*.access.log # /data0/logs/<projectname>.log   這裡寫自己的日誌目錄
  close_inactive: 24h
  exclude_files: ['/data0/logs/collection.access.log']    #排除檔案，也就是不採集的日誌檔案
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 3
setup.kibana:
output.kafka:
  hosts: ["192.168.1.10:9092", "192.168.1.11:9092", "192.168.1.12:9092"]
  topic: 'nginx-access-log'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
 

3、啟動服務並開機自啟

systemctl daemon-reload
systemctl enable filebeat
systemctl start filebeat

4、驗證kafka的topic有沒有訊息

在kafka伺服器上執行

export JMX_PORT=10058 && /usr/local/kafka_2.12-2.3.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic nginx-access-log

有資訊輸出則filebeat已經成功將日誌推送到kafka

filebeat部署到此就結束了。