Rsyslog+kafka+ELK(叢集) 配置xpack外掛
阿新 • • 發佈：2022-04-15
由於公司Policy要求，需要對ELK的kibana訪問進行安全認證
ELK7.*已經支援xpack的整合安裝，直接配置使用即可
本篇基於上篇部落格基礎上進行配置
1、生成證書
~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 430a218a4513 elasticsearch:7.7.0 "/tini -- /usr/loc..." 3 hours ago Up 3 hours elasticsearch 1a75637e85d6 kafka:2.12-2.5.0 "start-kafka.sh" 3 hours ago Up 3 hours kafka 03a9aa4a458c zookeeper:3.4.13 "/docker-entrypoin..." 3 hours ago Up 3 hours zookeeper 81ba9a81983c logstash:7.7.0 "/usr/local/bin/do..." 27 hours ago Up 8 minutes 5044/tcp, 0.0.0.0:4560->4560/tcp, 9600/tcp logstash 4bc2de14b119 kibana:7.7.0 "/usr/local/bin/du..." 2 days ago Up 7 minutes 0.0.0.0:5601->5601/tcp kibana ~]# docker stop kibana ~]# docker stop logstash #三臺節點都需要關閉 ~]# docker exec -it elasticsearch bash elasticsearch]# elasticsearch-certutil ca # 生成CA證書 ENTER ENTER elasticsearch]# elasticsearch-certutil cert --ca elastic-stack-ca.p12 # 生成節點證書 ENTER ENTER #在當前目錄生成相應的證書 #把證書傳輸到其他節點,叢集間使用證書進行通訊，由於之前掛載了/usr/share/elasticsearch/data目錄，先把證書放在data目錄下傳輸到宿主機 elasticsearch]# mv elastic* data/ elasticsearch]# exit ~]# cd /data/es/data ~]# mkdir /data/es/certs ~]# mv /data/es/data/elastic* /data/es/certs/
把證書檔案傳輸給其他2臺節點
~]# scp -rp /data/es/certs/elastic* [email protected]:/data/es/certs/
~]# scp -rp /data/es/certs/elastic* [email protected]:/data/es/certs/
3、修改內建使用者密碼
(1) 三臺節點修改docker-compose配置
~]# vi /data/elk/docker-compose.yml #掛載證書對應檔案 version: '2.1' services: elasticsearch: image: elasticsearch:7.7.0 container_name: elasticsearch depends_on: - kafka environment: ES_JAVA_OPTS: -Xms1g -Xmx1g network_mode: host volumes: - /data/es/conf/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml - /data/es/data:/usr/share/elasticsearch/data - /data/es/certs/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12 - /data/es/certs/elastic-stack-ca.p12:/usr/share/elasticsearch/config/elastic-stack-ca.p12 ......
(2) 三臺臺節點修改elasticsearch配置檔案
~]# vi /data/es/conf/elasticsearch.yml #新增認證選項 cluster.name: es-cluster network.host: 0.0.0.0 node.name: master1 http.cors.enabled: true http.cors.allow-origin: "*" node.master: true node.data: true network.publish_host: 10.10.27.125 discovery.zen.minimum_master_nodes: 1 discovery.zen.ping.unicast.hosts: ["10.10.27.125","10.10.27.126","10.10.27.127"] cluster.initial_master_nodes: ["10.10.27.125","10.10.27.126","10.10.27.127"] xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 #預設的elastic-certificates.p12檔案位置在容器例項的/usr/share/elasticsearch/config/路徑下，也可以直接指定具體路徑
(2) 啟動三臺節點的elasticsearch例項
master1 ~]# docker stop elasticsearch && docker rm elasticsearch
master1 ~]# cd /data/elk/ && docker-compose up -d elasticsearch
master2 ~]# docker stop elasticsearch && docker rm elasticsearch
master2 ~]# cd /data/elk/ && docker-compose up -d elasticsearch
master3 ~]# docker stop elasticsearch && docker rm elasticsearch
master3 ~]# cd /data/elk/ && docker-compose up -d elasticsearch
(3) 配置內建使用者密碼
master1 ~]# docker exec -it elasticsearch bash
elasticsearch]# elasticsearch-setup-passwords interactive
# 輸出結果
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y # 輸入y
# 直接輸入密碼，然後再重複一遍密碼，中括號裡是賬號
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
# 為了方便後續使用，這裡可以都設定成一樣
# password: Password123
elasticsearch]# exit
# 驗證叢集設定的賬號和密碼
# 開啟瀏覽器訪問這個地址，出現需要輸入賬號密碼的介面證明設定成功
#http://10.10.27.125:9200/_security/_authenticate?pretty
3、修改配置檔案
(1) 三臺節點修改logstash配置檔案
~]# vi /data/logstash/conf/logstash.conf
......
output{
elasticsearch{
hosts => ["10.10.27.125:9200","10.10.27.126:9200","10.10.27.127:9200"]
index => "system-log-%{+YYYY.MM.dd}"
user => "elastic" # 注意：這裡演示使用超級賬號，安全起見最好是使用自定義的賬號，並授予該使用者建立索引的許可權，具體看下方地址
password => "Password123"
}
stdout{
codec => rubydebug
}
}
# 使用自定義的賬號官方地址：https://www.elastic.co/cn/blog/configuring-ssl-tls-and-https-to-secure-elasticsearch-kibana-beats-and-logstash
~]# vi /data/logstash/conf/logstash.yml
http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://10.10.27.125:9200","http://10.10.27.126:9200","http://10.10.27.127:9200" ]
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: "Password123"
(2) 修改kibana配置檔案
~]# vi /data/kibana/conf/kibana.yml #新增kibana的使用者密碼
server.name: kibana
server.host: "0.0.0.0"
elasticsearch.hosts: [ "http://10.10.27.125:9200","http://10.10.27.126:9200","http://10.10.27.127:9200" ]
monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.username: "kibana" # 注意：此處不用超級賬號elastic，而是使用kibana跟es連線的賬號kibana
elasticsearch.password: "Password123"
4、啟動ELK
#重啟logstash
master1 ~]# docker restart logstash
master2 ~]# docker restart logstash
master3 ~]# docker restart logstash
#重啟kibana
master1 ~]# docker restart kibana
5、訪問kibana
訪問kibana：http://10.10.27.125:5601 使用使用者elastic訪問成功，在Management下面的Kibana最後出現一個Security，有User和Role，方便kibana多使用者建立及角色許可權控制