elastersearch7.6.1搭建及基本操作詳解
阿新 • • 發佈：2021-10-08
vim /etc/security/limits.conf
* soft nofile 655350 * hard nofile 655350 * soft nofile 655350 * hard nofile 655350 es soft memlock unlimited es hard memlock unlimited
vim /etc/sysctl.conf
vm.max_map_count=262144 fs.file-max=655360 vm.swappiness=1
儲存後執行 sysctl -p
編輯配置檔案
vim elastersearch.yaml
cluster.name: elastersearch node.name: cs-es1 path.data: /home/es/data path.logs: /home/es/logs # 是否鎖住記憶體，避免交換記憶體帶來效能損失，預設為false bootstrap.memory_lock: true # 本機ip network.host: 192.168.1.2 http.port: 9200 transport.tcp.port: 9300 # 節點角色，預設data master client 都有,如果分開，則改為 false node.data: true node.master: true # 所有節點ip discovery.seed_hosts: ["192.168.1.2","192.168.1.3","192.168.1.4"] # 初始化 master 節點 cluster.initial_master_nodes: ["192.168.1.2","192.168.1.3","192.168.1.4"] discovery.zen.ping_timeout: 60s discovery.zen.join_timeout: 60s # 忽略不符合master的節點 discovery.zen.master_election.ignore_non_master_pings: true # 選舉最多同意數 discovery.zen.mininum_master_nodes:1 gateway.recover_after_nodes: 1 gateway.recover_after_time: 5m gateway.expected_nodes: 1 # xpack,如果不需要開啟，註釋掉 xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
此配置檔案為模板，如需多臺伺服器組叢集，則要將本模板的node.name、network.host欄位更改為實際的名稱和地址，discovery.seed_hosts和cluster.initial_master_nodes也要做相應的更改
若開啟 xpath ,需要建立證書
執行命令生成證書
bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""
需要將此證書放在每臺節點的 config/ 目錄下
然後重啟es，然後設定密碼
需要為4個使用者分別設定密碼：elastic、kibana、logstash_system、beats_system
1.可以自動設定密碼
bin/elasticsearch-setup-passwords auto
2.也可以手動設定密碼
bin/elasticsearch-setup-passwords interactive
訪問時加上密碼訪問即可
curl -u elastic:baidu@123 '192.168.1.2/_cat/nodes'
如果kibina等需要連結ES，需要在連線的配置檔案中指定密碼
加入systemd管理
vim /usr/lib/systemd/system/elasticsearch.service
[Unit] Description=es user elasticsearch.service After=network.target [Service] Type=simple User=es Group=es ExecStart=/home/es/elasticsearch7/bin/elasticsearch ExecStop=/bin/kill -9 $MAINPID Restart=no LimitMEMLOCK=infinity LimitNOFILE=1048576 LimitNPROC=1048576 PrivateTmp=true [Install] WantedBy=multi-user.target
其中引數不太理解可以參考此博文：http://www.jinbuguo.com/systemd/systemd.service.html
重新載入systemd
systemctl daemon-reload
systemctl start elasticsearch.service
基本操作
1、檢視叢集健康狀態curl '10.231.13.177:9200/_cluster/health?pretty' { "cluster_name" : "elasticsearch", "status" : "green", //叢集的狀態紅綠燈，綠：健康，黃：亞健康，紅：病態 "timed_out" : false, "number_of_nodes" : 4, //節點數 "number_of_data_nodes" : 1, //資料節點數 "active_primary_shards" : 0, //分片數，0個Index庫 "active_shards" : 0, //正常的分片數 "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 } ## 檢視指定索引庫的健康狀態 http://localhost:9200/_cluster/health/index_name?pretty2、查詢叢集所有節點
curl '10.231.13.177:9200/_cat/nodes?v' ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name 10.231.66.181 18 96 14 3.12 4.11 4.44 di - ngd-es5-elasticsearch-data-0 10.231.66.172 26 96 14 3.12 4.11 4.44 - - ngd-es5-elasticsearch-client-7795566789-6qc5x 10.231.66.183 40 96 14 3.12 4.11 4.44 m - ngd-es5-elasticsearch-master-0 10.231.64.85 35 99 2 0.37 0.63 0.76 m * ngd-es5-elasticsearch-master-1 heap.percent：堆記憶體佔的記憶體百分比 ram.percent：實體記憶體佔用百分比 cpu：表示使用的cpu核心 load_1m load_5m load_15m：1分鐘 5分鐘 15分鐘 佔用系統cpu百分比 node.role：表示節點能充當的角色主(m)、資料(di) 、client(-)節點，節點未區分開為dilm master：表示當前是否為主節點，*表示當前為主 ## 檢視MASTER節點 curl '10.233.0.93:9200/_cat/master?v' id host ip node 2oaDofmzQ6KQK9BlorvO0Q 10.233.110.41 10.233.110.41 ngd-es5-elasticsearch-master-13、列出所有索引
curl '10.231.13.177:9200/_cat/indices?v' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open vfantest acyI9dArRAqTOHNAwGO_oQ 5 1 0 0 650b 650b
當前只有一個索引
4、檢視各個節點的總分片數及分佈情況，及磁碟使用率
curl '10.233.8.243:9200/_cat/allocation?v' shards disk.indices disk.used disk.avail disk.total disk.percent host ip node 28 84.8kb 284.4gb 57.8gb 342.2gb 83 10.233.64.133 10.233.64.133 ngd-es7-master-1 29 68.2kb 140.3gb 201.9gb 342.2gb 40 10.233.66.72 10.233.66.72 ngd-es7-master-2 29 106.4kb 91.4gb 250.8gb 342.2gb 26 10.233.65.219 10.233.65.219 ngd-es7-master-05、檢視叢集分片的詳細分佈，分片的大小等
curl '10.233.8.243:9200/_cat/shards?v' index shard prirep state docs store ip node dialog-statistics-scene-202106 1 p STARTED 0 283b 10.233.64.133 ngd-es7-master-1 dialog-statistics-scene-202106 1 r STARTED 0 283b 10.233.65.219 ngd-es7-master-0 dialog-statistics-scene-202106 2 p STARTED 0 283b 10.233.66.72 ngd-es7-master-2 dialog-statistics-scene-202106 2 r STARTED 0 283b 10.233.65.219 ngd-es7-master-0 dialog-statistics-scene-202106 0 r STARTED 0 283b 10.233.64.133 ngd-es7-master-1 dialog-statistics-scene-202106 0 p STARTED 0 283b 10.233.66.72 ngd-es7-master-2 search-suggest-online-data 1 r STARTED 0 283b 10.233.64.133 ngd-es7-master-1 search-suggest-online-data 1 p STARTED 0 283b 10.233.65.219 ngd-es7-master-0 search-suggest-online-data 2 p STARTED 0 283b 10.233.66.72 ngd-es7-master-2 search-suggest-online-data 2 r STARTED 0 283b 10.233.65.219 ngd-es7-master-0 search-suggest-online-data 0 r STARTED 0 283b 10.233.64.133 ngd-es7-master-1 search-suggest-online-data 0 p STARTED 0 283b 10.233.66.72 ngd-es7-master-26、檢視 segments 分佈資訊
curl '10.233.8.243:9200/_cat/segments?v' index shard prirep ip segment generation docs.count docs.deleted size size.memory committed searchable version compound data-record-index-202106 0 p 10.233.64.133 _0 0 9 0 19.4kb 6477 true true 8.4.0 true data-record-index-202106 0 r 10.233.66.72 _0 0 9 0 19.4kb 6477 true true 8.4.0 true data-record-index-202106 1 p 10.233.64.133 _0 0 7 0 19.4kb 6461 true true 8.4.0 true data-record-index-202106 1 p 10.233.64.133 _1 1 9 0 38.1kb 6621 true true 8.4.0 true data-record-index-202106 1 r 10.233.65.219 _0 0 7 0 19.4kb 6461 true true 8.4.0 true data-record-index-202106 1 r 10.233.65.219 _1 1 9 0 38.1kb 6621 true true 8.4.0 true data-record-index-202106 2 p 10.233.66.72 _0 0 4 0 16.2kb 6437 true true 8.4.0 true data-record-index-202106 2 p 10.233.66.72 _1 1 5 0 24.4kb 6541 true true 8.4.0 true data-record-index-202106 2 r 10.233.65.219 _0 0 4 0 16.2kb 6437 true true 8.4.0 true data-record-index-202106 2 r 10.233.65.219 _1 1 5 0 24.4kb 6541 true true 8.4.0 true7、建立一個索引：
## 可以在建立中指定settings 和 mappings，加-d '{}' 引數指定即可 curl -XPUT '10.231.13.177:9200/20210122?pretty' -H 'content-type: application/json' -d '{ "acknowledged" : true, "shards_acknowledged" : true }' settings格式： "settings": { "number_of_shards": 5, "number_of_replicas": 1 } mappings格式： "mappings": { "sms_up": { "dynamic_templates": [ { "strings_as_keywords": { "match_mapping_type": "string", "mapping": { "type": "keyword" } } } ], ## 再次檢視索引 curl '10.231.13.177:9200/_cat/indices?v' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open vfantest acyI9dArRAqTOHNAwGO_oQ 5 1 0 0 650b 650b yellow open 20210122 urFLBfkQRGmIQVcXqITG5w 5 1 0 0 260b 260b
看到名稱為"20210122"的索引已經建立，它有5個私有的分片以及一個副本
8、插入和獲取
現在來插入一些資料到叢集索引，插入時必須要指定資料型別，例如 external 型別 ID為 1 的資料
curl -XPUT 'localhost:9200/20210122/external/1?pretty' -d ' > { > "name": "Tony", > "sex": "boy" > }' 回顯： { "_index" : "20210122", "_type" : "external", "_id" : "1", "_version" : 1, "result" : "created", "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "created" : true }
返回為""created" : true" 代表建立成功獲取 20210122 索引下型別為 external 、ID =1 的資料，pretty引數表示返回結果格式美觀。
curl 'localhost:9200/20210122/external/1?pretty' { "_index" : "20210122", "_type" : "external", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "name" : "Tony", "sex" : "boy" } }9、刪除索引
curl -XDELETE 'localhost:9200/20210122' {"acknowledged":true}10、檢視ES有哪些外掛
curl '10.233.0.93:9200/_cat/plugins?v' name component version ngd-es5-elasticsearch-master-2 analysis-ik 5.4.2 ngd-es5-elasticsearch-master-2 discovery-kubernetes 5.4.2 ngd-es5-elasticsearch-master-2 elasticsearch-analysis-ansj 5.4.2.0 ngd-es5-elasticsearch-master-2 ingest-geoip 5.4.2 ngd-es5-elasticsearch-master-2 ingest-user-agent 5.4.2 ngd-es5-elasticsearch-master-2 x-pack 5.4.2 ... ngd-es5-elasticsearch-client-7795566789-6w2ns x-pack 5.4.211、檢視當前ES節點資訊
curl '10.233.0.93:9200' { "name" : "ngd-es5-elasticsearch-client-7795566789-6w2ns", "cluster_name" : "elasticsearch", "cluster_uuid" : "KJAcgz-xQ1awOxG4T-WOlA", "version" : { "number" : "5.4.2", "build_hash" : "929b078", "build_date" : "2017-06-15T02:29:28.122Z", "build_snapshot" : false, "lucene_version" : "6.5.1" }, "tagline" : "You Know, for Search" }
You Know，for Search 表示節點正常