Elasticsearch（一）入門
阿新 • • 發佈：2022-12-06
第一章、Elasticsearch概述
1.1 開篇
結構化資料
結構化資料
半結構化資料
1.2 技術選型
Elasticsearch 是什麼
The Elastic Stack, 包括 Elasticsearch、 Kibana、 Beats 和 Logstash（也稱為 ELK Stack）。能夠安全可靠地獲取任何來源、任何格式的資料，然後實時地對資料進行搜尋、分析和視覺化。
Elaticsearch，簡稱為 ES， ES 是一個開源的高擴充套件的分散式全文搜尋引擎， 是整個 ElasticStack 技術棧的核心。
它可以近乎實時的儲存、檢索資料；本身擴充套件性很好，可以擴充套件到上百臺伺服器，處理 PB 級別的資料。
elastic
英 [ɪˈlæstɪk] 美 [ɪˈlæstɪk]
n. 橡皮圈(或帶);鬆緊帶
adj. 橡皮圈(或帶)的;有彈性的;有彈力的;靈活的;可改變的;可伸縮的
全文搜尋引擎
Google，百度類的網站搜尋，它們都是根據網頁中的關鍵字生成索引，我們在搜尋的時候輸入關鍵字，它們會將該關鍵字即索引匹配到的所有網頁返回；還有常見的專案中應用日誌的搜尋等等。對於這些非結構化的資料文字，關係型資料庫搜尋不是能很好的支援。
一般傳統資料庫，全文檢索都實現的很雞肋，因為一般也沒人用資料庫存文字欄位。進行全文檢索需要掃描整個表，如果資料量大的話即使對 SQL 的語法優化，也收效甚微。建立了索引，但是維護起來也很麻煩，對於 insert 和 update 操作都會重新構建索引。
基於以上原因可以分析得出，在一些生產環境中，使用常規的搜尋方式，效能是非常差的：
- 搜尋的資料物件是大量的非結構化的文字資料。
- 檔案記錄量達到數十萬或數百萬個甚至更多。
- 支援大量基於互動式文字的查詢。
- 需求非常靈活的全文搜尋查詢。
- 對高度相關的搜尋結果的有特殊需求，但是沒有可用的關係資料庫可以滿足。
- 對不同記錄型別、非文字資料操作或安全事務處理的需求相對較少的情況。為了解決結構化資料搜尋和非結構化資料搜尋效能問題，我們就需要專業，健壯，強大的全文搜尋引擎 。
裡說到的全文搜尋引擎指的是目前廣泛應用的主流搜尋引擎。它的工作原理是計算機索引程式通過掃描文章中的每一個詞，對每一個詞建立一個索引，指明該詞在文章中出現的次數和位置，當用戶查詢時，檢索程式就根據事先建立的索引進行查詢，並將查詢的結果反饋給使用者的檢索方式。這個過程類似於通過字典中的檢索字表查字的過程。
Elasticsearch 應用案例
- GitHub: 2013 年初，拋棄了 Solr，採取 Elasticsearch 來做 PB 級的搜尋。 “GitHub 使用Elasticsearch 搜尋 20TB 的資料，包括 13 億檔案和 1300 億行程式碼”。
- 維基百科：啟動以 Elasticsearch 為基礎的核心搜尋架構
- 百度：目前廣泛使用 Elasticsearch 作為文字資料分析，採集百度所有伺服器上的各類指標資料及使用者自定義資料，通過對各種資料進行多維分析展示，輔助定位分析例項異常或業務層面異常。目前覆蓋百度內部 20 多個業務線（包括雲分析、網盟、預測、文庫、直達號、錢包、 風控等），單叢集最大 100 臺機器， 200 個 ES 節點，每天匯入 30TB+資料。
- 新浪：使用 Elas ticsearch 分析處理 32 億條實時日誌。
- 阿里：使用 Elasticsearch 構建日誌採集和分析體系。
- Stack Overflow：解決 Bug 問題的網站，全英文，程式設計人員交流的網站。
第二章、Elasticsearch入門
2.1 環境準備
官方網址：https://www.elastic.co/cn/
官方文件：https://www.elastic.co/guide/index.html
Elasticsearch 7.8.0下載頁面:https://www.elastic.co/cn/downloads/past-releases/elasticsearch-7-8-0
Windows 版的 Elasticsearch 壓縮包，解壓即安裝完畢，解壓後的 Elasticsearch 的目錄結構如下 ：
| 目錄 | 含義 |
|---|---|
| bin | 可執行指令碼目錄 |
| config | 配置目錄 |
| jdk | 內建 JDK 目錄 |
| lib | 類庫 |
| logs | 日誌目錄 |
| modules | 模組目錄 |
| plugins | 外掛目錄 |
解壓後，進入 bin 檔案目錄，點選 elasticsearch.bat 檔案啟動 ES 服務 。
注意： 9300 埠為 Elasticsearch 叢集間元件的通訊埠， 9200 埠為瀏覽器訪問的 http協議 RESTful 埠。
開啟瀏覽器，輸入地址： http://localhost:9200，測試返回結果，返回結果如下：
{
"name" : "DESKTOP-LNJQ0VF",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "nCZqBhfdT1-pw8Yas4QU9w",
"version" : {
"number" : "7.8.0",
"build_flavor" : "default",
"build_type" : "zip",
"build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
"build_date" : "2020-06-14T19:35:50.234439Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
2.2 RESTful & JSON
REST 指的是一組架構約束條件和原則。滿足這些約束條件和原則的應用程式或設計就是 RESTful。 Web 應用程式最重要的 REST 原則是，客戶端和伺服器之間的互動在請求之間是無狀態的。從客戶端到伺服器的每個請求都必須包含理解請求所必需的資訊。如果伺服器在請求之間的任何時間點重啟，客戶端不會得到通知。此外，無狀態請求可以由任何可用伺服器回答，這十分適合雲端計算之類的環境。客戶端可以快取資料以改進效能。
在伺服器端，應用程式狀態和功能可以分為各種資源。資源是一個有趣的概念實體，它向客戶端公開。資源的例子有：應用程式物件、資料庫記錄、演算法等等。每個資源都使用 URI(Universal Resource Identifier) 得到一個唯一的地址。所有資源都共享統一的介面，以便在客戶端和伺服器之間傳輸狀態。使用的是標準的 HTTP 方法，比如 GET、 PUT、 POST 和DELETE。
在 REST 樣式的 Web 服務中，每個資源都有一個地址。資源本身都是方法呼叫的目標，方法列表對所有資源都是一樣的。這些方法都是標準方法，包括 HTTP GET、 POST、PUT、 DELETE，還可能包括 HEAD 和 OPTIONS。簡單的理解就是，如果想要訪問網際網路上的資源，就必須向資源所在的伺服器發出請求，請求體中必須包含資源的網路路徑， 以及對資源進行的操作(增刪改查)。
REST 樣式的 Web 服務若有返回結果，大多數以JSON字串形式返回。
2.3 Postman客戶端工具
如果直接通過瀏覽器向 Elasticsearch 伺服器發請求，那麼需要在傳送的請求中包含
HTTP 標準的方法，而 HTTP 的大部分特性且僅支援 GET 和 POST 方法。所以為了能方便地進行客戶端的訪問，可以使用 Postman 軟體Postman 是一款強大的網頁除錯工具，提供功能強大的 Web API 和 HTTP 請求除錯。
軟體功能強大，介面簡潔明晰、操作方便快捷，設計得很人性化。 Postman 中文版能夠傳送任何型別的 HTTP 請求 (GET, HEAD, POST, PUT…)，不僅能夠表單提交，且可以附帶任意型別請求體。
Postman下載頁面 :https://www.postman.com/downloads/
2.4 倒排索引
正排索引（傳統）
| id | content |
|---|---|
| 1001 | my name is zhang san |
| 1002 | my name is li si |
倒排索引
| keyword | id |
|---|---|
| name | 1001, 1002 |
| zhang | 1001 |
Elasticsearch 是面向文件型資料庫，一條資料在這裡就是一個文件。 為了方便大家理解，我們將 Elasticsearch 裡儲存文件資料和關係型資料庫 MySQL 儲存資料的概念進行一個類比
ES 裡的 Index 可以看做一個庫，而 Types 相當於表， Documents 則相當於表的行。這裡 Types 的概念已經被逐漸弱化， Elasticsearch 6.X 中，一個 index 下已經只能包含一個type， Elasticsearch 7.X 中, Type 的概念已經被刪除了。
2.5 索引
建立索引
對比關係型資料庫，建立索引就等同於建立資料庫。
在 Postman 中，向 ES 伺服器發 PUT 請求 ： http://127.0.0.1:9200/shopping
請求後，伺服器返回響應：
{
"acknowledged": true,//響應結果
"shards_acknowledged": true,//分片結果
"index": "shopping"//索引名稱
}
後臺日誌：
[2022-02-10T17:07:45,921][INFO ][o.e.c.m.MetadataCreateIndexService] [LAPTOP-7SVUMV4L] [shopping] creating index, cause [api], templates [], shards [1]/[1], mappings []
如果重複發 PUT 請求 ： http://127.0.0.1:9200/shopping 新增索引，會返回錯誤資訊 :
{
"error": {
"root_cause": [
{
"type": "resource_already_exists_exception",
"reason": "index [shopping/J0WlEhh4R7aDrfIc3AkwWQ] already exists",
"index_uuid": "J0WlEhh4R7aDrfIc3AkwWQ",
"index": "shopping"
}
],
"type": "resource_already_exists_exception",
"reason": "index [shopping/J0WlEhh4R7aDrfIc3AkwWQ] already exists",
"index_uuid": "J0WlEhh4R7aDrfIc3AkwWQ",
"index": "shopping"
},
"status": 400
}
檢視所有索引
在 Postman 中，向 ES 伺服器發 GET 請求 ： http://127.0.0.1:9200/_cat/indices?v
這裡請求路徑中的_cat 表示檢視的意思， indices 表示索引，所以整體含義就是檢視當前 ES伺服器中的所有索引，就好像 MySQL 中的 show tables 的感覺，伺服器響應結果如下 :
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open shopping abXoyvUySAKaNWNuKKy12g 1 1 2 0 11.7kb 11.7kb
| 表頭 | 含義 |
|---|---|
| health | 當前伺服器健康狀態： green(叢集完整) yellow(單點正常、叢集不完整) red(單點不正常) |
| status | 索引開啟、關閉狀態 |
| index | 索引名 |
| uuid | 索引統一編號 |
| pri | 主分片數量 |
| rep | 副本數量 |
| docs.count | 可用文件數量 |
| docs.deleted | 文件刪除狀態（邏輯刪除） |
| store.size | 主分片和副分片整體佔空間大小 |
| pri.store.size | 主分片佔空間大小 |
檢視單個索引
在 Postman 中，向 ES 伺服器發 GET 請求 ： http://127.0.0.1:9200/shopping
返回結果如下：
{
"shopping": {//索引名
"aliases": {},//別名
"mappings": {},//對映
"settings": {//設定
"index": {//設定 - 索引
"creation_date": "1617861426847",//設定 - 索引 - 建立時間
"number_of_shards": "1",//設定 - 索引 - 主分片數量
"number_of_replicas": "1",//設定 - 索引 - 主分片數量
"uuid": "J0WlEhh4R7aDrfIc3AkwWQ",//設定 - 索引 - 主分片數量
"version": {//設定 - 索引 - 主分片數量
"created": "7080099"
},
"provided_name": "shopping"//設定 - 索引 - 主分片數量
}
}
}
}
刪除索引
在 Postman 中，向 ES 伺服器發 DELETE 請求 ： http://127.0.0.1:9200/shopping
返回結果如下：
{
"acknowledged": true
}
再次檢視所有索引，GET http://127.0.0.1:9200/_cat/indices?v，返回結果如下：
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
成功刪除。
2.6 文件
建立文件
假設索引已經建立好了，接下來我們來建立文件，並新增資料。這裡的文件可以類比為關係型資料庫中的表資料，新增的資料格式為 JSON 格式
在 Postman 中，向 ES 伺服器發 POST 請求 ： http://127.0.0.1:9200/shopping/_doc，請求體JSON內容為：
{
"title":"小米手機",
"category":"小米",
"images":"http://www.gulixueyuan.com/xm.jpg",
"price":3999.00
}
注意，此處傳送請求的方式必須為 POST，不能是 PUT，否則會發生錯誤 。
返回結果：
{
"_index": "shopping",//索引
"_type": "_doc",//型別-文件
"_id": "ANQqsHgBaKNfVnMbhZYU",//唯一標識，可以類比為 MySQL 中的主鍵，隨機生成
"_version": 1,//版本
"result": "created",//結果，這裡的 create 表示建立成功
"_shards": {//
"total": 2,//分片 - 總數
"successful": 1,//分片 - 總數
"failed": 0//分片 - 總數
},
"_seq_no": 0,
"_primary_term": 1
}
上面的資料建立後，由於沒有指定資料唯一性標識（ID），預設情況下， ES 伺服器會隨機生成一個。
如果想要自定義唯一性標識，需要在建立時指定： http://127.0.0.1:9200/shopping/_doc/1001，請求體JSON內容為：
{
"title":"小米手機",
"category":"小米",
"images":"http://www.gulixueyuan.com/xm.jpg",
"price":3999.00
}
返回結果如下：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1",//<------------------自定義唯一性標識
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 1,
"_primary_term": 1
}
此處需要注意：如果增加資料時明確資料主鍵，那麼請求方式也可以為 PUT。
主鍵查詢
檢視文件時，需要指明文件的唯一性標識，類似於 MySQL 中資料的主鍵查詢
在 Postman 中，向 ES 伺服器發 GET 請求 ： http://127.0.0.1:9200/shopping/_doc/1001 。
返回結果如下：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1",
"_version": 1,
"_seq_no": 1,
"_primary_term": 1,
"found": true,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
}
查詢不存在的內容，向 ES 伺服器發 GET 請求 ： http://127.0.0.1:9200/shopping/_doc/1002。
返回結果如下：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1002",
"found": false
}
全查詢
檢視索引下所有資料，向 ES 伺服器發 GET 請求 ： http://127.0.0.1:9200/shopping/_search。
返回結果如下：
{
"took": 61,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "HWQO434B7UmemU84rLH4",
"_score": 1.0,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999.00
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "1001",
"_score": 1.0,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999.00
}
}
]
}
}
全量修改
和新增文件一樣，輸入相同的 URL 地址請求，如果請求體變化，會將原有的資料內容覆蓋
在 Postman 中，向 ES 伺服器發 POST 請求 ： http://127.0.0.1:9200/shopping/_doc/1001
請求體JSON內容為:
{
"title":"華為手機",
"category":"華為",
"images":"http://www.gulixueyuan.com/hw.jpg",
"price":1999.00
}
修改成功後，伺服器響應結果：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1",
"_version": 2,
"result": "updated",//<-----------updated 表示資料被更新
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 2,
"_primary_term": 1
}
區域性修改
修改資料時，也可以只修改某一給條資料的區域性資訊
在 Postman 中，向 ES 伺服器發 POST 請求 ： http://127.0.0.1:9200/shopping/_update/1001。
請求體JSON內容為:
{
"doc": {
"title":"小米手機",
"category":"小米"
}
}
返回結果如下：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1001",
"_version": 3,
"result": "updated",//<-----------updated 表示資料被更新
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 3,
"_primary_term": 1
}
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_doc/1，檢視修改內容：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1001",
"_version": 4,
"_seq_no": 4,
"_primary_term": 1,
"found": true,
"_source": {
"doc": {
"title": "小米手機",
"category": "小米"
}
}
}
刪除文件
刪除一個文件不會立即從磁碟上移除，它只是被標記成已刪除（邏輯刪除）。
在 Postman 中，向 ES 伺服器發 DELETE 請求 ： http://127.0.0.1:9200/shopping/_doc/1001
返回結果：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1001",
"_version": 4,
"result": "deleted",//<---刪除成功
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 4,
"_primary_term": 1
}
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_doc/1001，檢視是否刪除成功：
{
"_index": "shopping",
"_type": "_doc",
"_id": "1001",
"found": false
}
URL帶參查詢
查詢category為小米的文件，在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search?q=category:小米，返回結果如下：
{
"took": 94,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 1.3862942,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
}
}
上述為URL帶引數形式查詢，這很容易讓不善者心懷惡意，或者引數值出現中文會出現亂碼情況。為了避免這些情況，我們可用使用帶JSON請求體請求進行查詢。
請求體帶參查詢
接下帶JSON請求體，還是查詢category為小米的文件，在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match":{
"category":"小米"
}
}
}
返回結果如下：
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 1.3862942,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
}
}
帶請求體方式的查詢所有內容
查詢所有文件內容，也可以這樣，在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match_all":{}
}
}
則返回所有文件內容：
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": 1,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": 1,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": 1,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
}
}
查詢指定欄位
如果你想查詢指定欄位，在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match_all":{}
},
"_source":["title"]
}
返回結果如下：
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 1,
"_source": {
"title": "小米手機"
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 1,
"_source": {
"title": "小米手機"
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": 1,
"_source": {
"title": "小米手機"
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": 1,
"_source": {
"title": "華為手機"
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": 1,
"_source": {
"title": "華為手機"
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": 1,
"_source": {
"title": "華為手機"
}
}
]
}
}
分頁查詢
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match_all":{}
},
"from":0,
"size":2
}
返回結果如下：
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
}
}
查詢排序
如果你想通過排序查出價格最高的手機，在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match_all":{}
},
"sort":{
"price":{
"order":"desc"
}
}
}
返回結果如下：
{
"took": 96,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": null,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
},
"sort": [
3999
]
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": null,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"sort": [
1999
]
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": null,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"sort": [
1999
]
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": null,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"sort": [
1999
]
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": null,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"sort": [
1999
]
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": null,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"sort": [
1999
]
}
]
}
}
多條件查詢
假設想找出小米牌子，價格為3999元的。（must相當於資料庫的&&）
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下
{
"query":{
"bool":{
"must":[{
"match":{
"category":"小米"
}
},{
"match":{
"price":3999.00
}
}]
}
}
}
返回結果如下：
{
"took": 134,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 2.3862944,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 2.3862944,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
}
]
}
}
範圍查詢
假設想找出小米或華為價格高於2000的手機。（should相當於資料庫的||）
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"bool":{
"should":[{
"match":{
"category":"小米"
}
},{
"match":{
"category":"華為"
}
}]
},
"filter":{
"range":{
"price":{
"gt":2000
}
}
}
}
}
返回結果如下：
{
"took": 8,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": 1.3862942,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": 1.3862942,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": 1.3862942,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": 1.3862942,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": 1.3862942,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
}
}
全文檢索
這功能像搜尋引擎那樣，如品牌輸入“小華”，返回結果帶回品牌有“小米”和華為的。
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match":{
"category" : "小華"
}
}
}
返回結果如下：
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": 0.6931471,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 0.6931471,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 0.6931471,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": 0.6931471,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
}
}
完全匹配
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match_phrase":{
"category" : "為"
}
}
}
返回結果如下：
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.6931471,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
}
}
高亮查詢
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"query":{
"match_phrase":{
"category" : "為"
}
},
"highlight":{
"fields":{
"category":{}//<----高亮這欄位
}
}
}
返回結果如下：
{
"took": 100,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.6931471,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"highlight": {
"category": [
"華<em>為</em>"//<------高亮一個為字。
]
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"highlight": {
"category": [
"華<em>為</em>"
]
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": 0.6931471,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
},
"highlight": {
"category": [
"華<em>為</em>"
]
}
}
]
}
}
聚合查詢
聚合允許使用者對 es 文件進行統計分析，類似與關係型資料庫中的 group by，當然還有很多其他的聚合，例如取最大值max、平均值avg等等。
接下來按price欄位進行分組：
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"aggs":{//聚合操作
"price_group":{//名稱，隨意起名
"terms":{//分組
"field":"price"//分組欄位
}
}
}
}
返回結果如下：
{
"took": 63,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "shopping",
"_type": "_doc",
"_id": "ANQqsHgBaKNfVnMbhZYU",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 3999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "A9R5sHgBaKNfVnMb25Ya",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BNR5sHgBaKNfVnMb7pal",
"_score": 1,
"_source": {
"title": "小米手機",
"category": "小米",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "BtR6sHgBaKNfVnMbX5Y5",
"_score": 1,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "B9R6sHgBaKNfVnMbZpZ6",
"_score": 1,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
},
{
"_index": "shopping",
"_type": "_doc",
"_id": "CdR7sHgBaKNfVnMbsJb9",
"_score": 1,
"_source": {
"title": "華為手機",
"category": "華為",
"images": "http://www.gulixueyuan.com/xm.jpg",
"price": 1999
}
}
]
},
"aggregations": {
"price_group": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 1999,
"doc_count": 5
},
{
"key": 3999,
"doc_count": 1
}
]
}
}
}
上面返回結果會附帶原始資料的。若不想要不附帶原始資料的結果，在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"aggs":{
"price_group":{
"terms":{
"field":"price"
}
}
},
"size":0
}
返回結果如下：
{
"took": 60,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"aggregations": {
"price_group": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 1999,
"doc_count": 5
},
{
"key": 3999,
"doc_count": 1
}
]
}
}
}
若想對所有手機價格求平均值。
在 Postman 中，向 ES 伺服器發 GET請求 ： http://127.0.0.1:9200/shopping/_search，附帶JSON體如下：
{
"aggs":{
"price_avg":{//名稱，隨意起名
"avg":{//求平均
"field":"price"
}
}
},
"size":0
}
返回結果如下：
{
"took": 14,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 6,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"aggregations": {
"price_avg": {
"value": 2332.3333333333335
}
}
}
對映關係
有了索引庫，等於有了資料庫中的 database。
接下來就需要建索引庫(index)中的映射了，類似於資料庫(database)中的表結構(table)。
建立資料庫表需要設定欄位名稱，型別，長度，約束等；索引庫也一樣，需要知道這個型別下有哪些欄位，每個欄位有哪些約束資訊，這就叫做對映(mapping)。
先建立一個索引：
# PUT http://127.0.0.1:9200/user
返回結果：
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "user"
}
建立對映
# PUT http://127.0.0.1:9200/user/_mapping
{
"properties": {
"name":{
"type": "text",
"index": true
},
"sex":{
"type": "keyword",
"index": true
},
"tel":{
"type": "keyword",
"index": false
}
}
}
返回結果如下：
{
"acknowledged": true
}
查詢對映
#GET http://127.0.0.1:9200/user/_mapping
返回結果如下：
{
"user": {
"mappings": {
"properties": {
"name": {
"type": "text"
},
"sex": {
"type": "keyword"
},
"tel": {
"type": "keyword",
"index": false
}
}
}
}
}
增加資料
#PUT http://127.0.0.1:9200/user/_create/1001
{
"name":"小米",
"sex":"男的",
"tel":"1111"
}
返回結果如下：
{
"_index": "user",
"_type": "_doc",
"_id": "1001",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1
}
查詢name含有”小“資料：json
#GET http://127.0.0.1:9200/user/_search
{
"query":{
"match":{
"name":"小"
}
}
}
返回結果如下：
{
"took": 495,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.2876821,
"hits": [
{
"_index": "user",
"_type": "_doc",
"_id": "1001",
"_score": 0.2876821,
"_source": {
"name": "小米",
"sex": "男的",
"tel": "1111"
}
}
]
}
}
查詢sex含有”男“資料：
#GET http://127.0.0.1:9200/user/_search
{
"query":{
"match":{
"sex":"男"
}
}
}
返回結果如下：
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
}
}
找不想要的結果，只因建立對映時"sex"的型別為"keyword"。
"sex"只能完全為”男的“，才能得出原資料。
#GET http://127.0.0.1:9200/user/_search
{
"query":{
"match":{
"sex":"男的"
}
}
}
返回結果如下：
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.2876821,
"hits": [
{
"_index": "user",
"_type": "_doc",
"_id": "1001",
"_score": 0.2876821,
"_source": {
"name": "小米",
"sex": "男的",
"tel": "1111"
}
}
]
}
}
查詢電話
# GET http://127.0.0.1:9200/user/_search
{
"query":{
"match":{
"tel":"11"
}
}
}
返回結果如下：
{
"error": {
"root_cause": [
{
"type": "query_shard_exception",
"reason": "failed to create query: Cannot search on field [tel] since it is not indexed.",
"index_uuid": "ivLnMfQKROS7Skb2MTFOew",
"index": "user"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "user",
"node": "4P7dIRfXSbezE5JTiuylew",
"reason": {
"type": "query_shard_exception",
"reason": "failed to create query: Cannot search on field [tel] since it is not indexed.",
"index_uuid": "ivLnMfQKROS7Skb2MTFOew",
"index": "user",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Cannot search on field [tel] since it is not indexed."
}
}
}
]
},
"status": 400
}
報錯只因建立對映時"tel"的"index"為false。
第三章 JavaAPI
新建Maven工程。
新增依賴：
<dependencies>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.8.0</version>
</dependency>
<!-- elasticsearch 的客戶端 -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.8.0</version>
</dependency>
<!-- elasticsearch 依賴 2.x 的 log4j -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.9</version>
</dependency>
<!-- junit 單元測試 -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
</dependencies>
HelloElasticsearch
import java.io.IOException;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
public class HelloElasticsearch {
public static void main(String[] args) throws IOException {
// 建立客戶端物件
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
// ...
System.out.println(client);
// 關閉客戶端連線
client.close();
}
}
3.1 索引操作
建立索引
import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.action.admin.indices.create.CreateIndexResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import java.io.IOException;
public class CreateIndex {
public static void main(String[] args) throws IOException {
// 建立客戶端物件
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
// 建立索引 - 請求物件
CreateIndexRequest request = new CreateIndexRequest("user2");
// 傳送請求，獲取響應
CreateIndexResponse response = client.indices().create(request,
RequestOptions.DEFAULT);
boolean acknowledged = response.isAcknowledged();
// 響應狀態
System.out.println("操作狀態 = " + acknowledged);
// 關閉客戶端連線
client.close();
}
}
後臺列印
二月 10, 2022 8:32:24 下午 org.elasticsearch.client.RestClient logResponse
警告: request [PUT http://localhost:9200/user2?master_timeout=30s&include_type_name=true&timeout=30s] returned 1 warnings: [299 Elasticsearch-7.8.0-757314695644ea9a1dc2fecd26d1a43856725e65 "[types removal] Using include_type_name in create index requests is deprecated. The parameter will be removed in the next major version."]
操作狀態 = true
查詢索引
import org.apache.http.HttpHost;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.client.indices.GetIndexResponse;
import java.io.IOException;
public class SearchIndex {
public static void main(String[] args) throws IOException {
// 建立客戶端物件
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
// 查詢索引 - 請求物件
GetIndexRequest request = new GetIndexRequest("user2");
// 傳送請求，獲取響應
GetIndexResponse response = client.indices().get(request,
RequestOptions.DEFAULT);
System.out.println("aliases:"+response.getAliases());
System.out.println("mappings:"+response.getMappings());
System.out.println("settings:"+response.getSettings());
client.close();
}
}
後臺列印：
aliases:{user2=[]}
mappings:{user2=org.elasticsearch.cluster.metadata.MappingMetadata@ceb7e09a}
settings:{user2={"index.creation_date":"1644496343959","index.number_of_replicas":"1","index.number_of_shards":"1","index.provided_name":"user2","index.uuid":"dnRxs3_iSm6SM39LSe8_vA","index.version.created":"7080099"}}
刪除索引
import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import java.io.IOException;
public class DeleteIndex {
public static void main(String[] args) throws IOException {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
// 刪除索引 - 請求物件
DeleteIndexRequest request = new DeleteIndexRequest("user2");
// 傳送請求，獲取響應
AcknowledgedResponse response = client.indices().delete(request,RequestOptions.DEFAULT);
// 操作結果
System.out.println("操作結果 ： " + response.isAcknowledged());
client.close();
}
}
後臺列印：
操作結果 ： true
Process finished with exit code 0
3.2 重構優化
上文由於頻繁使用以下連線Elasticsearch和關閉它的程式碼，於是個人對它進行重構。
public class SomeClass {
public static void main(String[] args) throws IOException {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
...
client.close();
}
}
重構後的程式碼：
import org.elasticsearch.client.RestHighLevelClient;
public interface ElasticsearchTask {
void doSomething(RestHighLevelClient client) throws Exception;
}
使用
public class ConnectElasticsearch{
public static void connect(ElasticsearchTask task){
// 建立客戶端物件
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
try {
task.doSomething(client);
// 關閉客戶端連線
client.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
接下來，如果想讓Elasticsearch完成一些操作，就編寫一個lambda式即可。
public class SomeClass {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
//do something
});
}
}
封裝好User類
public class User {
private String name;
private String sex;
private Integer age;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getSex() {
return sex;
}
public void setSex(String sex) {
this.sex = sex;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
}
3.3 文件操作
新增文件
import com.fasterxml.jackson.databind.ObjectMapper;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.common.xcontent.XContentType;
public class InsertDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
// 新增文件 - 請求物件
IndexRequest request = new IndexRequest();
// 設定索引及唯一性標識
request.index("user").id("1001");
// 建立資料物件
User user = new User();
user.setName("zhangsan");
user.setAge(30);
user.setSex("男");
ObjectMapper objectMapper = new ObjectMapper();
String productJson = objectMapper.writeValueAsString(user);
// 新增文件資料，資料格式為 JSON 格式
request.source(productJson, XContentType.JSON);
// 客戶端傳送請求，獲取響應物件
IndexResponse response = client.index(request, RequestOptions.DEFAULT);
// 列印結果資訊
System.out.println("_index:" + response.getIndex());
System.out.println("_id:" + response.getId());
System.out.println("_result:" + response.getResult());
});
}
}
後臺列印：
_index:user
_id:1001
_result:UPDATED
Process finished with exit code 0
修改文件
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.common.xcontent.XContentType;
public class UpdateDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
// 修改文件 - 請求物件
UpdateRequest request = new UpdateRequest();
// 配置修改引數
request.index("user").id("1001");
// 設定請求體，對資料進行修改
request.doc(XContentType.JSON, "sex", "女");
// 客戶端傳送請求，獲取響應物件
UpdateResponse response = client.update(request, RequestOptions.DEFAULT);
System.out.println("_index:" + response.getIndex());
System.out.println("_id:" + response.getId());
System.out.println("_result:" + response.getResult());
});
}
}
後臺列印：
_index:user
_id:1001
_result:UPDATED
Process finished with exit code 0
查詢文件
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.client.RequestOptions;
public class GetDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
//1.建立請求物件
GetRequest request = new GetRequest().index("user").id("1001");
//2.客戶端傳送請求，獲取響應物件
GetResponse response = client.get(request, RequestOptions.DEFAULT);
//3.列印結果資訊
System.out.println("_index:" + response.getIndex());
System.out.println("_type:" + response.getType());
System.out.println("_id:" + response.getId());
System.out.println("source:" + response.getSourceAsString());
});
}
}
後臺列印：
_index:user
_type:_doc
_id:1001
source:{"name":"zhangsan","sex":"女","age":30}
Process finished with exit code 0
刪除文件
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.client.RequestOptions;
public class DeleteDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
//建立請求物件
DeleteRequest request = new DeleteRequest().index("user").id("1001");
//客戶端傳送請求，獲取響應物件
DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);
//列印資訊
System.out.println(response.toString());
});
}
}
後臺列印：
DeleteResponse[index=user,type=_doc,id=1001,version=3,result=deleted,shards=ShardInfo{total=2, successful=1, failures=[]}]
Process finished with exit code 0
3.4 批量操作
批量新增
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.common.xcontent.XContentType;
public class BatchInsertDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
//建立批量新增請求物件
BulkRequest request = new BulkRequest();
request.add(new
IndexRequest().index("user").id("1001").source(XContentType.JSON, "name",
"zhangsan"));
request.add(new
IndexRequest().index("user").id("1002").source(XContentType.JSON, "name",
"lisi"));
request.add(new
IndexRequest().index("user").id("1003").source(XContentType.JSON, "name",
"wangwu"));
//客戶端傳送請求，獲取響應物件
BulkResponse responses = client.bulk(request, RequestOptions.DEFAULT);
//列印結果資訊
System.out.println("took:" + responses.getTook());
System.out.println("items:" + responses.getItems());
});
}
}
後臺列印
took:4ms
items:[Lorg.elasticsearch.action.bulk.BulkItemResponse;@5552768b
Process finished with exit code 0
批量刪除
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.client.RequestOptions;
public class BatchDeleteDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
//建立批量刪除請求物件
BulkRequest request = new BulkRequest();
request.add(new DeleteRequest().index("user").id("1001"));
request.add(new DeleteRequest().index("user").id("1002"));
request.add(new DeleteRequest().index("user").id("1003"));
//客戶端傳送請求，獲取響應物件
BulkResponse responses = client.bulk(request, RequestOptions.DEFAULT);
//列印結果資訊
System.out.println("took:" + responses.getTook());
System.out.println("items:" + responses.getItems());
});
}
}
後臺列印
took:4ms
items:[Lorg.elasticsearch.action.bulk.BulkItemResponse;@5c671d7f
3.5 高階查詢
全量查詢
先批量增加資料
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.common.xcontent.XContentType;
public class BatchInsertDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
//建立批量新增請求物件
BulkRequest request = new BulkRequest();
request.add(new IndexRequest().index("user").id("1001").source(XContentType.JSON, "name", "zhangsan", "age", "10", "sex","女"));
request.add(new IndexRequest().index("user").id("1002").source(XContentType.JSON, "name", "lisi", "age", "30", "sex","女"));
request.add(new IndexRequest().index("user").id("1003").source(XContentType.JSON, "name", "wangwu1", "age", "40", "sex","男"));
request.add(new IndexRequest().index("user").id("1004").source(XContentType.JSON, "name", "wangwu2", "age", "20", "sex","女"));
request.add(new IndexRequest().index("user").id("1005").source(XContentType.JSON, "name", "wangwu3", "age", "50", "sex","男"));
request.add(new IndexRequest().index("user").id("1006").source(XContentType.JSON, "name", "wangwu4", "age", "20", "sex","男"));
//客戶端傳送請求，獲取響應物件
BulkResponse responses = client.bulk(request, RequestOptions.DEFAULT);
//列印結果資訊
System.out.println("took:" + responses.getTook());
System.out.println("items:" + responses.getItems());
});
}
}
查詢所有索引資料
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
public class QueryDoc {
public static void main(String[] args) {
ConnectElasticsearch.connect(client -> {
// 建立搜尋請求物件
SearchRequest request = new SearchRequest();
request.indices("user");
// 構建查詢的請求體
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 查詢所有資料
sourceBuilder.query(QueryBuilders.matchAllQuery());
request.source(sourceBuilder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
// 查詢匹配
SearchHits hits = response.getHits();
System.out.println("took:" + response.getTook());
System.out.println("timeout:" + response.isTimedOut());
System.out.println("total:" + hits.getTotalHits());
System.out.println("MaxScore:" + hits.getMaxScore());
System.out.println("hits========>>");
for (SearchHit hit : hits) {
//輸出每條查詢的結果資訊
System.out.println(hit.getSourceAsString());
}
System.out.println("<<========");
});
}
}
後臺列印
took:743ms
timeout:false
total:6 hits
MaxScore:1.0
hits========>>
{"name":"zhangsan","age":"10","sex":"女"}
{"name":"lisi","age":"30","sex":"女"}
{"name":"wangwu1","age":"40","sex":"男"}
{"name":"wangwu2","age":"20","sex":"女"}
{"name":"wangwu3","age":"50","sex":"男"}
{"name":"wangwu4","age":"20","sex":"男"}
<<========
條件查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.sort.SortOrder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_BY_CONDITION = client -> {
// 建立搜尋請求物件
SearchRequest request = new SearchRequest();
request.indices("user");
// 構建查詢的請求體
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.termQuery("age", "30"));
request.source(sourceBuilder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
// 查詢匹配
SearchHits hits = response.getHits();
System.out.println("took:" + response.getTook());
System.out.println("timeout:" + response.isTimedOut());
System.out.println("total:" + hits.getTotalHits());
System.out.println("MaxScore:" + hits.getMaxScore());
System.out.println("hits========>>");
for (SearchHit hit : hits) {
//輸出每條查詢的結果資訊
System.out.println(hit.getSourceAsString());
}
System.out.println("<<========");
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_BY_CONDITION);
}
}
後臺列印
took:29ms
timeout:false
total:1 hits
MaxScore:1.0
hits========>>
{"name":"lisi","age":"30","sex":"女"}
<<========
分頁查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.sort.SortOrder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_BY_PAGING = client -> {
// 建立搜尋請求物件
SearchRequest request = new SearchRequest();
request.indices("user");
// 構建查詢的請求體
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.matchAllQuery());
// 分頁查詢
// 當前頁其實索引(第一條資料的順序號)， from
sourceBuilder.from(0);
// 每頁顯示多少條 size
sourceBuilder.size(2);
request.source(sourceBuilder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
// 查詢匹配
SearchHits hits = response.getHits();
System.out.println("took:" + response.getTook());
System.out.println("timeout:" + response.isTimedOut());
System.out.println("total:" + hits.getTotalHits());
System.out.println("MaxScore:" + hits.getMaxScore());
System.out.println("hits========>>");
for (SearchHit hit : hits) {
//輸出每條查詢的結果資訊
System.out.println(hit.getSourceAsString());
}
System.out.println("<<========");
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_BY_PAGING);
}
}
後臺列印
took:2ms
timeout:false
total:6 hits
MaxScore:1.0
hits========>>
{"name":"zhangsan","age":"10","sex":"女"}
{"name":"lisi","age":"30","sex":"女"}
<<========
Process finished with exit code 0
查詢排序
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.sort.SortOrder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_WITH_ORDER = client -> {
// 建立搜尋請求物件
SearchRequest request = new SearchRequest();
request.indices("user");
// 構建查詢的請求體
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.matchAllQuery());
// 排序
sourceBuilder.sort("age", SortOrder.ASC);
request.source(sourceBuilder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
// 查詢匹配
SearchHits hits = response.getHits();
System.out.println("took:" + response.getTook());
System.out.println("timeout:" + response.isTimedOut());
System.out.println("total:" + hits.getTotalHits());
System.out.println("MaxScore:" + hits.getMaxScore());
System.out.println("hits========>>");
for (SearchHit hit : hits) {
//輸出每條查詢的結果資訊
System.out.println(hit.getSourceAsString());
}
System.out.println("<<========");
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_WITH_ORDER);
}
}
後臺列印
took:81ms
timeout:false
total:6 hits
MaxScore:NaN
hits========>>
{"name":"zhangsan","age":"10","sex":"女"}
{"name":"wangwu2","age":"20","sex":"女"}
{"name":"wangwu4","age":"20","sex":"男"}
{"name":"lisi","age":"30","sex":"女"}
{"name":"wangwu1","age":"40","sex":"男"}
{"name":"wangwu3","age":"50","sex":"男"}
<<========
Process finished with exit code 0
組合查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.sort.SortOrder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_BY_BOOL_CONDITION = client -> {
// 建立搜尋請求物件
SearchRequest request = new SearchRequest();
request.indices("user");
// 構建查詢的請求體
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
// 必須包含
boolQueryBuilder.must(QueryBuilders.matchQuery("age", "30"));
// 一定不含
boolQueryBuilder.mustNot(QueryBuilders.matchQuery("name", "zhangsan"));
// 可能包含
boolQueryBuilder.should(QueryBuilders.matchQuery("sex", "男"));
sourceBuilder.query(boolQueryBuilder);
request.source(sourceBuilder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
// 查詢匹配
SearchHits hits = response.getHits();
System.out.println("took:" + response.getTook());
System.out.println("timeout:" + response.isTimedOut());
System.out.println("total:" + hits.getTotalHits());
System.out.println("MaxScore:" + hits.getMaxScore());
System.out.println("hits========>>");
for (SearchHit hit : hits) {
//輸出每條查詢的結果資訊
System.out.println(hit.getSourceAsString());
}
System.out.println("<<========");
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_BY_BOOL_CONDITION);
}
}
後臺列印
took:42ms
timeout:false
total:1 hits
MaxScore:1.0
hits========>>
{"name":"lisi","age":"30","sex":"女"}
<<========
Process finished with exit code 0
範圍查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.RangeQueryBuilder;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_BY_RANGE = client -> {
// 建立搜尋請求物件
SearchRequest request = new SearchRequest();
request.indices("user");
// 構建查詢的請求體
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("age");
// 大於等於
//rangeQuery.gte("30");
// 小於等於
rangeQuery.lte("40");
sourceBuilder.query(rangeQuery);
request.source(sourceBuilder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
// 查詢匹配
SearchHits hits = response.getHits();
System.out.println("took:" + response.getTook());
System.out.println("timeout:" + response.isTimedOut());
System.out.println("total:" + hits.getTotalHits());
System.out.println("MaxScore:" + hits.getMaxScore());
System.out.println("hits========>>");
for (SearchHit hit : hits) {
//輸出每條查詢的結果資訊
System.out.println(hit.getSourceAsString());
}
System.out.println("<<========");
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_BY_RANGE);
}
}
後臺列印
took:8ms
timeout:false
total:5 hits
MaxScore:1.0
hits========>>
{"name":"zhangsan","age":"10","sex":"女"}
{"name":"lisi","age":"30","sex":"女"}
{"name":"wangwu1","age":"40","sex":"男"}
{"name":"wangwu2","age":"20","sex":"女"}
{"name":"wangwu4","age":"20","sex":"男"}
<<========
Process finished with exit code 0
模糊查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.common.unit.Fuzziness;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.RangeQueryBuilder;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.sort.SortOrder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_BY_FUZZY_CONDITION = client -> {
// 建立搜尋請求物件
SearchRequest request = new SearchRequest();
request.indices("user");
// 構建查詢的請求體
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.fuzzyQuery("name","wangwu").fuzziness(Fuzziness.ONE));
request.source(sourceBuilder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
// 查詢匹配
SearchHits hits = response.getHits();
System.out.println("took:" + response.getTook());
System.out.println("timeout:" + response.isTimedOut());
System.out.println("total:" + hits.getTotalHits());
System.out.println("MaxScore:" + hits.getMaxScore());
System.out.println("hits========>>");
for (SearchHit hit : hits) {
//輸出每條查詢的結果資訊
System.out.println(hit.getSourceAsString());
}
System.out.println("<<========");
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_BY_FUZZY_CONDITION);
}
}
後臺列印
took:41ms
timeout:false
total:4 hits
MaxScore:1.2837042
hits========>>
{"name":"wangwu1","age":"40","sex":"男"}
{"name":"wangwu2","age":"20","sex":"女"}
{"name":"wangwu3","age":"50","sex":"男"}
{"name":"wangwu4","age":"20","sex":"男"}
<<========
Process finished with exit code 0
高亮查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.TermsQueryBuilder;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import java.util.Map;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_WITH_HIGHLIGHT = client -> {
// 高亮查詢
SearchRequest request = new SearchRequest().indices("user");
//2.建立查詢請求體構建器
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
//構建查詢方式：高亮查詢
TermsQueryBuilder termsQueryBuilder =
QueryBuilders.termsQuery("name","zhangsan");
//設定查詢方式
sourceBuilder.query(termsQueryBuilder);
//構建高亮欄位
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.preTags("<font color='red'>");//設定標籤字首
highlightBuilder.postTags("</font>");//設定標籤字尾
highlightBuilder.field("name");//設定高亮欄位
//設定高亮構建物件
sourceBuilder.highlighter(highlightBuilder);
//設定請求體
request.source(sourceBuilder);
//3.客戶端傳送請求，獲取響應物件
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//4.列印響應結果
SearchHits hits = response.getHits();
System.out.println("took::"+response.getTook());
System.out.println("time_out::"+response.isTimedOut());
System.out.println("total::"+hits.getTotalHits());
System.out.println("max_score::"+hits.getMaxScore());
System.out.println("hits::::>>");
for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
System.out.println(sourceAsString);
//列印高亮結果
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
System.out.println(highlightFields);
}
System.out.println("<<::::");
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_WITH_HIGHLIGHT);
}
}
後臺列印
took::85ms
time_out::false
total::1 hits
max_score::1.0
hits::::>>
{"name":"zhangsan","age":"10","sex":"女"}
{name=[name], fragments[[<font color='red'>zhangsan</font>]]}
<<::::
Process finished with exit code 0
最大值查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.builder.SearchSourceBuilder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_WITH_MAX = client -> {
// 高亮查詢
SearchRequest request = new SearchRequest().indices("user");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.aggregation(AggregationBuilders.max("maxAge").field("age"));
//設定請求體
request.source(sourceBuilder);
//3.客戶端傳送請求，獲取響應物件
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//4.列印響應結果
SearchHits hits = response.getHits();
System.out.println(response);
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_WITH_MAX);
}
}
後臺列印
{"took":16,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":6,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"user","_type":"_doc","_id":"1001","_score":1.0,"_source":{"name":"zhangsan","age":"10","sex":"女"}},{"_index":"user","_type":"_doc","_id":"1002","_score":1.0,"_source":{"name":"lisi","age":"30","sex":"女"}},{"_index":"user","_type":"_doc","_id":"1003","_score":1.0,"_source":{"name":"wangwu1","age":"40","sex":"男"}},{"_index":"user","_type":"_doc","_id":"1004","_score":1.0,"_source":{"name":"wangwu2","age":"20","sex":"女"}},{"_index":"user","_type":"_doc","_id":"1005","_score":1.0,"_source":{"name":"wangwu3","age":"50","sex":"男"}},{"_index":"user","_type":"_doc","_id":"1006","_score":1.0,"_source":{"name":"wangwu4","age":"20","sex":"男"}}]},"aggregations":{"max#maxAge":{"value":50.0}}}
Process finished with exit code 0
分組查詢
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.builder.SearchSourceBuilder;
public class QueryDoc {
public static final ElasticsearchTask SEARCH_WITH_GROUP = client -> {
SearchRequest request = new SearchRequest().indices("user");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.aggregation(AggregationBuilders.terms("age_groupby").field("age"));
//設定請求體
request.source(sourceBuilder);
//3.客戶端傳送請求，獲取響應物件
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//4.列印響應結果
SearchHits hits = response.getHits();
System.out.println(response);
};
public static void main(String[] args) {
ConnectElasticsearch.connect(SEARCH_WITH_GROUP);
}
}
後臺列印
{"took":14,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":6,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"user","_type":"_doc","_id":"1001","_score":1.0,"_source":{"name":"zhangsan","age":"10","sex":"女"}},{"_index":"user","_type":"_doc","_id":"1002","_score":1.0,"_source":{"name":"lisi","age":"30","sex":"女"}},{"_index":"user","_type":"_doc","_id":"1003","_score":1.0,"_source":{"name":"wangwu1","age":"40","sex":"男"}},{"_index":"user","_type":"_doc","_id":"1004","_score":1.0,"_source":{"name":"wangwu2","age":"20","sex":"女"}},{"_index":"user","_type":"_doc","_id":"1005","_score":1.0,"_source":{"name":"wangwu3","age":"50","sex":"男"}},{"_index":"user","_type":"_doc","_id":"1006","_score":1.0,"_source":{"name":"wangwu4","age":"20","sex":"男"}}]},"aggregations":{"lterms#age_groupby":{"doc_count_error_upper_bound":0,"sum_other_doc_count":0,"buckets":[{"key":20,"doc_count":2},{"key":10,"doc_count":1},{"key":30,"doc_count":1},{"key":40,"doc_count":1},{"key":50,"doc_count":1}]}}}
Process finished with exit code 0
第四章、Elasticsearch環境
4.1 相關概念
單機 & 叢集
單臺 Elasticsearch 伺服器提供服務，往往都有最大的負載能力，超過這個閾值，伺服器
效能就會大大降低甚至不可用，所以生產環境中，一般都是執行在指定伺服器叢集中。
除了負載能力，單點伺服器也存在其他問題：
- 單臺機器儲存容量有限
- 單伺服器容易出現單點故障，無法實現高可用
- 單服務的併發處理能力有限
配置伺服器叢集時，叢集中節點數量沒有限制，大於等於 2 個節點就可以看做是叢集了。一
般出於高效能及高可用方面來考慮叢集中節點數量都是 3 個以上
總之，叢集能提高效能，增加容錯。
叢集 Cluster
一個叢集就是由一個或多個伺服器節點組織在一起，共同持有整個的資料，並一起提供索引和搜尋功能。一個 Elasticsearch 叢集有一個唯一的名字標識，這個名字預設就是”elasticsearch”。這個名字是重要的，因為一個節點只能通過指定某個叢集的名字，來加入這個叢集。
節點 Node
叢集中包含很多伺服器， 一個節點就是其中的一個伺服器。 作為叢集的一部分，它儲存資料，參與叢集的索引和搜尋功能。
一個節點也是由一個名字來標識的，預設情況下，這個名字是一個隨機的漫威漫畫角色的名字，這個名字會在啟動的時候賦予節點。這個名字對於管理工作來說挺重要的，因為在這個管理過程中，你會去確定網路中的哪些伺服器對應於 Elasticsearch 叢集中的哪些節點。
一個節點可以通過配置叢集名稱的方式來加入一個指定的叢集。預設情況下，每個節點都會被安排加入到一個叫做“elasticsearch”的叢集中，這意味著，如果你在你的網路中啟動了若干個節點，並假定它們能夠相互發現彼此，它們將會自動地形成並加入到一個叫做“elasticsearch”的叢集中。
在一個叢集裡，只要你想，可以擁有任意多個節點。而且，如果當前你的網路中沒有運
行任何 Elasticsearch 節點，這時啟動一個節點，會預設建立並加入一個叫做“elasticsearch”的
叢集。
4.2 Windows 叢集部署
部署叢集
一、建立 elasticsearch-cluster 資料夾
建立 elasticsearch-7.8.0-cluster 資料夾，在內部複製三個 elasticsearch 服務。
二、修改叢集檔案目錄中每個節點的 config/elasticsearch.yml 配置檔案
node-1001 節點
#節點 1 的配置資訊：
#叢集名稱，節點之間要保持一致
cluster.name: my-elasticsearch
#節點名稱，叢集內要唯一
node.name: node-1001
node.master: true
node.data: true
#ip 地址
network.host: localhost
#http 埠
http.port: 1001
#tcp 監聽埠
transport.tcp.port: 9301
#discovery.seed_hosts: ["localhost:9301", "localhost:9302","localhost:9303"]
#discovery.zen.fd.ping_timeout: 1m
#discovery.zen.fd.ping_retries: 5
#叢集內的可以被選為主節點的節點列表
#cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
#跨域配置
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
node-1002 節點
#節點 2 的配置資訊：
#叢集名稱，節點之間要保持一致
cluster.name: my-elasticsearch
#節點名稱，叢集內要唯一
node.name: node-1002
node.master: true
node.data: true
#ip 地址
network.host: localhost
#http 埠
http.port: 1002
#tcp 監聽埠
transport.tcp.port: 9302
discovery.seed_hosts: ["localhost:9301"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#叢集內的可以被選為主節點的節點列表
#cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
#跨域配置
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
node-1003 節點
#節點 3 的配置資訊：
#叢集名稱，節點之間要保持一致
cluster.name: my-elasticsearch
#節點名稱，叢集內要唯一
node.name: node-1003
node.master: true
node.data: true
#ip 地址
network.host: localhost
#http 埠
http.port: 1003
#tcp 監聽埠
transport.tcp.port: 9303
#候選主節點的地址，在開啟服務後可以被選為主節點
discovery.seed_hosts: ["localhost:9301", "localhost:9302"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#叢集內的可以被選為主節點的節點列表
#cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
#跨域配置
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
啟動叢集
刪除每個節點中的 data 目錄中所有內容 。
分別依次雙擊執行節點的bin/elasticsearch.bat, 啟動節點伺服器（可以編寫一個指令碼啟動），啟動後，會自動加入指定名稱的叢集。
測試叢集
一、用Postman，檢視叢集狀態
GET http://127.0.0.1:1001/_cluster/health
GET http://127.0.0.1:1002/_cluster/health
GET http://127.0.0.1:1003/_cluster/health
返回結果皆為如下：
{
"cluster_name": "my-application",
"status": "green",
"timed_out": false,
"number_of_nodes": 3,
"number_of_data_nodes": 3,
"active_primary_shards": 0,
"active_shards": 0,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 0,
"delayed_unassigned_shards": 0,
"number_of_pending_tasks": 0,
"number_of_in_flight_fetch": 0,
"task_max_waiting_in_queue_millis": 0,
"active_shards_percent_as_number": 100.0
}
status欄位指示著當前叢集在總體上是否工作正常。它的三種顏色含義如下：
- green：所有的主分片和副本分片都正常執行。
- yellow：所有的主分片都正常執行，但不是所有的副本分片都正常執行。
- red：有主分片沒能正常執行。
二、用Postman，在一節點增加索引，另一節點獲取索引
向叢集中的node-1001節點增加索引：
#PUT http://127.0.0.1:1001/user
返回結果：
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "user"
}
向叢集中的node-1003節點獲取索引：
#GET http://127.0.0.1:1003/user
返回結果：
{
"user": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1617993035885",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "XJKERwQlSJ6aUxZEN2EV0w",
"version": {
"created": "7080099"
},
"provided_name": "user"
}
}
}
}
如果在1003建立索引，同樣在1001也能獲取索引資訊，這就是叢集能力。
4.3 Linux 單機部署
軟體安裝
一、下載軟體
二、解壓軟體
# 解壓縮
tar -zxvf elasticsearch-7.8.0-linux-x86_64.tar.gz -C /opt/module
# 改名
mv elasticsearch-7.8.0 es
三、建立使用者
因為安全問題， Elasticsearch 不允許 root 使用者直接執行，所以要建立新使用者，在 root 使用者中建立新使用者。
useradd es #新增 es 使用者
passwd es #為 es 使用者設定密碼
userdel -r es #如果錯了，可以刪除再加
chown -R es:es /opt/module/es #資料夾所有者
四、修改配置檔案
修改/opt/module/es/config/elasticsearch.yml檔案。
# 加入如下配置
cluster.name: elasticsearch
node.name: node-1
network.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["node-1"]
修改/etc/security/limits.conf
# 在檔案末尾中增加下面內容
# 每個程序可以開啟的檔案數的限制
es soft nofile 65536
es hard nofile 65536
修改/etc/security/limits.d/20-nproc.conf
# 在檔案末尾中增加下面內容
# 每個程序可以開啟的檔案數的限制
es soft nofile 65536
es hard nofile 65536
# 作業系統級別對每個使用者建立的程序數的限制
* hard nproc 4096
# 注： * 帶表 Linux 所有使用者名稱稱
修改/etc/sysctl.conf
# 在檔案中增加下面內容
# 一個程序可以擁有的 VMA(虛擬記憶體區域)的數量,預設值為 65536
vm.max_map_count=655360
重新載入
sysctl -p
啟動軟體
使用 ES 使用者啟動
cd /opt/module/es/
#啟動
bin/elasticsearch
#後臺啟動
bin/elasticsearch -d
啟動時，會動態生成檔案，如果檔案所屬使用者不匹配，會發生錯誤，需要重新進行修改使用者和使用者組
關閉防火牆
#暫時關閉防火牆
systemctl stop firewalld
#永久關閉防火牆
systemctl enable firewalld.service #開啟防火牆永久性生效，重啟後不會復原
systemctl disable firewalld.service #關閉防火牆，永久性生效，重啟後不會復原
測試軟體
瀏覽器中輸入地址： http://linux1:9200/
4.4 Linux 叢集部署
軟體安裝
一、下載軟體
二、解壓軟體
# 解壓縮
tar -zxvf elasticsearch-7.8.0-linux-x86_64.tar.gz -C /opt/module
# 改名
mv elasticsearch-7.8.0 es-cluster
將軟體分發到其他節點： linux2, linux3
三、建立使用者
因為安全問題， Elasticsearch 不允許 root 使用者直接執行，所以要建立新使用者，在 root 使用者中建立新使用者。
useradd es #新增 es 使用者
passwd es #為 es 使用者設定密碼
userdel -r es #如果錯了，可以刪除再加
chown -R es:es /opt/module/es #資料夾所有者
四、修改配置檔案
修改/opt/module/es/config/elasticsearch.yml 檔案，分發檔案。
# 加入如下配置
#叢集名稱
cluster.name: cluster-es
#節點名稱， 每個節點的名稱不能重複
node.name: node-1
#ip 地址， 每個節點的地址不能重複
network.host: linux1
#是不是有資格主節點
node.master: true
node.data: true
http.port: 9200
# head 外掛需要這開啟這兩個配置
http.cors.allow-origin: "*"
http.cors.enabled: true
http.max_content_length: 200mb
#es7.x 之後新增的配置，初始化一個新的叢集時需要此配置來選舉 master
cluster.initial_master_nodes: ["node-1"]
#es7.x 之後新增的配置，節點發現
discovery.seed_hosts: ["linux1:9300","linux2:9300","linux3:9300"]
gateway.recover_after_nodes: 2
network.tcp.keep_alive: true
network.tcp.no_delay: true
transport.tcp.compress: true
#叢集內同時啟動的資料任務個數，預設是 2 個
cluster.routing.allocation.cluster_concurrent_rebalance: 16
#新增或刪除節點及負載均衡時併發恢復的執行緒個數，預設 4 個
cluster.routing.allocation.node_concurrent_recoveries: 16
#初始化資料恢復時，併發恢復執行緒的個數，預設 4 個
cluster.routing.allocation.node_initial_primaries_recoveries: 16
修改/etc/security/limits.conf ，分發檔案
# 在檔案末尾中增加下面內容
es soft nofile 65536
es hard nofile 65536
修改/etc/security/limits.d/20-nproc.conf，分發檔案
# 在檔案末尾中增加下面內容
es soft nofile 65536
es hard nofile 65536
\* hard nproc 4096
\# 注： * 帶表 Linux 所有使用者名稱稱
修改/etc/sysctl.conf
# 在檔案中增加下面內容
vm.max_map_count=655360
重新載入
sysctl -p
啟動軟體
分別在不同節點上啟動 ES 軟體
cd /opt/module/es-cluster
#啟動
bin/elasticsearch
#後臺啟動
bin/elasticsearch -d