ElasticSearch實戰系列三: ElasticSearch的JAVA API使用教程
前言
在上一篇中介紹了ElasticSearch實戰系列二: ElasticSearch的DSL語句使用教程---圖文詳解,本篇文章就來講解下 ElasticSearch 6.x官方Java API的使用。
ElasticSearch JAVA API
目前市面上有幾種常見的ElasticSearch Java API架包,JestClient、SpringBoot整合的SpringData、Spring整合的ElasticsearchTemplate、Elasticsearch Bboss等一些開源架包,上述這些第三方整合的架包中,基本已經支援日常的使用,除了支援的ES版本會低一些而已。
本文介紹的是ElasticSearch官方的Java High Level REST Client
Java High Level REST Client
是ElasticSearch官方目前推薦使用的,適用於6.x以上的版本,要求JDK在1.8以上,可以很好的在大版本中進行相容,並且該架包自身也包含Java Low Level REST Client
中的方法,可以應對一些特需的情況進行特殊的處理, 它對於一些常用的方法封裝Restful風格,可以直接對應操作名呼叫使用即可,支援同步和非同步(Async)呼叫。
這裡我們的使用也可以直接對應上一篇文章中的DSL語句使用,這樣的話可以非常方便的對照和學習使用。
在對下述進行操作時,我們先來看下Elasticsearch Java High Level REST Client
RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost(elasticIp, elasticPort)));
是不是很簡單呢,關閉也很簡單,client不為空直接close即可!
一、新增資料
ElasticSearch可以直接新增資料,只要你指定了index(索引庫名稱)和type(型別)即可。在新增的時候你可以自己指定主鍵ID,也可以不指定,由 ElasticSearch自身生成。Elasticsearch Java High Level REST Client
新增資料程式碼示例一,通過jsonString進行建立:
String index = "test1";
String type = "_doc";
// 唯一編號
String id = "1";
IndexRequest request = new IndexRequest(index, type, id);
String jsonString = "{" + "\"uid\":\"1234\","+ "\"phone\":\"12345678909\","+ "\"msgcode\":\"1\"," + "\"sendtime\":\"2019-03-14 01:57:04\","
+ "\"message\":\"xuwujing study Elasticsearch\"" + "}";
request.source(jsonString, XContentType.JSON);
IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);
新增資料程式碼示例二,通過map建立,會自動轉換成json的資料:
String index = "test1";
String type = "_doc";
// 唯一編號
String id = "1";
IndexRequest request = new IndexRequest(index, type, id);
Map<String, Object> jsonMap = new HashMap<>();
jsonMap.put("uid", 1234);
jsonMap.put("phone", 12345678909L);
jsonMap.put("msgcode", 1);
jsonMap.put("sendtime", "2019-03-14 01:57:04");
jsonMap.put("message", "xuwujing study Elasticsearch");
request.source(jsonMap);
IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);
新增資料程式碼示例三,通過XContentBuilder物件進行建立:
String index = "test1";
String type = "_doc";
// 唯一編號
String id = "1";
IndexRequest request = new IndexRequest(index, type, id);
XContentBuilder builder = XContentFactory.jsonBuilder();
builder.startObject();
{
builder.field("uid", 1234);
builder.field("phone", 12345678909L);
builder.field("msgcode", 1);
builder.timeField("sendtime", "2019-03-14 01:57:04");
builder.field("message", "xuwujing study Elasticsearch");
}
builder.endObject();
request.source(builder);
IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);
上述三種方法中,個人推薦第二種,比較容易理解和使用。
二、建立索引庫
在上述示例中,我們通過直接通過建立資料從而建立了索引庫,但是沒有建立索引庫而通過ES自身生成的這種並不友好,因為它會使用預設的配置,欄位結構都是text(text的資料會分詞,在儲存的時候也會額外的佔用空間),分片和索引副本採用預設值,預設是5和1,ES的分片數在建立之後就不能修改,除非reindex,所以這裡我們還是指定資料模板進行建立。
使用JAVA API 建立索引庫的方法和上述中新增資料的一樣,有三種方式,不過這裡就只介紹一種。
新增索引庫的程式碼示例:
private static void createIndex() throws IOException {
String type = "_doc";
String index = "test1";
// setting 的值
Map<String, Object> setmapping = new HashMap<>();
// 分割槽數、副本數、快取重新整理時間
setmapping.put("number_of_shards", 10);
setmapping.put("number_of_replicas", 1);
setmapping.put("refresh_interval", "5s");
Map<String, Object> keyword = new HashMap<>();
//設定型別
keyword.put("type", "keyword");
Map<String, Object> lon = new HashMap<>();
//設定型別
lon.put("type", "long");
Map<String, Object> date = new HashMap<>();
//設定型別
date.put("type", "date");
date.put("format", "yyyy-MM-dd HH:mm:ss");
Map<String, Object> jsonMap2 = new HashMap<>();
Map<String, Object> properties = new HashMap<>();
//設定欄位message資訊
properties.put("uid", lon);
properties.put("phone", lon);
properties.put("msgcode", lon);
properties.put("message", keyword);
properties.put("sendtime", date);
Map<String, Object> mapping = new HashMap<>();
mapping.put("properties", properties);
jsonMap2.put(type, mapping);
GetIndexRequest getRequest = new GetIndexRequest();
getRequest.indices(index);
getRequest.local(false);
getRequest.humanReadable(true);
boolean exists2 = client.indices().exists(getRequest, RequestOptions.DEFAULT);
//如果存在就不建立了
if(exists2) {
System.out.println(index+"索引庫已經存在!");
return;
}
// 開始建立庫
CreateIndexRequest request = new CreateIndexRequest(index);
try {
// 載入資料型別
request.settings(setmapping);
//設定mapping引數
request.mapping(type, jsonMap2);
//設定別名
request.alias(new Alias("pancm_alias"));
CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);
boolean falg = createIndexResponse.isAcknowledged();
if(falg){
System.out.println("建立索引庫:"+index+"成功!" );
}
} catch (IOException e) {
e.printStackTrace();
}
}
注:建立索引庫的時候,一定要先判斷索引庫是否存在!!!
這裡建立索引庫的時候順便也指定了別名(alias),這個別名是一個好東西,使用恰當可以提升查詢效能,這裡我們留著下次在講。
三、修改資料
ES提供修改API的時候,有兩種方式,一種是直接修改,但是若資料不存在會丟擲異常,另一種則是存在更新,不存著就插入。相比第一種,第二種會更加好用一些,不過在寫入速度上是不如第一種的。
ES修改的程式碼示例:
private static void update() throws IOException {
String type = "_doc";
String index = "test1";
// 唯一編號
String id = "1";
UpdateRequest upateRequest = new UpdateRequest();
upateRequest.id(id);
upateRequest.index(index);
upateRequest.type(type);
// 依舊可以使用Map這種集合作為更新條件
Map<String, Object> jsonMap = new HashMap<>();
jsonMap.put("uid", 12345);
jsonMap.put("phone", 123456789019L);
jsonMap.put("msgcode", 2);
jsonMap.put("sendtime", "2019-03-14 01:57:04");
jsonMap.put("message", "xuwujing study Elasticsearch");
upateRequest.doc(jsonMap);
// upsert 方法表示如果資料不存在,那麼就新增一條
upateRequest.docAsUpsert(true);
client.update(upateRequest, RequestOptions.DEFAULT);
System.out.println("更新成功!");
}
注:upsert 方法表示如果資料不存在,那麼就新增一條,預設是false。
四、刪除資料
根據上述的幾個操作,想必不用多說,已經知道了是DELETE方法了,那我們就直接開始吧。
ES根據ID刪除程式碼示例:
private static void delete() throws IOException {
String type = "_doc";
String index = "test1";
// 唯一編號
String id = "1";
DeleteRequest deleteRequest = new DeleteRequest();
deleteRequest.id(id);
deleteRequest.index(index);
deleteRequest.type(type);
// 設定超時時間
deleteRequest.timeout(TimeValue.timeValueMinutes(2));
// 設定重新整理策略"wait_for"
// 保持此請求開啟,直到重新整理使此請求的內容可以搜尋為止。此重新整理策略與高索引和搜尋吞吐量相容,但它會導致請求等待響應,直到發生重新整理
deleteRequest.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
// 同步刪除
DeleteResponse deleteResponse = client.delete(deleteRequest, RequestOptions.DEFAULT);
}
ES根據條件進行刪除:
private static void deleteByQuery() throws IOException {
String type = "_doc";
String index = "test1";
DeleteByQueryRequest request = new DeleteByQueryRequest(index,type);
// 設定查詢條件
request.setQuery(QueryBuilders.termsQuery("uid",1234));
// 同步執行
BulkByScrollResponse bulkResponse = client.deleteByQuery(request, RequestOptions.DEFAULT);
}
測試結果
示例圖:
查詢語句
幾個常用的查詢API這裡就簡單的介紹下用法,然後再直接給出所有的查詢語句程式碼。
查詢API
- 等值(term查詢:QueryBuilders.termQuery(name,value);
- 多值(terms)查詢:QueryBuilders.termsQuery(name,value,value2,value3...);
- 範圍(range)查詢:QueryBuilders.rangeQuery(name).gte(value).lte(value);
- 存在(exists)查詢:QueryBuilders.existsQuery(name);
- 模糊(wildcard)查詢:QueryBuilders.wildcardQuery(name,+value+);
- 組合(bool)查詢: BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
查詢所有程式碼示例
private static void allSearch() throws IOException {
SearchRequest searchRequestAll = new SearchRequest();
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
searchRequestAll.source(searchSourceBuilder);
// 同步查詢
SearchResponse searchResponseAll = client.search(searchRequestAll, RequestOptions.DEFAULT);
System.out.println("所有查詢總數:" + searchResponseAll.getHits().getTotalHits());
}
一般查詢程式碼示例
其實就是等值查詢,只不過在裡面加入了分頁、排序、超時、路由等等設定,並且在查詢結果裡面增加了一些處理。
private static void genSearch() throws IOException {
String type = "_doc";
String index = "test1";
// 查詢指定的索引庫
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.types(type);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 設定查詢條件
sourceBuilder.query(QueryBuilders.termQuery("uid", "1234"));
// 設定起止和結束
sourceBuilder.from(0);
sourceBuilder.size(5);
sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
// 設定路由
// searchRequest.routing("routing");
// 設定索引庫表示式
searchRequest.indicesOptions(IndicesOptions.lenientExpandOpen());
// 查詢選擇本地分片,預設是叢集分片
searchRequest.preference("_local");
// 排序
// 根據預設值進行降序排序
// sourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC));
// 根據欄位進行升序排序
// sourceBuilder.sort(new FieldSortBuilder("id").order(SortOrder.ASC));
// 關閉suorce查詢
// sourceBuilder.fetchSource(false);
String[] includeFields = new String[]{"title", "user", "innerObject.*"};
String[] excludeFields = new String[]{"_type"};
// 包含或排除欄位
// sourceBuilder.fetchSource(includeFields, excludeFields);
searchRequest.source(sourceBuilder);
System.out.println("普通查詢的DSL語句:"+sourceBuilder.toString());
// 同步查詢
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
// HTTP狀態程式碼、執行時間或請求是否提前終止或超時
RestStatus status = searchResponse.status();
TimeValue took = searchResponse.getTook();
Boolean terminatedEarly = searchResponse.isTerminatedEarly();
boolean timedOut = searchResponse.isTimedOut();
// 供關於受搜尋影響的切分總數的統計資訊,以及成功和失敗的切分
int totalShards = searchResponse.getTotalShards();
int successfulShards = searchResponse.getSuccessfulShards();
int failedShards = searchResponse.getFailedShards();
// 失敗的原因
for (ShardSearchFailure failure : searchResponse.getShardFailures()) {
// failures should be handled here
}
// 結果
searchResponse.getHits().forEach(hit -> {
Map<String, Object> map = hit.getSourceAsMap();
System.out.println("普通查詢的結果:" + map);
});
System.out.println("\n=================\n");
}
或查詢
其實這個或查詢也是bool查詢中的一種,這裡的查詢語句相當於SQL語句中的
SELECT * FROM test1 where (uid = 1 or uid =2) and phone = 12345678919
程式碼示例:
private static void orSearch() throws IOException {
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("test1");
searchRequest.types("_doc");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
BoolQueryBuilder boolQueryBuilder2 = new BoolQueryBuilder();
/**
* SELECT * FROM test1 where (uid = 1234 or uid =12345) and phone = 12345678909
* */
boolQueryBuilder2.should(QueryBuilders.termQuery("uid", 1234));
boolQueryBuilder2.should(QueryBuilders.termQuery("uid", 12345));
boolQueryBuilder.must(boolQueryBuilder2);
boolQueryBuilder.must(QueryBuilders.termQuery("phone", "12345678909"));
searchSourceBuilder.query(boolQueryBuilder);
System.out.println("或查詢語句:" + searchSourceBuilder.toString());
searchRequest.source(searchSourceBuilder);
// 同步查詢
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
searchResponse.getHits().forEach(documentFields -> {
System.out.println("查詢結果:" + documentFields.getSourceAsMap());
});
}
模糊查詢
相當於SQL語句中的like查詢。
private static void likeSearch() throws IOException {
String type = "_doc";
String index = "test1";
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices(index);
searchRequest.types(type);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
/**
* SELECT * FROM p_test where message like '%xu%';
* */
boolQueryBuilder.must(QueryBuilders.wildcardQuery("message", "*xu*"));
searchSourceBuilder.query(boolQueryBuilder);
System.out.println("模糊查詢語句:" + searchSourceBuilder.toString());
searchRequest.source(searchSourceBuilder);
// 同步查詢
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
searchResponse.getHits().forEach(documentFields -> {
System.out.println("模糊查詢結果:" + documentFields.getSourceAsMap());
});
System.out.println("\n=================\n");
}
多值查詢
也就是相當於SQL語句中的in查詢。
private static void inSearch() throws IOException {
String type = "_doc";
String index = "test1";
// 查詢指定的索引庫
SearchRequest searchRequest = new SearchRequest(index,type);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
/**
* SELECT * FROM p_test where uid in (1,2)
* */
// 設定查詢條件
sourceBuilder.query(QueryBuilders.termsQuery("uid", 1, 2));
searchRequest.source(sourceBuilder);
System.out.println("in查詢的DSL語句:"+sourceBuilder.toString());
// 同步查詢
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
// 結果
searchResponse.getHits().forEach(hit -> {
Map<String, Object> map = hit.getSourceAsMap();
String string = hit.getSourceAsString();
System.out.println("in查詢的Map結果:" + map);
System.out.println("in查詢的String結果:" + string);
});
System.out.println("\n=================\n");
}
存在查詢
判斷是否存在該欄位,用法和SQL語句中的exist類似。
private static void existSearch() throws IOException {
String type = "_doc";
String index = "test1";
// 查詢指定的索引庫
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.types(type);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 設定查詢條件
sourceBuilder.query(QueryBuilders.existsQuery("msgcode"));
searchRequest.source(sourceBuilder);
System.out.println("存在查詢的DSL語句:"+sourceBuilder.toString());
// 同步查詢
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
// 結果
searchResponse.getHits().forEach(hit -> {
Map<String, Object> map = hit.getSourceAsMap();
String string = hit.getSourceAsString();
System.out.println("存在查詢的Map結果:" + map);
System.out.println("存在查詢的String結果:" + string);
});
System.out.println("\n=================\n");
}
範圍查詢
和SQL語句中<>使用方法一樣,其中gt是大於,lt是小於,gte是大於等於,lte是小於等於。
private static void rangeSearch() throws IOException{
String type = "_doc";
String index = "test1";
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.types(type);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 設定查詢條件
sourceBuilder.query(QueryBuilders.rangeQuery("sendtime").gte("2019-01-01 00:00:00").lte("2019-12-31 23:59:59"));
searchRequest.source(sourceBuilder);
System.out.println("範圍查詢的DSL語句:"+sourceBuilder.toString());
// 同步查詢
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
// 結果
searchResponse.getHits().forEach(hit -> {
String string = hit.getSourceAsString();
System.out.println("範圍查詢的String結果:" + string);
});
System.out.println("\n=================\n");
}
正則查詢
ES可以使用正則進行查詢,查詢方式也非常的簡單,程式碼示例如下:
private static void regexpSearch() throws IOException{
String type = "_doc";
String index = "test1";
// 查詢指定的索引庫
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.types(type);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 設定查詢條件
sourceBuilder.query(QueryBuilders.regexpQuery("message","xu[0-9]"));
searchRequest.source(sourceBuilder);
System.out.println("正則查詢的DSL語句:"+sourceBuilder.toString());
// 同步查詢
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
// 結果
searchResponse.getHits().forEach(hit -> {
Map<String, Object> map = hit.getSourceAsMap();
String string = hit.getSourceAsString();
System.out.println("正則查詢的Map結果:" + map);
System.out.println("正則查詢的String結果:" + string);
});
System.out.println("\n=================\n");
}
查詢測試結果
所有查詢總數:6
普通查詢的DSL語句:{"from":0,"size":5,"timeout":"60s","query":{"term":{"uid":{"value":"1234","boost":1.0}}}}=================
或查詢語句:{"query":{"bool":{"must":[{"bool":{"should":[{"term":{"uid":{"value":1234,"boost":1.0}}},{"term":{"uid":{"value":12345,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}},{"term":{"phone":{"value":"12345678909","boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}}
或查詢結果:{msgcode=1, uid=12345, phone=12345678909, message=qq, sendtime=2019-03-14 01:57:04}=================
模糊查詢語句:{"query":{"bool":{"must":[{"wildcard":{"message":{"wildcard":"xu","boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}}
模糊查詢結果:{msgcode=2, uid=12345, phone=123456789019, sendtime=2019-03-14 01:57:04, message=xuwujing study Elasticsearch}
模糊查詢結果:{uid=123456, phone=12345678909, message=xu1, sendtime=2019-03-14 01:57:04}=================
存在查詢的DSL語句:{"query":{"exists":{"field":"msgcode","boost":1.0}}}
存在查詢的Map結果:{msgcode=2, uid=12345, phone=123456789019, sendtime=2019-03-14 01:57:04, message=xuwujing study Elasticsearch}
存在查詢的String結果:{"uid":12345,"phone":123456789019,"msgcode":2,"sendtime":"2019-03-14 01:57:04","message":"xuwujing study Elasticsearch"}
存在查詢的Map結果:{msgcode=1, uid=12345, phone=12345678909, message=qq, sendtime=2019-03-14 01:57:04}
存在查詢的String結果:{"uid":"12345","phone":"12345678909","message":"qq","msgcode":"1","sendtime":"2019-03-14 01:57:04"}=================
範圍查詢的DSL語句:{"query":{"range":{"sendtime":{"from":"2019-01-01 00:00:00","to":"2019-12-31 23:59:59","include_lower":true,"include_upper":true,"boost":1.0}}}}
範圍查詢的String結果:{"uid":12345,"phone":123456789019,"msgcode":2,"sendtime":"2019-03-14 01:57:04","message":"xuwujing study Elasticsearch"}
範圍查詢的String結果:{"uid":"123456","phone":"12345678909","message":"xu1","sendtime":"2019-03-14 01:57:04"}
範圍查詢的String結果:{"uid":"12345","phone":"12345678909","message":"qq","msgcode":"1","sendtime":"2019-03-14 01:57:04"}=================
正則查詢的DSL語句:{"query":{"regexp":{"message":{"value":"xu[0-9]","flags_value":65535,"max_determinized_states":10000,"boost":1.0}}}}
正則查詢的Map結果:{uid=123456, phone=12345678909, message=xu1, sendtime=2019-03-14 01:57:04}
正則查詢的String結果:{"uid":"123456","phone":"12345678909","message":"xu1","sendtime":"2019-03-14 01:57:04"}=================
組合查詢的DSL語句:{"query":{"bool":{"must":[{"term":{"uid":{"value":12345,"boost":1.0}}},{"term":{"msgcode":{"value":1,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}}
組合查詢的String結果:{"uid":"12345","phone":"12345678909","message":"qq","msgcode":"1","sendtime":"2019-03-14 01:57:04"}=================
其它
參考ES官方文件:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html
關於SpringBoot整合ElasticSearch和JestClient的使用可以檢視這篇文章:SpringBoot整合ElasticSearch實現多版本的相容
關於ElasticSearch Java API的選擇,如果ElasticSearch版本在6.x以前的話,推薦使用JestClient。如果是6.x之後並且有意升級到7.x的話,那麼直接使用ES官方的Java High Level REST Client
,因為在7.x之後將直接會捨棄Transport client
的連線方式,目前Spring和SpringBoot整合的ES就是使用該方式(不知後續是否會做調整)。
本篇文章的程式碼已收錄在本人的java-study專案中,若有興趣,歡迎star、fork和issues。
專案地址:https://github.com/xuwujing/java-study
ElasticSearch實戰系列:
ElasticSearch實戰系列一: ElasticSearch叢集+Kinaba安裝教程
ElasticSearch實戰系列二: ElasticSearch的DSL語句使用教程---圖文詳解
音樂推薦
原創不易,如果感覺不錯,希望給個推薦!您的支援是我寫作的最大動力!
版權宣告:
作者:虛無境
部落格園出處:http://www.cnblogs.com/xuwujing
CSDN出處:http://blog.csdn.net/qazwsxpcm
個人部落格出處:http://www.panchengming.