筆記:elasticsearch在專案中的使用需求
阿新 • • 發佈:2018-12-22
(摘抄自公司大牛的筆記,自己總結一下)
類比mysql,我們需要=、>、>=、<、<= 、or、and、in、like、count、sum、group by、order by、limit
在高版本的ES裡面使用了boolquery替換了filter
1、等於號=
QueryBuilders.termQuery("isDel",1));
//termQuery相當於=,相當於isDel=1
2、大於小於號
QueryBuilders.rangeQuery("overdueStartTime").lte(startDate.getTime())
//rangeQuery
//lte <=
//lt <
//gte >=
//gt >
3、or、and
boolQueryBuilder.must(QueryBuilders.termQuery("isDel",req.getIs_del()));
//must相當於and
caseStatusQuery.should(QueryBuilders.termQuery("caseStatus",s));
//should相當於or
4、in
QueryBuilders.termsQuery("clientCompanyId",req.getClientCompanyIdList())
//termQuery相當於in ,可以傳遞list
5、like
QueryBuilders.wildcardQuery("phone1", "*" + req.getTelephone() + "*")
//wildcard 正則匹配
QueryBuilders.matchPhraseQuery("debtorName", req.getDebtor_name())
//matchPhrase 不會分詞,直接匹配有完整相關短語的記錄
6、sum&&count
AggregationBuilder debtTotalSum = AggregationBuilders.sum(DEBT_TOTAL_SUM_KEY).field ("debtTotal");
AggregationBuilder debtTotalCount = AggregationBuilders.count(DEBT_TOTAL_COUNT_KEY).field("debtTotal");
7、order by and limit
SearchResponse searchResponse = client.prepareSearch(ElasticSearchUtil.getIndexName())
.setTypes(ElasticSearchUtil.TYPE_NAME)
.setQuery(boolQueryBuilder)
//指定查詢欄位
.addStoredField("id")
.addSort(order, sortOrder)
//分頁
.setFrom((currentPage - 1) * limit).setSize(limit)
.execute()
.actionGet();
8、批量插入
public void batchInsert(List<CaseTable> list,Client client) {
LOGGER.info("---------案件批量插入索引開始--------");
BulkRequestBuilder bulkRequest = client.prepareBulk();
long start = System.currentTimeMillis();
BulkRequestBuilder delRequest = client.prepareBulk();
for (CaseTable caseTable :list){
IndexRequestBuilder ir = client.prepareIndex()
.setIndex(ElasticSearchUtil.INDEX_NAME_CASE)
.setType(ElasticSearchUtil.TYPE_NAME_CASE)
.setId(caseTable.getId().toString())
.setSource(JSON.toJSONString(caseTable));
bulkRequest.add(ir);
start = System.currentTimeMillis();
BulkResponse bulkResponse = bulkRequest.execute().actionGet();
LOGGER.info("案件批量插入迴圈所用時間 search time:{}ms", System.currentTimeMillis() - start);
if (!bulkResponse.hasFailures()) {
LOGGER.info("----------案件批量插入成功一輪------------");
}
}
9、踩過的坑
- 當查詢條件多餘1024個時,會報錯
- 當分頁的記錄大於10000時,會報錯
- 當使用浮點數進行sum時會有精度丟失
- 當group by欄位是非數字時,會報錯
- 如果terms(in)條件比較多的話,使用must則非常慢
- 大數值的比較不生效,如1505446228090(13位)居然小於905446228090(12位)
- ES的資料存放目錄被設定到了系統盤,導致磁碟被沾滿