1. 程式人生 > >elasticsearch elk最全java api 搜尋 聚合、巢狀查詢

elasticsearch elk最全java api 搜尋 聚合、巢狀查詢

一、 一般查詢... 2

(一) matchAllQuery(client). 2

(二) matchQuery(client);3

(三) multiMatchQuery(client);3

(四) wildcardQuery()模糊查詢... 3

(五) commonTermQuery(client);3

(六) termQuery(client);4

(七) testPrefixQuery字首... 4

(八) rangeQuery(client); 範圍查詢... 4

1、 兩種寫法... 5

(九) nested query. 5

(十) 其他查詢... 6

二、 聚合查詢AggsQueryTest7

(一) avgQuery(client);7

(二) minQuery(client);8

(三) maxQuery(client). 8

(四) valueCountQuery(client); //統計個數... 8

值計算聚合... 8

(五) extendedStatsQuery(client);//統計聚合(一堆). 8

(六) percentileQuery(client). 9

(七) percentileRankQuery(client);//百分比... 9

(八) rangeQuery(client)//範圍... 9

(九) histogramQuery(client);//柱狀圖聚合... 10

(十) dateHistogramQuery(client);// 按日期間隔分組... 10

(十一) 獲取聚合裡面的結果... 10

(十二) 巢狀的聚合... 10

(十三) 反轉巢狀... 10

三、 二級分組的例子:... 10

四、 巢狀查詢... 11

(一) constantScoreQuery(client);11

(二) booQuery

(client)(最常用)... 12

1、 經典案例... 12

(三) disMaxQuery(client);13

五、 本案例資料匯入... 14

 

 

 

 

 

 

 

 

一、一般查詢

 

(一)matchAllQuery(client)

matchAllQuery()方法用來匹配全部文件

 

         public static void matchAllQuery(Client client ) {

                  SearchResponse res = null;

                  QueryBuilder qb = QueryBuilders.matchAllQuery();

                 

                  res = client.prepareSearch("search_test")

                                   .setTypes("article")

                                   .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)

                                   .setQuery(qb)

                                   .setFrom(0)

                                   .setSize(10)

                                   .execute().actionGet();

                  for (SearchHit hit: res.getHits().getHits()){

                          System.out.println(hit.getSourceAsString());

                  }

 

for有選擇的列印

1.  for (SearchHit searchHit : searchHits) {  

2.              String name = (String) searchHit.getSource().get("name");  

3.              String birth = (String) searchHit.getSource().get("birth");  

4.              String interest = (String) searchHit.getSource().get("interest");  

5.              System.out.println("-------------" + (++i) + "------------");  

6.              System.out.println(name);  

7.              System.out.println(birth);  

8.              System.out.println(interest);  

9.          }  

 

 

(二)matchQuery(client);

不能寫為matchQuery("name", "to*")

matchQuery("filedname","value")匹配單個欄位,匹配欄位名為filedname,值為value的文件

 

         QueryBuilder qb = QueryBuilders.matchQuery("title", "article");

 

(三)multiMatchQuery(client);

 

多個欄位匹配某一個值

 

1.  QueryBuilder queryBuilder = QueryBuilders.multiMatchQuery("music",  

2.             "name", "interest");//搜尋name中或interest中包含有music的文件(必須與music一致) 

 

 

(四)wildcardQuery()模糊查詢

模糊查詢,?匹配單個字元,*匹配多個字元

[java] view plain copy

1.  WildcardQueryBuilder queryBuilder = QueryBuilders.wildcardQuery("name",  

2.             "*jack*");//搜尋名字中含有jack文件(name中只要包含jack即可)

 

 

(五)commonTermQuery(client);

一種略高階的查詢,充分考慮了stop-word的低優先順序,提高了查詢精確性。

將terms分為了兩種:more-importent(low-frequency) and less important(high-frequency)。less-important比如stop-words,eg:the and。

 

QueryBuilder qb = QueryBuilders

                                   .commonTermsQuery("title","article");

 

 

(六)termQuery(client);

 

* termQuery("key", obj) 完全匹配
* termsQuery("key", obj1, obj2..)   一次匹配多個值

 

 

 

    QueryBuilder qb =QueryBuilders

                .termQuery("title","article");

       

//      QueryBuilder qb = QueryBuilders

//              .termsQuery("title","article","relevence");

 

 

(七)testPrefixQuery字首

參考網址:https://www.cnblogs.com/wenbronk/p/6432990.html

 

/**
     * 字首查詢
     */
    @Test
    public void testPrefixQuery() {
        QueryBuilder queryBuilder = QueryBuilders.matchQuery("user", "kimchy");
        searchFunction(queryBuilder);
    }

 

 

(八)rangeQuery(client);範圍查詢

 

// 閉區間 QueryBuilderquery = QueryBuilders.rangeQuery("age").from(10).to(20); // 開區間 QueryBuilder query = QueryBuilders.rangeQuery("age").gt(10).lt(20);


 

1、兩種寫法

QueryBuilder qb = QueryBuilders

                                   .rangeQuery("like")

                                    .gte(5)

                                   .lt(7);

                 

                 

//               QueryBuilderqb = QueryBuilders

//                                 .rangeQuery("like")

//                                 .from(5)

//                                  .to(7)

//                                  .includeLower(true)// 包含上屆

 

//                                  .includeUpper(false);// 包含下屆

 

(九)nested query

在關係查詢中,存在一對多和多對一的關係。因為就會出現兩種查詢情況。

在解釋查詢關係之前,需要理解一下Relationship Name,如文件中contact和account的關係  ,一個Account會有多個contact,一個Contact也會有多個Account,但是最終歸結的關係為Account對contact的關係為一對多。也就是說 在contact上儲存有對account'的引用,這個引用的名稱就是RelationshipName(區別於field name),類似於外來鍵的名稱。

下面介紹兩種查詢

1、多對一的查詢。

     salesforce 中特有的__r模式,直接關聯到parent上,如contact上存有對account的引用,那麼我可以直接關聯出account上的相關欄位。

    

[sql] view plain copy

1. select id,name ,account.name,account.id from contact  


2、一對多的查詢

     嵌入式查詢(nestedquery),這種方式適合在父的一端查詢相關子的記錄。如:我想查詢到負責這個account的全部contact。

[sql] view plain copy

1. select id,name,(select id,name from contacts)  

2.  from account  

查詢結果如圖:

 

這樣就會關聯出所以的contact資料,contact部分的展示形式json串。注意contacts不是物件名稱,是Relationshipname

 

 

 

(十)其他查詢

QueryBuilder qb =QueryBuilders.existsQuery("str");

        //QueryBuilder qb =QueryBuilders.prefixQuery("name", "prefix");

 

        //QueryBuilder qb =QueryBuilders.regexpQuery("user", "k.*y");

           正則表示式

 

/**
     * 模糊查詢
     * 不能用萬用字元, 不知道幹啥用
     */

 

//QueryBuilder qb = QueryBuilders.fuzzyQuery("name", "kimzhy");

 

       

        //QueryBuilder qb =QueryBuilders.typeQuery("my_type");

 

 

 

/**
     * 只查詢一個id的
     * QueryBuilders.idsQuery(String...type).ids(Collection<String> ids)
     */

 

        //QueryBuilder qb = QueryBuilders.idsQuery("my_type","type2").addIds("1","2","5");

 

 

 

  

 

 

 

二、聚合查詢AggsQueryTest

(一)avgQuery(client);

 

         publicstatic void avgQuery(Client client ) {

                  SearchResponseres = null;

                  AvgBuilderagg = AggregationBuilders

                                   .avg("avg_num")

                                   .field("like");

                 

                  res= client.prepareSearch("search_test")

                                   .setTypes("article")

                                   .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)

                                   .addAggregation(agg)

                                   .setFrom(0)

                                   .setSize(10)

                                   .execute().actionGet();

                  System.out.println(res);

                 

                  //on shutdown

                  client.close();

         }

 

(二)minQuery(client);

         MinBuilderagg = AggregationBuilders

                                   .min("min_num")

                                   .field("like");

(三)maxQuery(client)

                  MaxBuilderagg = AggregationBuilders

                                   .max("max_num")

                                   .field("like");

(四)valueCountQuery(client); //統計個數

值計算聚合

 

                  SearchResponseres = null;

                  ExtendedStatsBuilderagg = AggregationBuilders

                                   .extendedStats("extended_stats_num")

                                   .field("like");

                 

(五)extendedStatsQuery(client);//統計聚合(一堆)

返回聚合分析後所有指標,比Stats多三個統計結果:平方和、方差、標準差

1

2

3

4

5

{

    "aggs" : {

    "grades_stats" : { "extended_stats" : { "field" : "grade" } }

    }

}

 

         ExtendedStatsBuilder agg =AggregationBuilders.extendedStats("extended_stats_num").field("like");

 

(六)percentileQuery(client)

 

                  PercentilesBuilderagg = AggregationBuilders

                                   .percentiles("percentile_num")

                                   .field("like")

                                   .percentiles(95,99,99.9);

 

 

(七)percentileRankQuery(client);//百分比

 

                  PercentileRanksBuilderagg = AggregationBuilders

                                   .percentileRanks("percentile_rank_num")

                                   .field("like")

                                   .percentiles(3,5);

 

(八)   rangeQuery(client)//範圍

 

AggregationBuilder agg =

                          AggregationBuilders

                                  .range("agg")

                                                    

                                  .field("like")

                                  .addUnboundedTo(3)

                                  .addRange(3, 5)

                                  .addUnboundedFrom(5); 

 

(九)histogramQuery(client);//柱狀圖聚合

 

(十)dateHistogramQuery(client);// 按日期間隔分組

(十一)獲取聚合裡面的結果

TopHitsBuilder thb=  AggregationBuilders.topHits("top_result");

(十二)巢狀的聚合

NestedBuilder nb= AggregationBuilders.nested("negsted_path").path("quests");

(十三)反轉巢狀

AggregationBuilders.reverseNested("res_negsted").path("kps ");

 

 

三、二級分組的例子:

上面這些基本就是常用的聚合查詢了,在巢狀(nested)下面的子聚合查詢就是巢狀查詢了,除了巢狀查詢,其他的聚合查詢也可以無限級新增子查詢

舉個例子

SearchRequestBuilder search = client.prepareSearch("index").setTypes("type");
 
TermsBuilder one=  AggregationBuilders.terms("group_name").field("name");
TermsBuilder two=  AggregationBuilders.terms("group_age").field("age");
one.subAggregation(two)
search.addAggregation(one);
 
 
        Terms terms= search.get().getAggregations().get("group_name");
                 for(Terms.Bucket name_buk:terms.getBuckets()){
                         //一級分組的內容
                         Terms terms_age= name_buk.getAggregations().get("group_age");
                         for(Terms.Bucket age_buk:terms_age.getBuckets()){
                                 //二級分組的內容  
                                  System.out.println(name_buk.getKey()+"  "+age_buk.getKey()+"  "+age_buk.getDocCount());
 
                         }
}

 

 

 

四、巢狀查詢

 

(一)constantScoreQuery(client);

 

/**
     * 包裹查詢, 高於設定分數, 不計算相關性
     */
    @Test
    public void testConstantScoreQuery() {
        QueryBuilder queryBuilder = QueryBuilders.constantScoreQuery(QueryBuilders.termQuery("name", "kimchy")).boost(2.0f);
        searchFunction(queryBuilder);

 

 

 

 

(二)booQuery(client)(最常用)

 

   /**
     * 組合查詢
     * must(QueryBuilders) :   AND
     * mustNot(QueryBuilders): NOT
     * should:                  : OR
     */

 

    publicstaticvoid booQuery(Client client) {//最有用的巢狀查詢

        SearchResponse res = null;

        QueryBuilder qb =QueryBuilders.boolQuery()

                .should(QueryBuilders.termQuery("title", "02"))

//              .mustNot(QueryBuilders.termQuery("title","article"))

                .should(QueryBuilders.termQuery("title", "relevance"));

//              .filter(QueryBuilders.termQuery("title","article"));

       

        res = client.prepareSearch("search_test").setTypes("article").setSearchType(SearchType.DFS_QUERY_THEN_FETCH)

                .setQuery(qb).setFrom(0).setSize(10).execute().actionGet();

        for (SearchHit hit : res.getHits().getHits()) {

            System.out.println(hit.getSourceAsString());

       

    }

 

 

1、經典案例

 

如果需要查詢(addr = Beijing) && (sex = false) && (10 < age< 20)的doc:

    public static QueryBuilder createQuery() {

        BoolQueryBuilder query =QueryBuilders.boolQuery();

        // addr = Beijing

        query.must(new QueryStringQueryBuilder("Beijing").field("addr"));

        // sex = falese

        query.must(new QueryStringQueryBuilder("false").field("sex"));

        // age ∈ (10,20)

        query.must(new RangeQueryBuilder("age").gt(10).lt(20));

        return query;

    }

返回結果:

{"pid":168,"age":16,"sex":false,"name":"Tom","addr":"Beijing"}

{"pid":276,"age":19,"sex":false,"name":"Bill","addr":"Beijing"}

{"pid":565,"age":16,"sex":false,"name":"Brown","addr":"Beijing"}

{"pid":73,"age":13,"sex":false,"name":"David","addr":"Beijing"}



作者:唐影若凡
連結:https://www.jianshu.com/p/a3694b13bf89
來源:簡書
著作權歸作者所有。商業轉載請聯絡作者獲得授權,非商業轉載請註明出處。

 

 

 

 

 

 

(三)disMaxQuery(client);

 

/**
     * disMax查詢
     * 對子查詢的結果做union, score沿用子查詢score的最大值,
     * 廣泛用於muti-field查詢
     */
    @Test
    public void testDisMaxQuery() {
        QueryBuilder queryBuilder = QueryBuilders.disMaxQuery()
            .add(QueryBuilders.termQuery("user", "kimch"))  // 查詢條件
            .add(QueryBuilders.termQuery("message", "hello"))
            .boost(1.3f)
            .tieBreaker(0.7f);
        searchFunction(queryBuilder);
    }
   

 

 

 

 

 

 

 

 

 

 

五、本案例資料匯入

 

 

 

curl -XPUT'http://169.254.135.217:9200/search_test/' -d '{

   "settings" : {

     "index" : {

     "number_of_shards" : 3,

     "number_of_replicas" : 1

     }

   },

   "mappings" : {

       "article" : {

         "properties" : {

           "title" : { "type" : "string"},

           "body" : { "type" : "string"},

           "like" : { "type" : "long"},

           "publish_date" : { "type" : "date"}

         }

       }

    }

}'

 

curl -XGET'http://169.254.135.217:9200/search_test/_mapping?pretty'

curl -XGET'http://169.254.135.217:9200/search_test/_mapping/article?pretty'

 

curl -XHEAD -i'http://169.254.135.217:9200/search_test/article'

 

 

/search_test/article/1

{

 "title": "What's relevance?",

 "body": "atticle body of relevence:Term frequency/inversedocument frequency",

 "like": "1",

 "publish_date": "2016-03-24"

}

 

/search_test/article/2

{

"title": "article 02",

"body": "article 02 atticlebody of relevence:Term frequency/inverse document frequency",

"like": "2",

"publish_date":"2016-05-24"

}

 

/search_test/article/3

{

"title": "article 03",

"body": "article 03 atticlebody of relevence:Term frequency/inverse document frequency",

"like": "3",

"publish_date":"2016-07-24"

}

 

/search_test/article/4

{

"title": "article 04",

"body": "article 04 atticlebody of relevence:Term frequency/inverse document frequency",

"like": "4",

"publish_date":"2016-09-24"

}

 

/search_test/article/5

{

"title": "article 05",

"body": "article 04 atticlebody of relevence:Term frequency/inverse document frequency",

"like": "5",

"publish_date":"2016-11-24"

}

 

/search_test/article/6

{

"title": "Quick brownrabbits",

"body": "Brown rabbits arecommonly seen.",

"like": "6",

"publish_date":"2016-12-24"

}

 

/search_test/article/7

{

"title": "Keeping petshealthy",

"body": "My quick brown foxeats rabbits on a regular basis.",

"like": "7",

"publish_date":"