Elasticsearch 5.5 Mapping詳解
阿新 • • 發佈：2018-12-06
- 前言
- 一Field datatype欄位資料型別
- 二Meta-Fields元資料
- 三Mapping引數
- 1 analyzer
- 2 normalizer
- 3 boost
- 4 coerce
- 5 copy_to
- 6 doc_values
- 7 dynamic
- 8 enabled
- 9 fielddata
- 10 format
- 11 ignore_above
- 12 ignore_malformed
- 13 include_in_all
- 14 index
- 15 index_options
- 16 fields
- 17 norms
- 18 null_value
- 19 position_increment_gap
- 20 properties
- 21 search_analyzer
- 22 similarity
- 23 store
- 24 term_vector
- 四動態Mapping
前言
一、Field datatype(欄位資料型別)
1.1string型別
ELasticsearch 5.X之後的欄位型別不再支援string，由text或keyword取代。 如果仍使用string，會給出警告。
測試：
PUT my_index { "mappings": { "my_type": { "properties": { "title": { "type": "string" } } } } }結果：
#! Deprecation: The [string] field is deprecated, please use [text] or [keyword] instead on [title] { "acknowledged": true, "shards_acknowledged": true }1.2 text型別
text取代了string，當一個欄位是要被全文搜尋的，比如Email內容、產品描述，應該使用text型別。設定text型別以後，欄位內容會被分析，在生成倒排索引以前，字串會被分析器分成一個一個詞項。text型別的欄位不用於排序，很少用於聚合（termsAggregation除外）。
把full_name欄位設為text型別的Mapping如下：
PUT my_index { "mappings": { "my_type": { "properties": { "full_name": { "type": "text" } } } } }1.3 keyword型別
keyword型別適用於索引結構化的欄位，比如email地址、主機名、狀態碼和標籤。如果欄位需要進行過濾(比如查詢已釋出部落格中status屬性為published的文章)、排序、聚合。keyword型別的欄位只能通過精確值搜尋到。
1.4 數字型別
對於數字型別，ELasticsearch支援以下幾種：
| 型別 | 取值範圍 |
|---|---|
| long | -2^63至2^63-1 |
| integer | -2^31至2^31-1 |
| short | -32,768至32768 |
| byte | -128至127 |
| double | 64位雙精度IEEE 754浮點型別 |
| float | 32位單精度IEEE 754浮點型別 |
| half_float | 16位半精度IEEE 754浮點型別 |
| scaled_float | 縮放型別的的浮點數（比如價格只需要精確到分，price為57.34的欄位縮放因子為100，存起來就是5734） |
對於float、half_float和scaled_float,-0.0和+0.0是不同的值，使用term查詢查詢-0.0不會匹配+0.0，同樣range查詢中上邊界是-0.0不會匹配+0.0，下邊界是+0.0不會匹配-0.0。
對於數字型別的資料，選擇以上資料型別的注意事項：
- 在滿足需求的情況下，儘可能選擇範圍小的資料型別。比如，某個欄位的取值最大值不會超過100，那麼選擇byte型別即可。迄今為止吉尼斯記錄的人類的年齡的最大值為134歲，對於年齡欄位，short足矣。欄位的長度越短，索引和搜尋的效率越高。
- 優先考慮使用帶縮放因子的浮點型別。
例子：
PUT my_index { "mappings": { "my_type": { "properties": { "number_of_bytes": { "type": "integer" }, "time_in_seconds": { "type": "float" }, "price": { "type": "scaled_float", "scaling_factor": 100 } } } } }1.5 Object型別
JSON天生具有層級關係，文件會包含巢狀的物件：
PUT my_index/my_type/1 { "region": "US", "manager": { "age": 30, "name": { "first": "John", "last": "Smith" } } }上面的文件中，整體是一個JSON，JSON中包含一個manager,manager又包含一個name。最終，文件會被索引成一平的key-value對：
{ "region": "US", "manager.age": 30, "manager.name.first": "John", "manager.name.last": "Smith" }上面文件結構的Mapping如下：
PUT my_index { "mappings": { "my_type": { "properties": { "region": { "type": "keyword" }, "manager": { "properties": { "age": { "type": "integer" }, "name": { "properties": { "first": { "type": "text" }, "last": { "type": "text" } } } } } } } } }1.6 date型別
JSON中沒有日期型別，所以在ELasticsearch中，日期型別可以是以下幾種：
- 日期格式的字串：e.g. “2015-01-01” or “2015/01/01 12:10:30”.
- long型別的毫秒數( milliseconds-since-the-epoch)
- integer的秒數(seconds-since-the-epoch)
日期格式可以自定義，如果沒有自定義，預設格式如下：
"strict_date_optional_time||epoch_millis"
例子:
PUT my_index { "mappings": { "my_type": { "properties": { "date": { "type": "date" } } } } } PUT my_index/my_type/1 { "date": "2015-01-01" } PUT my_index/my_type/2 { "date": "2015-01-01T12:10:30Z" } PUT my_index/my_type/3 { "date": 1420070400001 } GET my_index/_search { "sort": { "date": "asc"} }
檢視三個日期型別：
{ "took": 0, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 3, "max_score": 1, "hits": [ { "_index": "my_index", "_type": "my_type", "_id": "2", "_score": 1, "_source": { "date": "2015-01-01T12:10:30Z" } }, { "_index": "my_index", "_type": "my_type", "_id": "1", "_score": 1, "_source": { "date": "2015-01-01" } }, { "_index": "my_index", "_type": "my_type", "_id": "3", "_score": 1, "_source": { "date": 1420070400001 } } ] } }排序結果：
{ "took": 2, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 3, "max_score": null, "hits": [ { "_index": "my_index", "_type": "my_type", "_id": "1", "_score": null, "_source": { "date": "2015-01-01" }, "sort": [ 1420070400000 ] }, { "_index": "my_index", "_type": "my_type", "_id": "3", "_score": null, "_source": { "date": 1420070400001 }, "sort": [ 1420070400001 ] }, { "_index": "my_index", "_type": "my_type", "_id": "2", "_score": null, "_source": { "date": "2015-01-01T12:10:30Z" }, "sort": [ 1420114230000 ] } ] } }1.7 Array型別
ELasticsearch沒有專用的陣列型別，預設情況下任何欄位都可以包含一個或者多個值，但是一個數組中的值要是同一種類型。例如：
- 字元陣列: [ “one”, “two” ]
- 整型陣列：[1,3]
- 巢狀陣列：[1,[2,3]],等價於[1,2,3]
- 物件陣列：[ { “name”: “Mary”, “age”: 12 }, { “name”: “John”, “age”: 10 }]
注意事項：
- 動態新增資料時，陣列的第一個值的型別決定整個陣列的型別
- 混合陣列型別是不支援的，比如：[1,”abc”]
- 陣列可以包含null值，空陣列[ ]會被當做missing field對待。
1.8 binary型別
binary型別接受base64編碼的字串，預設不儲存也不可搜尋。
PUT my_index { "mappings": { "my_type": { "properties": { "name": { "type": "text" }, "blob": { "type": "binary" } } } } } PUT my_index/my_type/1 { "name": "Some binary blob", "blob": "U29tZSBiaW5hcnkgYmxvYg==" }搜尋blog欄位：
GET my_index/_search { "query": { "match": { "blob": "test" } } } 返回結果： { "error": { "root_cause": [ { "type": "query_shard_exception", "reason": "Binary fields do not support searching", "index_uuid": "fgA7UM5XSS-56JO4F4fYug", "index": "my_index" } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": "my_index", "node": "3dQd1RRVTMiKdTckM68nPQ", "reason": { "type": "query_shard_exception", "reason": "Binary fields do not support searching", "index_uuid": "fgA7UM5XSS-56JO4F4fYug", "index": "my_index" } } ] }, "status": 400 }Base64加密、解碼工具：http://www1.tc711.com/tool/BASE64.htm
1.9 ip型別
ip型別的欄位用於儲存IPV4或者IPV6的地址。
PUT my_index { "mappings": { "my_type": { "properties": { "ip_addr": { "type": "ip" } } } } } PUT my_index/my_type/1 { "ip_addr": "192.168.1.1" } GET my_index/_search { "query": { "term": { "ip_addr": "192.168.0.0/16" } } }1.10 range型別
range型別支援以下幾種：
| 型別 | 範圍 |
|---|---|
| integer_range | -2^31至2^31-1 |
| float_range | 32-bit IEEE 754 |
| long_range | -2^63至2^63-1 |
| double_range | 64-bit IEEE 754 |
| date_range | 64位整數，毫秒計時 |
range型別的使用場景：比如前端的時間選擇表單、年齡範圍選擇表單等。
例子：
PUT range_index { "mappings": { "my_type": { "properties": { "expected_attendees": { "type": "integer_range" }, "time_frame": { "type": "date_range", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis" } } } } } PUT range_index/my_type/1 { "expected_attendees" : { "gte" : 10, "lte" : 20 }, "time_frame" : { "gte" : "2015-10-31 12:00:00", "lte" : "2015-11-01" } }上面程式碼建立了一個range_index索引，expected_attendees的人數為10到20，時間是2015-10-31 12:00:00至2015-11-01。
查詢：
POST range_index/_search { "query" : { "range" : { "time_frame" : { "gte" : "2015-08-01", "lte" : "2015-12-01", "relation" : "within" } } } }查詢結果：
{ "took": 2, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 1, "hits": [ { "_index": "range_index", "_type": "my_type", "_id": "1", "_score": 1, "_source": { "expected_attendees": { "gte": 10, "lte": 20 }, "time_frame": { "gte": "2015-10-31 12:00:00", "lte": "2015-11-01" } } } ] } }1.11 nested型別
nested巢狀型別是object中的一個特例，可以讓array型別的Object獨立索引和查詢。 使用Object型別有時會出現問題，比如文件 my_index/my_type/1的結構如下：
PUT my_index/my_type/1 { "group" : "fans", "user" : [ { "first" : "John", "last" : "Smith" }, { "first" : "Alice", "last" : "White" } ] }
user欄位會被動態新增為Object型別。
最後會被轉換為以下平整的形式：
{ "group" : "fans", "user.first" : [ "alice", "john" ], "user.last" : [ "smith", "white" ] }
user.first和user.last會被平鋪為多值欄位，Alice和White之間的關聯關係會消失。上面的文件會不正確的匹配以下查詢(雖然能搜尋到,實際上不存在Alice Smith)：
GET my_index/_search { "query": { "bool": { "must": [ { "match": { "user.first": "Alice" }}, { "match": { "user.last": "Smith" }} ] } } }使用nested欄位型別解決Object型別的不足：
PUT my_index { "mappings": { "my_type": { "properties": { "user": { "type": "nested" } } } } } PUT my_index/my_type/1 { "group" : "fans", "user" : [ { "first" : "John", "last" : "Smith" }, { "first" : "Alice", "last" : "White" } ] } GET my_index/_search { "query": { "nested": { "path": "user", "query": { "bool": { "must": [ { "match": { "user.first": "Alice" }}, { "match": { "user.last": "Smith" }} ] } } } } } GET my_index/_search { "query": { "nested": { "path": "user", "query": { "bool": { "must": [ { "match": { "user.first": "Alice" }}, { "match": { "user.last": "White" }} ] } }, "inner_hits": { "highlight": { "fields": { "user.first": {} } } } } } }1.12token_count型別
token_count用於統計詞頻：
PUT my_index { "mappings": { "my_type": { "properties": { "name": { "type": "text", "fields": { "length": { "type": "token_count", "analyzer": "standard" } } } } } } } PUT my_index/my_type/1 { "name": "John Smith" } PUT my_index/my_type/2 { "name": "Rachel Alice Williams" } GET my_index/_search { "query": { "term": { "name.length": 3 } } }1.13 geo point 型別
地理位置資訊型別用於儲存地理位置資訊的經緯度：
PUT my_index { "mappings": { "my_type": { "properties": { "location": { "type": "geo_point" } } } } } PUT my_index/my_type/1 { "text": "Geo-point as an object", "location": { "lat": 41.12, "lon": -71.34 } } PUT my_index/my_type/2 { "text": "Geo-point as a string", "location": "41.12,-71.34" } PUT my_index/my_type/3 { "text": "Geo-point as a geohash", "location": "drm3btev3e86" } PUT my_index/my_type/4 { "text": "Geo-point as an array", "location": [ -71.34, 41.12 ] } GET my_index/_search { "query": { "geo_bounding_box": { "location": { "top_left": { "lat": 42, "lon": -72 }, "bottom_right": { "lat": 40, "lon": -74 } } } } }二、Meta-Fields(元資料)
2.1 _all
_all欄位是把其它欄位拼接在一起的超級欄位，所有的欄位用空格分開，_all欄位會被解析和索引，但是不儲存。當你只想返回包含某個關鍵字的文件但是不明確地搜某個欄位的時候就需要使用_all欄位。
例子：
PUT my_index/blog/1 { "title": "Master Java", "content": "learn java", "author": "Tom" }_all欄位包含:[ “Master”, “Java”, “learn”, “Tom” ]
搜尋：
GET my_index/_search { "query": { "match": { "_all": "Java" } } }返回結果：
{ "took": 1, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 0.39063013, "hits": [ { "_index": "my_index", "_type": "blog", "_id": "1", "_score": 0.39063013, "_source": { "title": "Master Java", "content": "learn java", "author": "Tom" } } ] } }使用copy_to自定義_all欄位：
PUT myindex { "mappings": { "mytype": { "properties": {