技術標籤：Elasticsearch實戰elasticsearch

Elasticsearch實戰

一個field的設定是不能被修改的，如果要修改一個Field，那麼應該重新按照新的mapping，建立一個index，然後將資料批量查詢出來，重新用bulk api寫入index中

批量查詢的時候，建議採用scroll api，並且採用多執行緒併發的方式來reindex資料，每次scoll就查詢指定日期的一段資料，交給一個執行緒即可

step1

首先我們建兩個文件，然後看下mapping

PUT /vehicle/car/1
{
  "name":"benz",
  "create": "2020-01-01"
}

PUT /vehicle/car/2
{
  "name":"bmw",
  "create": "2019-01-01"
}


GET /vehicle/_mapping
 

{
"vehicle" : {
"mappings" : {
"properties" : {
"create" : {
"type" : "date"
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",

"ignore_above" : 256
}
}
}
}
}
}
}

step2

一開始，依靠dynamic mapping，插入資料，但是不小心有些資料是2020-01-01這種日期格式的，所以title這種field被自動對映為了date型別，實際上它應該是string型別的，當後期向索引中加入string型別的title值的時候，就會報錯，如果此時想修改title的型別，是不可能的

PUT /vehicle/car/3
{
  "name":"Range Rover",
  "create":"china"
}
 

{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [create] of type [date] in document with id '3'. Preview of field's value: 'china'"
}
],
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [create] of type [date] in document with id '3'. Preview of field's value: 'china'",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "failed to parse date field [china] with format [strict_date_optional_time||epoch_millis]",
"caused_by" : {
"type" : "date_time_parse_exception",
"reason" : "Failed to parse with all enclosed parsers"
}
}
},
"status" : 400
}

PUT /vehicle/_mapping/car?include_type_name=true
{
  "properties":{
    "create":{
      "type":"text"
    }
  }
}

{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "mapper [create] cannot be changed from type [date] to [text]"
}
],
"type" : "illegal_argument_exception",
"reason" : "mapper [create] cannot be changed from type [date] to [text]"
},
"status" : 400
}

step3

此時，唯一的辦法，就是進行reindex，也就是說，重新建立一個索引，將舊索引的資料查詢出來，再匯入新索引, 給java應用一個別名，這個別名是指向舊索引的，java應用先用著，java應用先用old_vehicle alias來操作，此時實際指向的是舊的vehicle

PUT /vehicle/_alias/alias_vehicle

{
"acknowledged" : true
}

查詢使用alias

GET /alias_vehicle/car/_search

#! Deprecation: [types removal] Specifying types in search requests is deprecated.
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "vehicle",
"_type" : "car",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "benz",
"create" : "2020-01-01"
}
},
{
"_index" : "vehicle",
"_type" : "car",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"name" : "bmw",
"create" : "2019-01-01"
}
}
]
}
}

step4

建立新的索引,並通過scrool分頁查詢原資料

PUT new_vehicle?include_type_name=true
{
  "mappings": {
    "car":{
      "properties":{
        "name":{
          "type":"text"
        },
        "create":{
          "type":"text"
        }
      }
    }
  }
}

#! Deprecation: [types removal] Using include_type_name in create index requests is deprecated. The parameter will be removed in the next major version.
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "new_vehicle"
}

scroll深度分頁查詢

GET /vehicle/car/_search?scroll=1m
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "_doc": {
        "order": "asc"
      }
    }
  ], 
  "size": 1
}


GET _search/scroll
{
  "scroll":"1m",
  "scroll_id":"FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFks2Qi1fV0ptUkJtNlhETTVSU2piSlEAAAAAAAAE5BZSemFEbHd6SlRaQ3E0R29YbXgtcVNR"
}

step5

採用bulk api將scoll查出來的一批資料，批量寫入新索引

POST /new_vehicle/car/_bulk
{"index":{"_id":1}}
{"name":"benz","create":"2020-01-01"}
{"index":{"_id":2}}
{"name":"bmw11","create":"2019-01-01"}

step6

將alias_vehicle alias切換到new_vehicle上去，java應用會直接通過index別名使用新的索引中的資料，java應用程式不需要停機，零提交，高可用

POST /_aliases
{
  "actions": [
    {
      "remove": {
        "index": "vehicle",
        "alias": "alias_vehicle"
      }
    },
    {
      "add": {
        "index": "new_vehicle",
        "alias": "alias_vehicle"
      }
    }
  ]
}

#! Deprecation: [types removal] Specifying types in search requests is deprecated.
{
"took" : 568,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "new_vehicle",
"_type" : "car",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "benz",
"create" : "2020-01-01"
}
},
{
"_index" : "new_vehicle",
"_type" : "car",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"name" : "bmw11",
"create" : "2019-01-01"
}
}
]
}
}

至此已經顯示了索引的重建以及資料遷移

歡迎關注公眾號《小馬JAVA》