Elasticsearch跨叢集搜尋(Cross Cluster Search)
1、簡介
Elasticsearch在5.3版本中引入了Cross Cluster Search(CCS 跨叢集搜尋)功能,用來替換掉要被廢棄的Tribe Node。類似Tribe Node,Cross Cluster Search用來實現跨叢集的資料搜尋。
2、配置Cross Cluster Search
假設我們有2個ES叢集:
Node | Address | Port | Transport Port | Cluster |
elasticsearch01 | 127.0.0.1 | 9201 | 9301 | America |
elasticsearch02 | 127.0.0.1 | 9202 | 9302 | America |
elasticsearch03 | 127.0.0.1 | 9203 | 9303 | Europe |
elasticsearch04 | 127.0.0.1 | 9204 | 9304 | Europe |
有2種方式可以用來配置CCS:
1)配置elasticsearch.yml
search: remote: america: seeds: 127.0.0.1:9301 seeds: 127.0.0.1:9302 europe: seeds: 127.0.0.1:9303 seeds: 127.0.0.1:9304
注意:以上方式,在配置的時候,需要remote cluster處在執行狀態。比如在配置“america”的叢集的時候,需要“europe”叢集處在執行狀態,否則節點無法啟動成功。
curl -XPUT -H'Content-Type: application/json' localhost:9201/_cluster/settings -d ' { "persistent": { "search.remote": { "america": { "skip_unavailable": "true", "seeds": ["127.0.0.1:9301","127.0.0.1:9302"] }, "europe": { "skip_unavailable": "true", "seeds": ["127.0.0.1:9303","127.0.0.1:9304"] } } } }'
推薦使用API方式,可以方便的修改remote cluster的seeds和其他配置。
3、驗證Cross Cluster Search
1)使用_remote/info檢視CCS連線狀態:
[[email protected] elasticsearch01]# curl -XGET -H 'Content-Type: application/json' localhost:9201/_remote/info?pretty
{
"america" : {
"seeds" : [
"127.0.0.1:9301",
"127.0.0.1:9302"
],
"http_addresses" : [
"127.0.0.1:9201",
"127.0.0.1:9202"
],
"connected" : true,
"num_nodes_connected" : 2,
"max_connections_per_cluster" : 3,
"initial_connect_timeout" : "30s"
},
"europe" : {
"seeds" : [
"127.0.0.1:9303",
"127.0.0.1:9304"
],
"http_addresses" : [
"127.0.0.1:9203",
"127.0.0.1:9204"
],
"connected" : true,
"num_nodes_connected" : 2,
"max_connections_per_cluster" : 3,
"initial_connect_timeout" : "30s"
}
}
2)使用跨叢集搜尋:
同時查詢2個叢集的資料:
GET /cluster_name:index,cluster_name:index/_search
GET */index/_search
java API 示例:
//查詢所有叢集,以appIndex-開頭的資料 SearchRequest searchRequest = Requests.searchRequest("*:appIndex-*"); SearchResponse response = es.getClient().search(searchRequest).get();
4、Disable Cross Cluster Search
使用API設定:
curl -XPUT -H'Content-Type: application/json' localhost:9201/_cluster/settings -d '
{
"persistent": {
"search.remote": {
"america": {
"skip_unavailable": null,
"seeds": null
},
"europe": {
"skip_unavailable": null,
"seeds": null
}
}
}
}'
5、CCS的配置
search.remote.${cluster_alias}.skip_unavailable:查詢的時候skip不可達的叢集,預設false,推薦設定為true
search.remote.connect:預設true,即任何node都作為一個cross-cluster client去連線remote cluster,跨叢集搜尋的請求必須發給cross-cluster client。
search.remote.node.attr:設定remote node的屬性,比如search.remote.node.attr:gateway這樣設定,只有node.attr.gateway: true的node才會被其他的cross-cluster client連線用來做CCS查詢。
參考:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-cross-cluster-search.html
http://kelonsoftware.com/elasticsearch-cross-cluster-search/