突破單機多實例Elasticsearch
默認大家都是單機單實例es,在實驗環境下想盡可能模擬各種場景。單機多實例就出來了。。。
實驗拓撲圖
01、es安裝這裏就不說了
詳情:http://www.cnblogs.com/xiaochina/p/6852677.html
02、講elasticsearch.yml配置
要做到單機上開多個實例,需要修改ES的默認配置:
node.max_local_storage_nodes: 2 #單機開啟2個實例
配置限制了單節點上可以開啟的ES存儲實例的個數
http.port: 9200
配置是elasticsearch對外提供服務的http端口配置,默認情況下ES會取用9200~9299之間的端口,如果9200被占用就會自動使用9201
在單機多實例的配置中這個配置實際是不需要修改的,但是為了更好地進行配置管理,建議根據需求修改9201 9202
transport.tcp.port: 9300
配置指定了elasticsearch集群內數據通訊使用的端口,默認情況下為9300,與上面的http.port配置類似,ES也會自動為已占用的端口選擇下一個端口號。我們可以將第一個實例的tcp傳輸端口配置為9300,第二實例配置為9301。
discovery.zen.ping.unicast.hosts 由於到了2.x版本之後,ES取消了默認的廣播模式來發現master節點,需要使用該配置來指定發現master節點。這個配置在單機雙實例的配置中需要特別註意下,因為習慣上我們配置時並未指定master節點的tcp端口,如果實例的transport.tcp.port配置為9301,那麽實例啟動後會認為discovery.zen.ping.unicast.hosts中指定的主機tcp端口也是9301,可能導致這些節點無法找到master節點。
該配置中需要指定master節點提供服務的tcp端口。
配置示例:
discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"]
-
node.name
同一主機上的兩個實例需要使用不同的node.name
-
path.data
同一主機上兩個實例需要對應不同的數據目錄
path.logs
由於默認情況下日誌用集群名稱來命名,因此同一主機兩個實例對應的日誌目錄需要分開
#禁止HTTP
http.enabled: false
配置樣例
node1-es-data
[elk@25 config]$ cat elasticsearch.yml #======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: mvpbang # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-1 node.master: false node.data: true #數據存儲 node.max_local_storage_nodes: 2 #本機最大兩個es transport.tcp.port: 9301 #節點通信端口 http.enabled: false #關閉http接口 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: false bootstrap.system_call_filter: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 172.24.0.25 # # Set a custom port for HTTP: # #http.port: 9201 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"] #候選主節點地址 # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl e nodes / 2 + 1):# discovery.zen.minimum_master_nodes: 1 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true
node2-es-data
[elk@25 config]$ cat elasticsearch.yml # ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: mvpbang # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-2 node.master: false node.data: true node.max_local_storage_nodes: 2 transport.tcp.port: 9302 http.enabled: false # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: false bootstrap.system_call_filter: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 172.24.0.25 # # Set a custom port for HTTP: # #http.port: 9202 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"] # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl e nodes / 2 + 1):# discovery.zen.minimum_master_nodes: 1 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true
node3-es-master
[elk@25 config]$ cat elasticsearch.yml # ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: mvpbang # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-3 node.master: true node.data: false node.max_local_storage_nodes: 2 transport.tcp.port: 9301 http.enabled: true # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: false bootstrap.system_call_filter: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 172.24.0.26 # # Set a custom port for HTTP: # http.port: 9201 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"] # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl e nodes / 2 + 1):# discovery.zen.minimum_master_nodes: 1 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true
node4-es-data
[elk@25 config]$ cat elasticsearch.yml # ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: mvpbang # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: node-4 node.master: false node.data: false
node.ingest: false
node.max_local_storage_nodes: 2 transport.tcp.port: 9302 http.enabled: true # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # bootstrap.memory_lock: false bootstrap.system_call_filter: false # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 172.24.0.26 # # Set a custom port for HTTP: # http.port: 9202 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["172.24.0.26:9301"] # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligibl e nodes / 2 + 1):# discovery.zen.minimum_master_nodes: 1 # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true
效果圖
註意:復制的elasticsearch文件夾下包含了data文件中示例一的節點數據,需要把示例二data文件下的文件清空
Kibana的elasticsearch.url:配置load-balancedi地址 http://172.24.0.26:9202
學習參考:http://www.tuicool.com/articles/VBVFzyi
https://www.elastic.co/guide/en/kibana/current/production.html#load-balancing #搜索請求節點負載
突破單機多實例Elasticsearch