CentOS7 下安裝 ElasticSearch 5.x 及填坑
什麽是ElasticSearch:
- ElasticSearch是基於Apache Lucene構建的開源搜索引擎
- 采用Java編寫,提供了簡單易用的RESTFul API
- 輕松的橫向擴展,可支持PB級的結構化或非結構化的數據處理
應用場景:
- 海量數據分析引擎
- 站內搜索引擎
- 作為數據倉庫
一線公司實際應用場景:
- 英國衛報 - 實時分析公眾對文章的回應
- 維基百科、GitHub - 站內實時搜索引擎
- 百度 - 實時日誌監控平臺
單實例安裝ElasticSearch
在安裝ElasticSearch之前,首先準備好Java環境:
[root@localhost /usr/local/jdk1.8]# java -version java version "1.8.0_171" Java(TM) SE Runtime Environment (build 1.8.0_171-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode) [root@localhost /usr/local/jdk1.8]#
到ElasticSearch官網上復制ElasticSearch的下載鏈接:
https://www.elastic.co/cn/
復制ElasticSearch 5.x版本的下載地址,到命令行上進行下載:
[root@localhost ~]# cd /usr/local/src/
[root@localhost /usr/local/src]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.2.tar.gz
下載好後,解壓到相應的目錄中:
[root@localhost /usr/local/src]# ls elasticsearch-5.5.2.tar.gz [root@localhost /usr/local/src]# tar -zvxf elasticsearch-5.5.2.tar.gz -C /usr/local [root@localhost /usr/local/src]# ls /usr/local/elasticsearch-5.5.2/ bin config lib LICENSE.txt modules NOTICE.txt plugins README.textile [root@localhost /usr/local/src]#
安裝完成後,我們需要新建一個普通用戶,因為ElasticSearch不允許在root用戶下運行,所以我們需要創建獨立的用戶來運行es。創建elsearch用戶組及elsearch用戶:
[root@localhost ~]# groupadd elsearch
[root@localhost ~]# useradd elsearch -g elsearch -p elasticsearch
[root@localhost ~]# passwd elsearch # 設置密碼
更改elasticsearch安裝目錄及其內部文件的所屬用戶及組為elsearch:elsearch,如下:
[root@localhost ~]# cd /usr/local/ [root@localhost /usr/local]# chown -R elsearch:elsearch elasticsearch-5.5.2/ [root@localhost /usr/local]# ll |grep elasticsearch-5.5.2 drwxr-xr-x. 8 elsearch elsearch 143 6月 19 10:46 elasticsearch-5.5.2 [root@localhost /usr/local]#
由於在CentOS7下,elasticsearch 5.x 設置了ip後,啟動會有些問題導致啟動報錯,我們需要設置如下系統參數來解決這個問題:
[root@localhost /usr/local/elasticsearch-5.5.2]# vim /etc/security/limits.conf
* hard nofile 65536
* soft nofile 131072
* soft nproc 2048
* hard nproc 4096
[root@localhost /usr/local/elasticsearch-5.5.2]# vim /etc/security/limits.d/90-nproc.conf
* soft nproc 2048
[root@localhost /usr/local/elasticsearch-5.5.2]# vim /etc/sysctl.conf
vm.max_map_count=655360
[root@localhost /usr/local/elasticsearch-5.5.2]# sysctl -p
vm.max_map_count = 655360
[root@localhost /usr/local/elasticsearch-5.5.2]#
關於這個問題可以參考如下issues:
https://github.com/elastic/elasticsearch/issues/22899
Elasticsearch5.5.1安裝問題匯總:
https://github.com/DimonHo/DH_Note/issues/3
完成以上的配置後,我們從root用戶切換到elsearch用戶:
[root@localhost ~]# su elsearch
[elsearch@localhost root]$
在啟動es前先設置一下綁定的ip,設置為可被外部訪問的ip,以及設置bootstrap相關的參數:
[elsearch@localhost root]$ cd /usr/local/elasticsearch-5.5.2/
[elsearch@localhost elasticsearch-5.5.2]$ vim config/elasticsearch.yml
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 0.0.0.0 # 找到該項,設置為可被外部訪問的ip
[elsearch@localhost elasticsearch-5.5.2]$
啟動ElasticSearch服務的命令:
[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch
加上-d參數表示後臺啟動,一般我們都采用後臺啟動的方式:
[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch -d
ElasticSearch服務啟動成功後,可看到如下進程信息:
[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep elasticsearch
elsearch 13353 107 83.1 4697512 1550164 pts/0 Sl 11:04 0:03 /usr/local/jdk1.8/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/local/elasticsearch-5.5.2 -cp /usr/local/elasticsearch-5.5.2/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
elsearch 13372 0.0 0.0 112720 988 pts/0 S+ 11:05 0:00 grep --color=auto elasticsearch
[elsearch@localhost elasticsearch-5.5.2]$
查看ElasticSearch服務所監聽的端口:
[elsearch@localhost elasticsearch-5.5.2]$ netstat -lntp |grep java
tcp6 0 0 :::9200 :::* LISTEN 14849/java
tcp6 0 0 :::9300 :::* LISTEN 14849/java
[elsearch@localhost elasticsearch-5.5.2]$
設置防火墻規則,開放ElasticSearch服務所監聽的端口:
[elsearch@localhost elasticsearch-5.5.2]$ sudo firewall-cmd --zone=public --add-port=9200/tcp --permanent
success
[elsearch@localhost elasticsearch-5.5.2]$ sudo firewall-cmd --zone=public --add-port=9300/tcp --permanent
success
[elsearch@localhost elasticsearch-5.5.2]$ sudo firewall-cmd --reload
success
[elsearch@localhost elasticsearch-5.5.2]$
開放了相應的端口後,從瀏覽器中訪問,看看能否訪問成功,如下則是訪問成功的:
關於關閉及重啟elasticsearch服務,elasticsearch沒有直接的關閉或重啟命令,關閉elasticsearch服務只能通過kill命令來殺掉服務的進程,如下:
[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep es # 查看進程id
[elsearch@localhost elasticsearch-5.5.2]$ kill -8 193460 # 通過進程id來殺掉服務
重啟:
[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep es # 查看進程id
[elsearch@localhost elasticsearch-5.5.2]$ kill -8 193460 # 通過進程id來殺掉服務
[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch -d
也可以編寫一個服務腳本,以此來方便的啟動或關閉elasticsearch服務。
插件安裝
本小節我們來安裝es的實用插件elasticsearch-head,這個插件可以提供一個圖形化的界面,能讓我們直觀的查看節點數據以及集群健康狀態等。github地址如下:
https://github.com/mobz/elasticsearch-head
進入elasticsearch-head的github地址,復制其.zip的下載鏈接,到命令行上進行下載:
[root@localhost /usr/local/src]# vim /etc/sudoers
[root@localhost /usr/local/src]# wget https://github.com/mobz/elasticsearch-head/archive/master.zip
註:切換成root用戶
解壓下載好的.zip壓縮包:
[root@localhost /usr/local/src]# unzip master.zip
[root@localhost /usr/local/src]# mv elasticsearch-head-master/ /usr/local/
[root@localhost /usr/local/src]# ls /usr/local/elasticsearch-head-master/
Dockerfile elasticsearch-head.sublime-project grunt_fileSets.js LICENCE plugin-descriptor.properties README.textile src
Dockerfile-alpine Gruntfile.js index.html package.json proxy _site test
[root@localhost /usr/local/src]#
elasticsearch-head是一個提供圖形化的插件,其圖形化是基於node.js的,所以我們需要先準備好node.js環境,node.js官網下載地址如下:
https://nodejs.org/en/download/
在命令行中下載並解壓安裝包:
[root@localhost /usr/local/src]# wget https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-x64.tar.xz
[root@localhost /usr/local/src]# tar -Jxvf node-v8.11.3-linux-x64.tar.xz -C /usr/local/
[root@localhost /usr/local/src]# mv /usr/local/node-v8.11.3-linux-x64/ /usr/local/node-8.11.3
配置環境變量:
[root@localhost /usr/local/src]# vim /etc/profile
export NODE_HOME=/usr/local/node-8.11.3
export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$NODE_HOME/bin
[root@localhost /usr/local/src]# source /etc/profile
[root@localhost /usr/local/src]# node -v # 輸出版本號則代表配置成功
v8.11.3
[root@localhost /usr/local/src]#
進入es-head的安裝目錄,執行npm install命令安裝所需的模塊:
[root@localhost /usr/local/src]# cd /usr/local/elasticsearch-head-master/
[root@localhost /usr/local/elasticsearch-head-master]# npm install [email protected] --ignore-scripts # 因為有墻,phantomjs-prebuilt需要單獨安裝,不然會失敗
[root@localhost /usr/local/elasticsearch-head-master]# npm install
es-head使用的是grunt server作為服務器,而grunt server默認監聽的是localhost,所以我們需要修改Gruntfile.js文件,增加一段配置,不然外部是無法訪問的,如下:
[root@localhost /usr/local/elasticsearch-head-master]# vim Gruntfile.js
connect: {
server: {
options: {
hostname: ‘0.0.0.0‘, # 增加這段
port: 9100,
base: ‘.‘,
keepalive: true
}
}
}
[root@localhost /usr/local/elasticsearch-head-master]#
設置防火墻規則,開放9100端口:
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --zone=public --add-port=9100/tcp --permanent
success
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --reload
success
[root@localhost /usr/local/elasticsearch-head-master]#
es與es-head是兩個獨立的進程,當es-head訪問es服務時,會存在一個跨域問題。所以我們需要修改es的配置文件,增加一些配置項來解決這個問題,如下:
[root@localhost /usr/local/elasticsearch-head-master]# cd ../elasticsearch-5.5.2/config/
[root@localhost /usr/local/elasticsearch-5.5.2/config]# vim elasticsearch.yml # 文件末尾加上如下配置
http.cors.enabled: true
http.cors.allow-origin: "*"
[root@localhost /usr/local/elasticsearch-5.5.2/config]#
註:修改完配置文件後需切換到elsearch用戶中重啟es服務。
接著再次進入es-head的安裝目錄,使用如下命令,啟動es-head:
[root@localhost /usr/local/elasticsearch-head-master]# npm run start
使用瀏覽器訪問如下:
偽分布式安裝ElasticSearch
以上我們介紹了es單實例的安裝以及es-head的安裝,本小節我們來介紹如何在單機情況下安裝偽分布式的es服務。首先配置之前已安裝好的es服務的配置文件,如下:
[root@localhost ~]# cd /usr/local/elasticsearch-5.5.2/config/
[root@localhost /usr/local/elasticsearch-5.5.2/config]# vim elasticsearch.yml # 在文件末尾加入如下內容
cluster.name: es
node.name: master
node.master: true
[root@localhost /usr/local/elasticsearch-5.5.2/config]#
參數說明:
- cluster.name 集群的名稱,可自定義
- node.name 節點的名稱,可自定義
- node.master 是否指定為master角色
新建一個存放slave的目錄,拷貝es壓縮包到該目錄中並解壓,然後再拷貝多一份slave,最後還需要更改es_slave目錄的所屬用戶及用戶組為elsearch:
[root@localhost ~]# mkdir /usr/locales_slave
[root@localhost ~]# cp /usr/local/src/elasticsearch-5.5.2.tar.gz /usr/local/es_slave/
[root@localhost ~]# cd /usr/local/es_slave/
[root@localhost /usr/local/es_slave]# tar -zxvf elasticsearch-5.5.2.tar.gz
[root@localhost /usr/local/es_slave]# mv elasticsearch-5.5.2 ./es_slave1
[root@localhost /usr/local/es_slave]# cp -r ./es_slave1/ es_slave2/
[root@localhost /usr/local]# cd ../
[root@localhost /usr/local]# chown -R elsearch:elsearch es_slave/
逐個修改slave的配置文件:
[root@localhost /usr/local/es_slave]# vim es_slave1/config/elasticsearch.yml
cluster.name: es
node.name: slave1
network.host: 127.0.0.1
http.port: 8200
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
[root@localhost /usr/local/es_slave]# vim es_slave1/config/elasticsearch.yml
cluster.name: es
node.name: slave2
network.host: 127.0.0.1
http.port: 7200
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
[root@localhost /usr/local]#
開放端口:
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --zone=public --add-port=8200/tcp --permanent
success
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --zone=public --add-port=7200/tcp --permanent
success
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --reload
success
[root@localhost /usr/local/elasticsearch-head-master]#
切換到elsearch用戶中重啟master節點,並啟動其他兩個slave節點:
[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep elastic
[elsearch@localhost elasticsearch-5.5.2]$ kill -8 16410
[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch -d
[elsearch@localhost elasticsearch-5.5.2]$ cd
[elsearch@localhost root]$ sh /usr/local/es_slave/es_slave1/bin/elasticsearch -d
[elsearch@localhost root]$ sh /usr/local/es_slave/es_slave1/bin/elasticsearch -d
啟動es-head:
[root@localhost ~]# cd /usr/local/elasticsearch-head-master/
[root@localhost /usr/local/elasticsearch-head-master]# npm run start
使用瀏覽器訪問如下:
CentOS7 下安裝 ElasticSearch 5.x 及填坑