1. 程式人生 > >CentOS7 下安裝 ElasticSearch 5.x 及填坑

CentOS7 下安裝 ElasticSearch 5.x 及填坑

用戶及用戶組 log4j rip 還需 -- 指定 png process serve

ElasticSearch簡介

什麽是ElasticSearch:

  • ElasticSearch是基於Apache Lucene構建的開源搜索引擎
  • 采用Java編寫,提供了簡單易用的RESTFul API
  • 輕松的橫向擴展,可支持PB級的結構化或非結構化的數據處理

應用場景:

  • 海量數據分析引擎
  • 站內搜索引擎
  • 作為數據倉庫

一線公司實際應用場景:

  • 英國衛報 - 實時分析公眾對文章的回應
  • 維基百科、GitHub - 站內實時搜索引擎
  • 百度 - 實時日誌監控平臺

單實例安裝ElasticSearch

在安裝ElasticSearch之前,首先準備好Java環境:

[root@localhost /usr/local/jdk1.8]# java -version
java version "1.8.0_171"
Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)
[root@localhost /usr/local/jdk1.8]# 

到ElasticSearch官網上復制ElasticSearch的下載鏈接:

https://www.elastic.co/cn/

復制ElasticSearch 5.x版本的下載地址,到命令行上進行下載:

[root@localhost ~]# cd /usr/local/src/
[root@localhost /usr/local/src]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.2.tar.gz

下載好後,解壓到相應的目錄中:

[root@localhost /usr/local/src]# ls
elasticsearch-5.5.2.tar.gz
[root@localhost /usr/local/src]# tar -zvxf elasticsearch-5.5.2.tar.gz -C /usr/local
[root@localhost /usr/local/src]# ls /usr/local/elasticsearch-5.5.2/
bin  config  lib  LICENSE.txt  modules  NOTICE.txt  plugins  README.textile
[root@localhost /usr/local/src]#

安裝完成後,我們需要新建一個普通用戶,因為ElasticSearch不允許在root用戶下運行,所以我們需要創建獨立的用戶來運行es。創建elsearch用戶組及elsearch用戶:

[root@localhost ~]# groupadd elsearch
[root@localhost ~]# useradd elsearch -g elsearch -p elasticsearch
[root@localhost ~]# passwd elsearch  # 設置密碼

更改elasticsearch安裝目錄及其內部文件的所屬用戶及組為elsearch:elsearch,如下:

[root@localhost ~]# cd /usr/local/
[root@localhost /usr/local]# chown -R elsearch:elsearch elasticsearch-5.5.2/
[root@localhost /usr/local]# ll |grep elasticsearch-5.5.2
drwxr-xr-x. 8 elsearch elsearch 143 6月  19 10:46 elasticsearch-5.5.2
[root@localhost /usr/local]#

由於在CentOS7下,elasticsearch 5.x 設置了ip後,啟動會有些問題導致啟動報錯,我們需要設置如下系統參數來解決這個問題:

[root@localhost /usr/local/elasticsearch-5.5.2]# vim /etc/security/limits.conf
*                hard    nofile          65536
*                soft    nofile          131072
*                soft    nproc           2048   
*                hard    nproc           4096
[root@localhost /usr/local/elasticsearch-5.5.2]# vim /etc/security/limits.d/90-nproc.conf
*                soft    nproc           2048
[root@localhost /usr/local/elasticsearch-5.5.2]# vim /etc/sysctl.conf
vm.max_map_count=655360
[root@localhost /usr/local/elasticsearch-5.5.2]# sysctl -p
vm.max_map_count = 655360
[root@localhost /usr/local/elasticsearch-5.5.2]# 

關於這個問題可以參考如下issues:

https://github.com/elastic/elasticsearch/issues/22899

Elasticsearch5.5.1安裝問題匯總:

https://github.com/DimonHo/DH_Note/issues/3


完成以上的配置後,我們從root用戶切換到elsearch用戶:

[root@localhost ~]# su elsearch
[elsearch@localhost root]$

在啟動es前先設置一下綁定的ip,設置為可被外部訪問的ip,以及設置bootstrap相關的參數:

[elsearch@localhost root]$ cd /usr/local/elasticsearch-5.5.2/
[elsearch@localhost elasticsearch-5.5.2]$ vim config/elasticsearch.yml
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 0.0.0.0  # 找到該項,設置為可被外部訪問的ip
[elsearch@localhost elasticsearch-5.5.2]$ 

啟動ElasticSearch服務的命令:

[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch

加上-d參數表示後臺啟動,一般我們都采用後臺啟動的方式:

[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch -d

ElasticSearch服務啟動成功後,可看到如下進程信息:

[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep elasticsearch
elsearch  13353  107 83.1 4697512 1550164 pts/0 Sl   11:04   0:03 /usr/local/jdk1.8/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/local/elasticsearch-5.5.2 -cp /usr/local/elasticsearch-5.5.2/lib/* org.elasticsearch.bootstrap.Elasticsearch -d
elsearch  13372  0.0  0.0 112720   988 pts/0    S+   11:05   0:00 grep --color=auto elasticsearch
[elsearch@localhost elasticsearch-5.5.2]$ 

查看ElasticSearch服務所監聽的端口:

[elsearch@localhost elasticsearch-5.5.2]$ netstat -lntp |grep java
tcp6       0      0 :::9200                 :::*                    LISTEN      14849/java          
tcp6       0      0 :::9300                 :::*                    LISTEN      14849/java          
[elsearch@localhost elasticsearch-5.5.2]$ 

設置防火墻規則,開放ElasticSearch服務所監聽的端口:

[elsearch@localhost elasticsearch-5.5.2]$  sudo firewall-cmd --zone=public --add-port=9200/tcp --permanent
success
[elsearch@localhost elasticsearch-5.5.2]$  sudo firewall-cmd --zone=public --add-port=9300/tcp --permanent
success
[elsearch@localhost elasticsearch-5.5.2]$  sudo firewall-cmd --reload
success
[elsearch@localhost elasticsearch-5.5.2]$ 

開放了相應的端口後,從瀏覽器中訪問,看看能否訪問成功,如下則是訪問成功的:
技術分享圖片

關於關閉及重啟elasticsearch服務,elasticsearch沒有直接的關閉或重啟命令,關閉elasticsearch服務只能通過kill命令來殺掉服務的進程,如下:

[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep es  # 查看進程id
[elsearch@localhost elasticsearch-5.5.2]$ kill -8 193460  # 通過進程id來殺掉服務

重啟:

[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep es  # 查看進程id
[elsearch@localhost elasticsearch-5.5.2]$ kill -8 193460  # 通過進程id來殺掉服務
[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch -d

也可以編寫一個服務腳本,以此來方便的啟動或關閉elasticsearch服務。


插件安裝

本小節我們來安裝es的實用插件elasticsearch-head,這個插件可以提供一個圖形化的界面,能讓我們直觀的查看節點數據以及集群健康狀態等。github地址如下:

https://github.com/mobz/elasticsearch-head

進入elasticsearch-head的github地址,復制其.zip的下載鏈接,到命令行上進行下載:

[root@localhost /usr/local/src]# vim /etc/sudoers
[root@localhost /usr/local/src]# wget https://github.com/mobz/elasticsearch-head/archive/master.zip

註:切換成root用戶

解壓下載好的.zip壓縮包:

[root@localhost /usr/local/src]# unzip master.zip
[root@localhost /usr/local/src]# mv elasticsearch-head-master/ /usr/local/
[root@localhost /usr/local/src]# ls /usr/local/elasticsearch-head-master/
Dockerfile         elasticsearch-head.sublime-project  grunt_fileSets.js  LICENCE       plugin-descriptor.properties  README.textile  src
Dockerfile-alpine  Gruntfile.js                        index.html         package.json  proxy                         _site           test
[root@localhost /usr/local/src]#

elasticsearch-head是一個提供圖形化的插件,其圖形化是基於node.js的,所以我們需要先準備好node.js環境,node.js官網下載地址如下:

https://nodejs.org/en/download/

在命令行中下載並解壓安裝包:

[root@localhost /usr/local/src]# wget https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-x64.tar.xz
[root@localhost /usr/local/src]# tar -Jxvf node-v8.11.3-linux-x64.tar.xz -C /usr/local/
[root@localhost /usr/local/src]# mv /usr/local/node-v8.11.3-linux-x64/ /usr/local/node-8.11.3

配置環境變量:

[root@localhost /usr/local/src]# vim /etc/profile
export NODE_HOME=/usr/local/node-8.11.3
export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$NODE_HOME/bin
[root@localhost /usr/local/src]# source /etc/profile
[root@localhost /usr/local/src]# node -v  # 輸出版本號則代表配置成功
v8.11.3
[root@localhost /usr/local/src]#

進入es-head的安裝目錄,執行npm install命令安裝所需的模塊:

[root@localhost /usr/local/src]# cd /usr/local/elasticsearch-head-master/
[root@localhost /usr/local/elasticsearch-head-master]# npm install [email protected] --ignore-scripts  # 因為有墻,phantomjs-prebuilt需要單獨安裝,不然會失敗
[root@localhost /usr/local/elasticsearch-head-master]# npm install

es-head使用的是grunt server作為服務器,而grunt server默認監聽的是localhost,所以我們需要修改Gruntfile.js文件,增加一段配置,不然外部是無法訪問的,如下:

[root@localhost /usr/local/elasticsearch-head-master]# vim Gruntfile.js
connect: {
        server: {
                options: {
                        hostname: ‘0.0.0.0‘,  # 增加這段
                        port: 9100,
                        base: ‘.‘,
                        keepalive: true
                }
        }
}
[root@localhost /usr/local/elasticsearch-head-master]#

設置防火墻規則,開放9100端口:

[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --zone=public --add-port=9100/tcp --permanent
success
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --reload
success
[root@localhost /usr/local/elasticsearch-head-master]# 

es與es-head是兩個獨立的進程,當es-head訪問es服務時,會存在一個跨域問題。所以我們需要修改es的配置文件,增加一些配置項來解決這個問題,如下:

[root@localhost /usr/local/elasticsearch-head-master]# cd ../elasticsearch-5.5.2/config/
[root@localhost /usr/local/elasticsearch-5.5.2/config]# vim elasticsearch.yml  # 文件末尾加上如下配置
http.cors.enabled: true
http.cors.allow-origin: "*"
[root@localhost /usr/local/elasticsearch-5.5.2/config]# 

註:修改完配置文件後需切換到elsearch用戶中重啟es服務。

接著再次進入es-head的安裝目錄,使用如下命令,啟動es-head:

[root@localhost /usr/local/elasticsearch-head-master]# npm run start

使用瀏覽器訪問如下:
技術分享圖片


偽分布式安裝ElasticSearch

以上我們介紹了es單實例的安裝以及es-head的安裝,本小節我們來介紹如何在單機情況下安裝偽分布式的es服務。首先配置之前已安裝好的es服務的配置文件,如下:

[root@localhost ~]# cd /usr/local/elasticsearch-5.5.2/config/
[root@localhost /usr/local/elasticsearch-5.5.2/config]# vim elasticsearch.yml   # 在文件末尾加入如下內容
cluster.name: es
node.name: master
node.master: true
[root@localhost /usr/local/elasticsearch-5.5.2/config]# 

參數說明:

  • cluster.name 集群的名稱,可自定義
  • node.name 節點的名稱,可自定義
  • node.master 是否指定為master角色

新建一個存放slave的目錄,拷貝es壓縮包到該目錄中並解壓,然後再拷貝多一份slave,最後還需要更改es_slave目錄的所屬用戶及用戶組為elsearch:

[root@localhost ~]# mkdir /usr/locales_slave
[root@localhost ~]# cp /usr/local/src/elasticsearch-5.5.2.tar.gz /usr/local/es_slave/
[root@localhost ~]# cd /usr/local/es_slave/
[root@localhost /usr/local/es_slave]# tar -zxvf elasticsearch-5.5.2.tar.gz
[root@localhost /usr/local/es_slave]# mv elasticsearch-5.5.2 ./es_slave1
[root@localhost /usr/local/es_slave]# cp -r ./es_slave1/ es_slave2/
[root@localhost /usr/local]# cd ../
[root@localhost /usr/local]# chown -R elsearch:elsearch es_slave/

逐個修改slave的配置文件:

[root@localhost /usr/local/es_slave]# vim es_slave1/config/elasticsearch.yml
cluster.name: es
node.name: slave1

network.host: 127.0.0.1
http.port: 8200

discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
[root@localhost /usr/local/es_slave]# vim es_slave1/config/elasticsearch.yml
cluster.name: es
node.name: slave2

network.host: 127.0.0.1
http.port: 7200

discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
[root@localhost /usr/local]# 

開放端口:

[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --zone=public --add-port=8200/tcp --permanent
success
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --zone=public --add-port=7200/tcp --permanent
success
[root@localhost /usr/local/elasticsearch-head-master]# firewall-cmd --reload
success
[root@localhost /usr/local/elasticsearch-head-master]# 

切換到elsearch用戶中重啟master節點,並啟動其他兩個slave節點:

[elsearch@localhost elasticsearch-5.5.2]$ ps aux |grep elastic
[elsearch@localhost elasticsearch-5.5.2]$ kill -8 16410
[elsearch@localhost elasticsearch-5.5.2]$ sh bin/elasticsearch -d
[elsearch@localhost elasticsearch-5.5.2]$ cd
[elsearch@localhost root]$ sh /usr/local/es_slave/es_slave1/bin/elasticsearch -d
[elsearch@localhost root]$ sh /usr/local/es_slave/es_slave1/bin/elasticsearch -d

啟動es-head:

[root@localhost ~]# cd /usr/local/elasticsearch-head-master/
[root@localhost /usr/local/elasticsearch-head-master]# npm run start

使用瀏覽器訪問如下:
技術分享圖片

CentOS7 下安裝 ElasticSearch 5.x 及填坑