手把手教你裝HBase
阿新 • • 發佈：2018-11-21
由於工作的需要，最近又要配置一遍HBase叢集了，正好先在自己的電腦上再熟悉一下。
Hbase的前提工作是必須要裝好Hadoop叢集，我在之前的部落格上講解了如何裝偽分散式和全分散式，這裡大家可以去看一下Hadoop叢集的搭建
啟動ZK和Hadoop叢集
若將zk和hadoop配置到環境變數，則使用：
zkServer.sh start
start-hdfs.sh
start-yarn.sh
若沒有配置環境變數，則需要進去zk的bin目錄下，hadoop的sbin目錄下啟動。
配置HBase的配置資訊
這裡我使用的是Xshell，可以直接將下好的tar包拉取到root目錄下，解壓到/usr/local下
tar -xvf hbase-1.3.1-bin.tar.gz -C /usr/local
進入到hbase的conf目錄下
cd /usr/local/hbase-1.3.1/conf
export JAVA_HOME=/usr/local/jdk1.8.0_152 //新增自己機器上的jdk
export HBASE_MANAGES_ZK=false //不使用hbase自帶的zookeeper
修改hbase-site.xml
<configuration> <property> <name>hbase.rootdir</name> <value>hdfs://node01:9000/hbase</value> <!--這裡的為hdfs的ip和埠,可能hdfs埠為8020--> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <!-- 0.98 後的新變動，之前版本沒有.port,預設埠為 60000 --> <property> <name>hbase.master.port</name> <value>16000</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>node01:2181,node02:2181,node03:2181</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/usr/local/zookeeper-3.4.10/zkData</value> </property> </configuration>
vi regionservers
node01
node02
node03
替換Jar包
因為hbase依賴於hadoop和zookeeper，而自帶的關於hadoop、zookeeper的jar包不合適，所以要將依賴的包替換成相對應hadoop、zookeeper版本的jar包。
jar包在lib目錄下
刪除如下jar包
rm -rf /usr/local/hbase-1.3.1/lib/hadoop-*
rm -rf /usr/local/hbase-1.3.1/lib/zookeeper-3.4.6.jar
依賴的新jar包列表
hadoop-annotations-2.7.2.jar hadoop-auth-2.7.2.jar hadoop-client-2.7.2.jar hadoop-common-2.7.2.jar hadoop-hdfs-2.7.2.jar hadoop-mapreduce-client-app-2.7.2.jar hadoop-mapreduce-client-common-2.7.2.jar hadoop-mapreduce-client-core-2.7.2.jar hadoop-mapreduce-client-hs-2.7.2.jar hadoop-mapreduce-client-hs-plugins-2.7.2.jar hadoop-mapreduce-client-jobclient-2.7.2.jar hadoop-mapreduce-client-jobclient-2.7.2-tests.jar hadoop-mapreduce-client-shuffle-2.7.2.jar hadoop-yarn-api-2.7.2.jar hadoop-yarn-applications-distributedshell-2.7.2.jar hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar hadoop-yarn-client-2.7.2.jar hadoop-yarn-common-2.7.2.jar hadoop-yarn-server-applicationhistoryservice-2.7.2.jar hadoop-yarn-server-common-2.7.2.jar hadoop-yarn-server-nodemanager-2.7.2.jar hadoop-yarn-server-resourcemanager-2.7.2.jar hadoop-yarn-server-tests-2.7.2.jar hadoop-yarn-server-web-proxy-2.7.2.jar zookeeper-3.4.5.jar
可以在你目前安裝好的hadoop和zookeeper的lib目錄下尋找對應的jar包，操作為
find /usr/local/hadoop-2.7.2/ -name hadoop-annotations*
建立軟連線
軟連線相當於建立了快捷連線方式
ln -s /usr/local/hadoop-2.7.2/etc/hadoop/core-site.xml /usr/local/hbase-1.3.1/conf/core-site.xml
ln -s /usr/local/hadoop-2.7.2/etc/hadoop/hdfs-site.xml /usr/local/hbase-1.3.1/conf/hdfs-site.xml
在conf中顯示如下
遠端拷貝HBase
需要將HBase拷貝到想建立的叢集的其他機器上，比如我這裡在node01上進行的操作，要將HBase拷貝到node02，node03上
scp -r /usr/local/hbase-1.3.1/ node02:/usr/local/
scp -r /usr/local/hbase-1.3.1/ node03:/usr/local/
啟動與關閉服務
方式一
bin/hbase-daemon.sh start master //啟動HMaster
bin/hbase-daemon.sh start regionserver //啟動HRegionServer
方式二
bin/start-hbase.sh //啟動
bin/stop-hbase.sh //關閉
使用JPS可以看到程序為
在瀏覽器上輸入ip:16010可以檢視HBase網頁
常見問題彙總
使用第一種方式，出現
org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server ub9001,60020,1343694176934 has been rejected; Reported time is too far out of sync with master. Time difference of 410865ms > max allowed of 180000ms
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
這是因為叢集時間不同步的原因，這裡有兩種方案：
方案一：配置叢集時間同步
方案二：修改hbase.master.maxclockskew
<property>
<name>hbase.master.maxclockskew</name>
<value>180000</value>
<description>Time difference of regionserver from master</description>
</property>
使用start-hbase.sh後，出現
node01: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
node01: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
node01: Error: A JNI error has occurred, please check your installation and try again
node01: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FileSystem
node01: at java.lang.Class.getDeclaredMethods0(Native Method)
node01: at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
node01: at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
node01: at java.lang.Class.getMethod0(Class.java:3018)
node01: at java.lang.Class.getMethod(Class.java:1784)
這時，需要將hbase-env.sh中的HBASE_MASTER_OPTS和HBASE_REGIONSERVER_OPTS前面加#註釋掉。
開啟後出現如下程式碼，且無HMaster，檢查叢集是否開啟，ZK和Hadoop是否正常。
node02: starting regionserver, logging to /usr/local/hbase-1.3.1/bin/../logs/hbase-root-regionserver-spark02.out
node03: starting regionserver, logging to /usr/local/hbase-1.3.1/bin/../logs/hbase-root-regionserver-spark03.out
node01: starting regionserver, logging to /usr/local/hbase-1.3.1/bin/../logs/hbase-root-regionserver-spark01.out
node03: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
node03: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
node02: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
node02: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
關閉時，出現：stopping hbasecat: /tmp/hbase-root-master.pid: No such file or directory，需要手動kill掉，這是因為沒有Master節點或者pid丟失，若為pid檔案丟失，可在hbase-env.sh中配置export HBASE_PID_DIR=/usr/local/hadoop-2.7.2/pids
開啟叢集發現只有一臺HregionServer線上，其餘為死亡狀態，日誌如下：
2018-11-20 16:53:19,226 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting
java.lang.RuntimeException: HRegionServer Aborted
at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68)
at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2721)
配置一下叢集時間同步，再從主節點重新拷貝到其他節點就好了。