1. 程式人生 > >Hive基礎學習——安裝Hive及簡單使用

Hive基礎學習——安裝Hive及簡單使用

轉載請註明出處:http://blog.csdn.net/dongdong9223/article/details/86030401
本文出自【我是幹勾魚的部落格

Ingredients:

1 下載Hive

執行命令:

wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz

將apache-hive-2.3.4-bin.tar.gz下載到目錄:

/opt/hive/

中。

2 解壓縮

解壓縮:

tar -xzvf apache-hive-2.3.4-bin.tar.gz

3 配置

在檔案:

/etc/profile

中加入內容:

# hive
export HIVE_HOME=/opt/hive/apache-hive-2.3.4-bin
export PATH=$HIVE_HOME/bin:$PATH

執行source命令使之生效:

source /etc/profile

4 初始化元資料

4.1 修改“metastore_db”名稱為“metastore_db.tmp”

進入Hive的bin目錄執行資料夾名稱修改命令:

# cd /opt/hive/apache-hive-2.3.4-bin/bin
# mv metastore_db metastore_db.tmp

這裡參考了Hive錯誤:Error: FUNCTION ‘NUCLEUS_ASCII’ already exists. (state=X0Y68,code=30000),做這個修改的原因是,在使用Derby儲存元資料的時候,如果不進行這個修改,初始化的過程會出現錯誤:

# schematool -dbType derby -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :	 org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:	 APP
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.derby.sql
Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

4.2 初始化元資料

正如Hive官網Running HiveServer2 and Beeline中所說,從Hive 2.1開始,需要使用schematool命令對元資料進行初始化。元資料既可以儲存在Hive自帶的Derby資料庫上,也可以儲存在指定資料庫(比如MySQL)上,簡單起見這裡就是用預設的Derby資料庫,初始化命令如下:

schematool -dbType derby -initSchema

5 執行Hive

5.1 安裝並啟動Hadoop

安裝Hadoop可以參考阿里雲ECS上搭建Hadoop叢集環境——使用兩臺ECS伺服器搭建“Cluster mode”的Hadoop叢集環境,啟動Hadoop命令為:

sbin/start-all.sh

5.2 建立HDFS上的資料夾及許可權設定

$ $HADOOP_HOME/bin/hadoop fs -mkdir       /tmp
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /user/hive/warehouse
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /tmp
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /user/hive/warehouse

5.3 執行Hive命令

5.3.1 進入Hive

執行命令:

# hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/hive-common-2.3.4.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive>

5.3.2 檢視所有資料庫

hive> show databases;
OK
default
Time taken: 0.02 seconds, Fetched: 1 row(s)

5.3.3 檢視所有資料表

hive> show tables;
OK
Time taken: 0.049 seconds

6 參考

阿里雲ECS上搭建Hadoop叢集環境——使用兩臺ECS伺服器搭建“Cluster mode”的Hadoop叢集環境

Hive錯誤:Error: FUNCTION ‘NUCLEUS_ASCII’ already exists. (state=X0Y68,code=30000)

關於hive異常:Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStor