Hive基礎學習——安裝Hive及簡單使用
轉載請註明出處:http://blog.csdn.net/dongdong9223/article/details/86030401
本文出自【我是幹勾魚的部落格】
Ingredients:
-
Java:Java SE Development Kit 8u162(Oracle Java Archive),Linux下安裝JDK修改環境變數
-
Hadoop:hadoop-2.9.1.tar.gz(Apache Hadoop Releases Downloads, All previous releases of Hadoop are available from the Apache release archive site
-
Hive:hive-2.3.4(mirrors.tuna.tsinghua.edu.cn,Mirror site for Hive)
1 下載Hive
執行命令:
wget -c https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz
將apache-hive-2.3.4-bin.tar.gz下載到目錄:
/opt/hive/
中。
2 解壓縮
解壓縮:
tar -xzvf apache-hive-2.3.4-bin.tar.gz
3 配置
在檔案:
/etc/profile
中加入內容:
# hive
export HIVE_HOME=/opt/hive/apache-hive-2.3.4-bin
export PATH=$HIVE_HOME/bin:$PATH
執行source命令使之生效:
source /etc/profile
4 初始化元資料
4.1 修改“metastore_db”名稱為“metastore_db.tmp”
進入Hive的bin目錄執行資料夾名稱修改命令:
# cd /opt/hive/apache-hive-2.3.4-bin/bin
# mv metastore_db metastore_db.tmp
這裡參考了Hive錯誤:Error: FUNCTION ‘NUCLEUS_ASCII’ already exists. (state=X0Y68,code=30000),做這個修改的原因是,在使用Derby儲存元資料的時候,如果不進行這個修改,初始化的過程會出現錯誤:
# schematool -dbType derby -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.derby.sql
Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
4.2 初始化元資料
正如Hive官網Running HiveServer2 and Beeline中所說,從Hive 2.1開始,需要使用schematool命令對元資料進行初始化。元資料既可以儲存在Hive自帶的Derby資料庫上,也可以儲存在指定資料庫(比如MySQL)上,簡單起見這裡就是用預設的Derby資料庫,初始化命令如下:
schematool -dbType derby -initSchema
5 執行Hive
5.1 安裝並啟動Hadoop
安裝Hadoop可以參考阿里雲ECS上搭建Hadoop叢集環境——使用兩臺ECS伺服器搭建“Cluster mode”的Hadoop叢集環境,啟動Hadoop命令為:
sbin/start-all.sh
5.2 建立HDFS上的資料夾及許可權設定
$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
$ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse
5.3 執行Hive命令
5.3.1 進入Hive
執行命令:
# hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/hive-common-2.3.4.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive>
5.3.2 檢視所有資料庫
hive> show databases;
OK
default
Time taken: 0.02 seconds, Fetched: 1 row(s)
5.3.3 檢視所有資料表
hive> show tables;
OK
Time taken: 0.049 seconds
6 參考
阿里雲ECS上搭建Hadoop叢集環境——使用兩臺ECS伺服器搭建“Cluster mode”的Hadoop叢集環境
Hive錯誤:Error: FUNCTION ‘NUCLEUS_ASCII’ already exists. (state=X0Y68,code=30000)
關於hive異常:Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStor