hive 安裝部署（一）
阿新 • • 發佈：2021-12-05
1. hive 安裝
1、下載解壓：
// 這裡選擇的是 1.2.1 的版本
[hadoop@hadoop1 apps]$ tar -zxvf apache-hive-1.2.1-bin.tar.gz
// 修改名稱為 hive
[hadoop@hadoop1 apps]$ mv apache-hive-1.2.1-bin hive
2、修改環境變數：
[hadoop@hadoop1 apps]$ vim ~/.bashrc export HIVE_HOME=/home/hadoop/apps/hive export PATH=$PATH:$HIVE_HOME/bin [hadoop@hadoop1 apps]$ source ~/.bashrc
3、配置 hive-env.sh：
[hadoop@hadoop1 apps]$ cd hive/conf/ [hadoop@hadoop1 conf]$ cp hive-env.sh.template hive-env.sh [hadoop@hadoop1 conf]$ vim hive-env.sh # Set HADOOP_HOME to point to a specific hadoop install directory # HADOOP_HOME=${bin}/../../hadoop // 1. 配置 hadoop 地址 export HADOOP_HOME=/home/hadoop/apps/hadoop-2.7.5 // 配置 hive 配置檔案 # Hive Configuration Directory can be controlled by: # export HIVE_CONF_DIR= export HIVE_CONF_DIR=/home/hadoop/apps/hive/conf
4、啟動 hadoop、hdfs，建立 /tmp 和 /user/hive/warehouse兩個目錄並修改他們的同組許可權可寫：
// hive 表預設儲存在 預設的檔案系統中的 /user/hive/warehouse [hadoop@hadoop1 conf]$ hadoop fs -mkdir /tmp [hadoop@hadoop1 conf]$ hadoop fs -chmod g+w /tmp [hadoop@hadoop1 conf]$ hadoop fs -mkdir -p /user/hive/warehouse [hadoop@hadoop1 conf]$ hadoop fs -chmod g+w /user/hive/warehouse
2. MySQL 安裝
hive 預設採用 derby 資料庫，一般不推薦使用，不能多個客戶端同時連線，生產環境推薦使用 MySQL
1、解除安裝系統自帶 MySQL：
// 檢查
[hadoop@hadoop1 apps]$ rpm -qa|grep mysql
mysql-libs-5.1.73-7.el6.x86_64
// 解除安裝
rpm -e --nodeps mysql-libs-5.1.73-7.el6.x86_64
2、安裝：
// 服務端安裝
[hadoop@hadoop1 apps]$ sudo rpm -ivh MySQL-server-5.6.24-1.el6.x86_64.rpm
[sudo] password for hadoop:
// 服務端安裝完畢後，可用命令檢視產生的隨機密碼
[hadoop@hadoop1 mysql-libs]$ sudo cat /root/.mysql_secret
# The random password set for the root user at Sun Nov 14 16:59:38 2021 (local time): Y4LsdXetH9s8f6XH
// 客戶端安裝
[hadoop@hadoop1 apps]$ sudo rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm
3、修改密碼：
[root@hadoop1 mysql-libs]# mysql -uroot -pY4LsdXetH9s8f6XH
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.24
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
// 修改密碼為 hadoop
mysql> SET PASSWORD=PASSWORD('hadoop');
Query OK, 0 rows affected (0.00 sec)
4、修改許可權使其能夠在任意位置連線 MySQL：
mysql> select user,password,host from mysql.user;
+------+-------------------------------------------+-----------+
| user | password | host |
+------+-------------------------------------------+-----------+
| root | *B34D36DA2C3ADBCCB80926618B9507F5689964B6 | localhost |
| root | *39492AB8B965E7E29B4E43035AF0C2089049DA6F | hadoop1 |
| root | *39492AB8B965E7E29B4E43035AF0C2089049DA6F | 127.0.0.1 |
| root | *39492AB8B965E7E29B4E43035AF0C2089049DA6F | ::1 |
+------+-------------------------------------------+-----------+
4 rows in set (0.00 sec)
// 刪除其他使用者，只保留 localhost，只有該 host 密碼為 hadoop，其他使用者密碼未知
mysql> drop user root@'hadoop1';
Query OK, 0 rows affected (0.00 sec)
mysql> drop user root@'127.0.0.1';
Query OK, 0 rows affected (0.00 sec)
mysql> drop user root@'::1';
Query OK, 0 rows affected (0.00 sec)
// 更新 host 為 %
mysql> update mysql.user set host='%' where user='root';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
mysql> select user,password,host from mysql.user;
+------+-------------------------------------------+------+
| user | password | host |
+------+-------------------------------------------+------+
| root | *B34D36DA2C3ADBCCB80926618B9507F5689964B6 | % |
+------+-------------------------------------------+------+
1 row in set (0.00 sec)
注意：更新了許可權後，需要重啟
MySQL，重啟命令：service mysql restart
3. hive 元資料配置為 MySQL
3.1 驅動拷貝
將 mysql-connector-java-5.1.27-bin.jar 包拷貝到 hive/lib 中：
[root@hadoop1 mysql-libs]# tar -zxvf mysql-connector-java-5.1.27.tar.gz^C
[root@hadoop1 mysql-libs]# ls
MySQL-client-5.6.24-1.el6.x86_64.rpm mysql-connector-java-5.1.27 mysql-connector-java-5.1.27.tar.gz MySQL-server-5.6.24-1.el6.x86_64.rpm
[root@hadoop1 mysql-libs]# cp mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar /home/hadoop/apps/hive/lib/
3.2 配置 Metastore 到 MySql
1、hive/conf 建立 hive-site.xml：
[hadoop@hadoop1 conf]$ vim hive-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hadoop1:3306/hivedb?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
<!-- 如果 mysql 和 hive 在同一個伺服器節點，那麼請更改 hadoop02 為 localhost -->
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hadoop</value>
<description>password to use against metastore database</description>
</property>
</configuration>
注意：啟動
hive時，MySQL許可權需要配置好，否則可能會提示無許可權訪問
3.3 hive 啟動測試
[hadoop@hadoop1 conf]$ hive
Logging initialized using configuration in jar:file:/home/hadoop/apps/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
hive> show databases;
OK
default
Time taken: 2.924 seconds, Fetched: 1 row(s)
// 建立資料庫 hive_1
hive> create database hive_1;
OK
Time taken: 0.366 seconds
hive> show databases;
OK
default
hive_1
Time taken: 0.071 seconds, Fetched: 2 row(s)
// 使用資料庫
hive> use hive_1;
OK
Time taken: 0.12 seconds
// 檢視當前正在使用的資料庫
hive> select current_database();
OK
hive_1
Time taken: 3.961 seconds, Fetched: 1 row(s)
// 建立表 student
hive> create table student(id int, name string, sex string, age int, department string) row format delimited fields terminated by ",";
OK
Time taken: 0.457 seconds
hive> show tables;
OK
student
Time taken: 0.111 seconds, Fetched: 1 row(s)
// 查詢
hive> select * from student;
OK
Time taken: 0.39 seconds
建立成功可在 web-ui 上檢視：http://192.168.131.137:50070/explorer.html#/user/hive/warehouse
參考文章