Hive 環境的安裝部署

阿新 • • 發佈：2018-05-14

resp www 詳情 apr 所有權限 hive數據使用 database root

Hive在客戶端上的安裝部署

一、客戶端準備：

到這我相信大家都已經打過三節點集群了，如果是的話則可以跳過一，直接進入二。如果不是則按流程來一遍！

1、克隆虛擬機，見我的博客：虛擬機克隆及網絡配置

2、實現客戶端和集群的連接（該步驟為多節點集群搭建，詳情見我博客：三節點Hadoop集群搭建，有多節點集群的請跳到二）

（1）配置時鐘同步：保證客戶端和集群的時間是同步的，具體操作參照分布式集群搭建的步驟。

（2）修改主機名：修改/etc/sysconfig/network文件,修改完之後要reboot重啟，保證生效。這一步不是必須的，因為我是克隆的，這裏是為了名稱的好記，所以建議修改一下。

（3）修改/etc/hosts文件，在各個節點上配置所有節點的主機名和IP之間的對應關系。

（4）實現這臺機器和集群中每臺機器之間的免密碼登錄。

（5）把集群的hadoop安裝包遠程copy到客戶端節點對應的目錄下。這就實現了客戶端和hadoop集群之間環境的一致性和連接

（6）在客戶端節點配置hadoop的環境變量並使環境變量生效。

使用命令vi /etc/profile ，在該文件中配置HADOOP_HOME和PATH並導出（export）

使環境變量生效：source /etc/profile

二、在客戶端上安裝hive

1、 Mysql安裝

1.1 在線安裝mysql

使用

yum在線安裝mysql：yum install mysql-server

技術分享圖片

1.2 啟動mysql服務

使用service mysqld start命令啟動mysql服務

[root@master hadoop]# service mysqld start
Initializing MySQL database:  Installing MySQL system tables...
OK
Filling help tables...
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

/usr/bin/mysqladmin -u root password ‘new-password‘
/usr/bin/mysqladmin -u root -h master password ‘new-password‘

Alternatively you can run:
/usr/bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd /usr/mysql-test ; perl mysql-test-run.pl

Please report any problems with the /usr/bin/mysqlbug script!

                                                           [  OK  ]
Starting mysqld:                                           [  OK  ]

1.3 設置mysql root用戶密碼Mysql剛剛安裝完成，默認root用戶是沒有密碼的，登錄mysql設置root用戶密碼。

[root@master hadoop]# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

Enter password: 默認沒有密碼，回車即可。

輸入如下命令設置root用戶密碼：

mysql> set password for root@localhost=password(‘root‘);
Query OK, 0 rows affected (0.00 sec)

設置完密碼之後退出

mysql> exit
Bye

重新登錄mysql，用戶名：root，密碼：root

[root@master hadoop]# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 15
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql>

此時mysql root用戶密碼設置成功。

1.4 創建hive賬號

首先創建hive賬戶

mysql> create user ‘hive‘ identified by ‘hive‘;
Query OK, 0 rows affected (0.00 sec)

將mysql所有權限授予hive賬戶

mysql> grant all on *.* to ‘hive‘@‘master‘ identified by ‘hive‘;
Query OK, 0 rows affected (0.00 sec)

使操作命令生效

mysql> grant all on *.* to ‘hive‘@‘master‘ identified by ‘hive‘;
Query OK, 0 rows affected (0.00 sec)

查看mysql 用戶表user

mysql> select host,user,password from mysql.user;
+-----------+------+-------------------------------------------+
| host      | user | password                                  |
+-----------+------+-------------------------------------------+
| localhost | root | *DE4EE97EBADA7D108BDD4EC90C49C6500E99E74A |
| master    | root |                                           |
| 127.0.0.1 | root |                                           |
| localhost |      |                                           |
| master    |      |                                           |
| %         | hive | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC |
| master    | hive | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC |
+-----------+------+-------------------------------------------+
7 rows in set (0.00 sec)

使用hive用戶登錄mysql數據庫：

[root@master hadoop]# mysql -h master -u hive -p
Enter password: （密碼為hive）
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

mysql>

創建數據庫hive

mysql> create database hive;
Query OK, 1 row affected (0.00 sec)

查看所有數據庫

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hive               |
| mysql              |
| test               |
+--------------------+
4 rows in set (0.00 sec)

2、 Hive安裝

2.1 下載Hive

選擇hive安裝包apache-hive-1.0.0-bin.tar.gz，選擇一臺客戶端client安裝hive（下載地址：http://ftp.ntu.edu.tw/MySQL/Downloads/Connector-J/），將hive安裝包上傳至指定目錄下。

[root@master hadoop-2.6.0]# cd /opt/ softwares/（上傳到自定義文件夾下就行，為了方便管理，我這裏上傳到放JDK和hadoop的文件夾下）
[root@master softwares]# rz

[root@master softwares]# ll
total 1133432
-rw-r--r-- 1 root   root    79775131 May 14 10:51 apache-hive-1.0.0-bin.tar.gz
-rw-r--r-- 1 hadoop hadoop 195257604 Apr  2 11:29 hadoop-2.6.0.tar.gz
-rwxr--r-- 1 hadoop hadoop 153512879 Jun  2  2017 jdk-7u79-linux-x64.tar.gz
-rwxr--r-- 1 hadoop hadoop 181238643 Sep  3  2016 jdk-8u60-linux-x64.tar.gz
-rw-r--r-- 1 hadoop hadoop  28688464 Mar 13 18:57 scala-2.11.8.tar.gz
-rw-r--r-- 1 hadoop hadoop 289405702 Jun  7  2017 spark-1.6.1-bin-hadoop2.6.tgz
-rw-r--r-- 1 hadoop hadoop 201706782 Aug 28  2017 spark-2.2.0-bin-hadoop2.6.tgz
-rwxr--r-- 1 hadoop hadoop  31032995 Dec 30 15:13 zookeeper-3.4.5-cdh5.10.0.tar.gz

2.2 解壓Hive

使用解壓命令解壓hive：

[root@master softwares]# tar -zxvf apache-hive-1.0.1-bin.tar.gz
apache-hive-1.0.1-bin/conf/hive-log4j.properties.template
apache-hive-1.0.1-bin/conf/hive-exec-log4j.properties.template
apache-hive-1.0.1-bin/conf/beeline-log4j.properties.template
apache-hive-1.0.1-bin/hcatalog/share/doc/hcatalog/README.txt
apache-hive-1.0.1-bin/LICENSE
apache-hive-1.0.1-bin/NOTICE
apache-hive-1.0.1-bin/README.txt
apache-hive-1.0.1-bin/RELEASE_NOTES.txt
……
[root@master softwares]# ll
total 1133436
drwxr-xr-x 8 root   root        4096 May 14 19:01 apache-hive-1.0.0-bin
-rw-r--r-- 1 root   root    79775131 May 14 10:51 apache-hive-1.0.1-bin.tar.gz
-rw-r--r-- 1 hadoop hadoop 195257604 Apr  2 11:29 hadoop-2.6.0.tar.gz
-rwxr--r-- 1 hadoop hadoop 153512879 Jun  2  2017 jdk-7u79-linux-x64.tar.gz
-rwxr--r-- 1 hadoop hadoop 181238643 Sep  3  2016 jdk-8u60-linux-x64.tar.gz
-rw-r--r-- 1 hadoop hadoop  28688464 Mar 13 18:57 scala-2.11.8.tar.gz
-rw-r--r-- 1 hadoop hadoop 289405702 Jun  7  2017 spark-1.6.1-bin-hadoop2.6.tgz
-rw-r--r-- 1 hadoop hadoop 201706782 Aug 28  2017 spark-2.2.0-bin-hadoop2.6.tgz
-rwxr--r-- 1 hadoop hadoop  31032995 Dec 30 15:13 zookeeper-3.4.5-cdh5.10.0.tar.gz
[root@master softwares]# mv apache-hive-1.0.0-bin ../modules/
[root@master softwares]# cd ../modules/
[root@master modules]# ll
total 28
drwxr-xr-x  8 root   root   4096 May 14 19:01 apache-hive-1.0.0-bin
drwxr-xr-x 12 hadoop hadoop 4096 Apr 11 00:00 hadoop-2.6.0
lrwxrwxrwx  1 hadoop hadoop   12 Apr  9 05:59 jdk -> jdk1.8.0_60/
drwxr-xr-x  8 hadoop hadoop 4096 Apr 11  2015 jdk1.7.0_79
drwxr-xr-x  8 hadoop hadoop 4096 Aug  5  2015 jdk1.8.0_60
drwxrwxr-x  6 hadoop hadoop 4096 Mar  4  2016 scala-2.11.8
drwxr-xr-x 15 hadoop hadoop 4096 Apr  9 06:27 spark-2.2.0-bin-hadoop2.6
drwxr-xr-x 14 hadoop hadoop 4096 Apr  9 00:00 zookeeper-3.4.5-cdh5.10.0

修改解壓包名稱為hive:mv apache-hive-1.0.0-bin hive1.0.0

[root@master modules]# mv apache-hive-1.0.1-bin hive1.0.0  //重命名
[root@master modules]# ll
total 28
drwxr-xr-x 12 hadoop hadoop 4096 Apr 11 00:00 hadoop-2.6.0
drwxr-xr-x  8 root   root   4096 May 14 19:01 hive1.0.0
lrwxrwxrwx  1 hadoop hadoop   12 Apr  9 05:59 jdk -> jdk1.8.0_60/
drwxr-xr-x  8 hadoop hadoop 4096 Apr 11  2015 jdk1.7.0_79
drwxr-xr-x  8 hadoop hadoop 4096 Aug  5  2015 jdk1.8.0_60
drwxrwxr-x  6 hadoop hadoop 4096 Mar  4  2016 scala-2.11.8
drwxr-xr-x 15 hadoop hadoop 4096 Apr  9 06:27 spark-2.2.0-bin-hadoop2.6
drwxr-xr-x 14 hadoop hadoop 4096 Apr  9 00:00 zookeeper-3.4.5-cdh5.10.0

2.3 修改hive配置hive-site.xml

hive-site.xml文件不存在，首先復制一份該文件

[root@master modules]# cd hive1.0.1/
[root@master hive1.0.1]# ll
total 396
drwxr-xr-x 3 root root   4096 May 14 19:01 bin
drwxr-xr-x 2 root root   4096 May 14 19:01 conf
drwxr-xr-x 4 root root   4096 May 14 19:01 examples
drwxr-xr-x 7 root root   4096 May 14 19:01 hcatalog
drwxr-xr-x 4 root root   4096 May 14 19:01 lib
-rw-r--r-- 1 root root  23828 May 15  2015 LICENSE
-rw-r--r-- 1 root root    397 May 15  2015 NOTICE
-rw-r--r-- 1 root root   4044 May 15  2015 README.txt
-rw-r--r-- 1 root root 345849 May 15  2015 RELEASE_NOTES.txt
drwxr-xr-x 3 root root   4096 May 14 19:01 scripts
[root@master hive1.0.1]# cd conf/
[root@master conf]# ll
total 160
-rw-r--r-- 1 root root   1139 May  2  2015 beeline-log4j.properties.template
-rw-r--r-- 1 root root 147301 May 15  2015 hive-default.xml.template
-rw-r--r-- 1 root root   2378 May  2  2015 hive-env.sh.template
-rw-r--r-- 1 root root   2662 May  2  2015 hive-exec-log4j.properties.template
-rw-r--r-- 1 root root   3050 May  2  2015 hive-log4j.properties.template
[root@master conf]# cp hive-default.xml.template hive-site.xml
[root@master conf]# ll
total 304
-rw-r--r-- 1 root root   1139 May  2  2015 beeline-log4j.properties.template
-rw-r--r-- 1 root root 147301 May 15  2015 hive-default.xml.template
-rw-r--r-- 1 root root   2378 May  2  2015 hive-env.sh.template
-rw-r--r-- 1 root root   2662 May  2  2015 hive-exec-log4j.properties.template
-rw-r--r-- 1 root root   3050 May  2  2015 hive-log4j.properties.template
-rw-r--r-- 1 root root 147301 May 14 19:07 hive-site.xml

修改hive-site.xml配置(配置文件內容很多，人眼查找費時費力，通過vim/vi進入文件後可輸入一個“/”然後追加字符串進行查找)

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql//:master:3306/hive?characterEncoding=UTF-8</value>
    <description>JDBC connect string for a JDBC metastore</description>
</property> 
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>driver class name for a JDBC metastore</decription>
</property>

上表中指定的hive數據庫實例，提前已經創建好了。

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive</value>
    <description>password to use against metastore database</description>
</property> 
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>Username to use against metastore database</description>
</property>

2.4 配置Hive環境變量

打開vi /etc/profile文件，添加如下內容：

技術分享圖片

保存退出，並是配置文件生效

[root@master conf]# source /etc/profile

2.5 將mysql驅動包拷貝到hive的lib目錄

下載mysql-connector-java-5.1.21.jar，並上傳至hive的lib目錄下。

技術分享圖片

2.6 修改hive數據目錄

修改配置文件vi hive-site.xml，更改相關數據目錄

 <property>
    <name>hive.querylog.location</name>
    <value>/opt/modules/hive/iotmp</value>
    <description>Local of hive run time structured log file</description>
  </property>
  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/opt/modules/hive/iotmp</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/opt/modules/hive/iotmp</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

2.7 測試運行hive

2.7.1啟動hadoop集群

首先，分別在個節點啟動Zookeeper

[hadoop@master hadoop-2.6.0]$ sbin/hadoop-daemon.sh start zkfc

然後，分別啟動zkfc（HA高熱備），hdfs和yarn等個項服務

[hadoop@master hadoop-2.6.0]$ sbin/start-dfs.sh
[hadoop@master hadoop-2.6.0]$ sbin/start-yarn.sh
[hadoop@master hadoop-2.6.0]$ cd ..
[hadoop@master modules]$ cd zookeeper-3.4.5-cdh5.10.0/（根據自己Zookeeper安裝路徑而定）
[hadoop@master zookeeper-3.4.5-cdh5.10.0]$ bin/zkServer.sh start
[hadoop@master zookeeper-3.4.5-cdh5.10.0]$ jps
6481 Jps
5330 DataNode
5733 ResourceManager
5077 DFSZKFailoverController
6456 QuorumPeerMain
5225 NameNode
5514 JournalNode
5837 NodeManager

2.7.2啟動Hive

[hadoop@master hive1.0.0]$ bin/hive
Logging initialized using configuration in file:/opt/modules/hive1.0.0/conf/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/modules/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/modules/hive1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
hive> show tables;
OK
Time taken: 0.499 seconds
hive> show databases;
OK
default
Time taken: 0.044 seconds, Fetched: 1 row(s)

Ok，出現上面的結果，hive安裝成功。

以上就是博主為大家介紹的這一板塊的主要內容，這都是博主自己的學習過程，希望能給大家帶來一定的指導作用，有用的還望大家點個支持，如果對你沒用也望包涵，有錯誤煩請指出。如有期待可關註博主以第一時間獲取更新哦，謝謝！　

Hive 環境的安裝部署

Hive 環境的安裝部署

Hbase 分布式環境安裝部署

Pyenv環境安裝部署

測試環境安裝部署文檔

yumerastcdwww3992019com環境安裝部署18669144441

ansible環境安裝部署

Spark環境安裝部署及詞頻統計例項

Flink環境安裝部署、詞頻統計例項、WordCount原始碼分析

Python 介紹及環境安裝部署

Sonar 本地開發環境安裝部署及應用

ubuntu16.04 docker環境安裝部署

Ubuntu 16.04 環境安裝部署專案部署文件

Spark環境安裝部署教程

分散式Hive環境安裝

Linux系統如何在離線環境或內網環境安裝部署Docker服務和其他服務

Hive 環境的安裝部署

Hive環境的安裝部署（完美安裝）（叢集內或叢集外都適用）（含解除安裝自帶mysql安裝指定版本）

hadoop分布式安裝部署具體視頻教程(網盤附配好環境的CentOS虛擬機文件/hadoop配置文件)

zookeeper與kafka安裝部署及java環境搭建

環境搭建/安裝部署

Apache-Tomcat-9在centos7.4環境中安裝部署

Hive 環境的安裝部署

相關推薦