Hive 安裝-使用HDFS檔案系統
阿新 • • 發佈:2018-12-04
1.下載、解壓Hive
下載網站:http://www.apache.org/dyn/closer.cgi/hive/
百度網盤:連結:https://pan.baidu.com/s/16QzSgxa_VLnJ7ksnLUrtEw
提取碼:ef9b
這裡下載的是2.3.4 版本。
- 解壓到 /home/hadoop/目,並修改名稱:
tar -zxvf apache-hive-2.3.4-bin.tar.gz -C ~/
mv apache-hive-2.3.4-bin hive-2.3.4
- 配置環境變數
vim ~/.bashrc
export HIVE_HOME=/home/hadoop/hive-2.3.4 export PATH=$PATH:$HIVE_HOME/bin/
執行 source ~/.bashrc
2. 安裝MySQL
- 不同版本請百度,這裡簡述Ubuntu版本安裝命令:
sudo apt-get install mysql-server
sudo apt-get isntall mysql-client
sudo apt-get install libmysqlclient-dev
- 建立使用者
- 使用root 使用者登入
- 建立資料庫
- 建立普通使用者:bee,密碼:123456:
- 授權bee使用者擁有剛才建立資料庫的所有許可權
- 重新整理許可權表
mysql -uroot -p create database hiveDB DEFAULT CHARSET utf8 COLLATE utf8_general_ci; create user 'bee' identified by '123456'; grant all privileges on hiveDB.* to 'bee'@'%' identified by '123456'; flush privileges;
登入 bee使用者檢視資料庫:
重點:
將JDBC驅動檔案複製到Hive的lib目錄:
3. 配置Hive
目錄:hive-2.3.4/conf
- hive-env.sh
在末尾加入Hadoop安裝目錄,如博主的目錄為:
HADOOP_HOME=/usr/local/hadoop
- hive-site.xml
目錄下應該不存在,所以自己建立,在此附上完整配置程式碼:
<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>hive.exec.scratchdir</name> <value>/tmp/hive</value> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>hdfs://master:9000/hive/warehouse</value> <description>location to default database for the warehouse</description> </property> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hiveDB?createDatabaseIfNotExist=true&characterEncoding=UTF-8&useSSL=false</value> <description>Hive access metastore using JDBC connectionURL</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>bee</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> <description>password to access metastore database</description> </property> <property> <name>javax.jdo.option.Multithreaded</name> <value>true</value> </property> <!-- 分割 --> <property> <name>hive.metasotre.schema.verification</name> <value>true</value> </property> </configuration>
- HDFS建立目錄
開啟HDFS情況下,建立上述配置中的目錄,並賦予許可權:
hdfs dfa -mkdir -p /tmp/hive
hdfs dfs -mkdir -p /hive/warehouse
hdfs dfs -chmod -R g+w /tmp
hdfs dfs -chmod -R g+w /hive
- 初始化資料庫(可以先跳過這步,因為博主看的書上沒有寫,如果後面操作時有錯誤可以返回來再操作,博主是先執行了)
命令:
[email protected]:~$ schematool -dbType mysql -initSchema
- 啟動Hive
Hive的HQL操作和SQL幾乎一樣,這裡簡單示例,default為系統自帶資料庫: