1. 程式人生 > >MapReduce基礎開發之九JDBC連線Hive

MapReduce基礎開發之九JDBC連線Hive

1、eclipse建工程,引入$HIVE_HOME/lib的包和$Hadoop_HOME/lib的包(一般核心包就可以,具體根據開發需求),匯出jar包(不帶第三方jar);

    這裡是Hive2,程式碼如下:

package com.hive;

import java.sql.SQLException;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.DriverManager;

public class HiveJdbc {
	private static String driverName = "org.apache.hive.jdbc.HiveDriver";

	public static void main(String[] args)  throws SQLException {
		
		try {
			Class.forName(driverName);
		} catch (ClassNotFoundException e) {
			e.printStackTrace();
			System.exit(1);
		}
		
		Connection con = DriverManager.getConnection("jdbc:hive2://IP:10000/default", "", "");
		Statement stmt = con.createStatement();
		String tableName = "JDBCTest";
		stmt.execute("drop table if exists " + tableName);
		stmt.execute("create table " + tableName +   " (key int, value string)");
		System.out.println("Create table success!");
		// show tables
		String sql = "show tables '" + tableName + "'";
		System.out.println("Running: " + sql);
		ResultSet res = stmt.executeQuery(sql);
		if (res.next()) {
			System.out.println(res.getString(1));
		}
		
		// describe table
		sql = "describe " + tableName;
		System.out.println("Running: " + sql);
		res = stmt.executeQuery(sql);
		while (res.next()) {
			System.out.println(res.getString(1) + "\t" + res.getString(2));
		}
		//select table
		sql = "select * from " + tableName;
		res = stmt.executeQuery(sql);
		while (res.next()) {
			System.out.println(String.valueOf(res.getInt(1)) + "\t"+ res.getString(2));
		}
	}
}
2、根據hadoop和hive叢集部署情況,編寫指令碼並執行,加入hive和hadoop的jar包。
   注意:shell指令碼最好在centos下編寫,如果是Windows下編寫因為字元問題執行起來會出現語法錯誤
   #vi /tmp/hivejdbc.sh
#!/bin/bash

HADOOP_HOME=/usr/lib/hadoop
HIVE_HOME=/usr/lib/hive
JAVA_HOME=/usr/lib/java

CLASSPATH=$CLASSPATH: 

for i in /usr/lib/hive/lib/*.jar; do 
CLASSPATH=$CLASSPATH:$i 
done

for i in /usr/lib/hadoop/client/*.jar; do 
CLASSPATH=$CLASSPATH:$i 
done

echo $CLASSPATH 
java -cp $CLASSPATH:/tmp/hivejdbc.jar com.hive.HiveJdbc 

  執行指令碼sh /tmp/hivejdbc.sh看結果。
  出現提示沒有hdfs中hive使用者的檔案許可權/user/hive/warehouse;
  執行命令hadoop fs -chmod 777 /user/hive 


3、實際上,在效能上,不建議通過jdbc連線hive來處理。