SpringBoot和hadoop元件Hive的整合(填坑)
阿新 • • 發佈:2019-02-13
說實話,SpringBoot和Hive的整合還是有不少坑的,最主要的坑就是jar包衝突的問題,這個坑我這裡添了我遇到的坑,我提供了原始碼,放在後邊的連結中,以下是部分的程式碼。
1.上maven依賴,如下:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>com.yarm</groupId> <artifactId>Spring-Boot-Demo</artifactId> <version>1.0.0-SNAPSHOT</version> </parent> <artifactId>Spring-Boot-Hadoop</artifactId> <description>Hadoop相關元件</description> <dependencies> <dependency> <groupId>com.yarm</groupId> <artifactId>Spring-Boot-Base</artifactId> <version>${spring-boot-demo.version}</version> </dependency> <dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-jdbc</artifactId> <version>1.1.0</version> <exclusions> <exclusion> <groupId>org.eclipse.jetty.aggregate</groupId> <artifactId>jetty-all</artifactId> </exclusion> <exclusion> <groupId>org.apache.hive</groupId> <artifactId>hive-shims</artifactId> </exclusion> <exclusion> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-slf4j-impl</artifactId> </exclusion> <exclusion> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.thrift</groupId> <artifactId>libthrift</artifactId> <version>0.9.1</version> </dependency> <dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-hadoop</artifactId> <version>${spring-data-hadoop.version}</version> <exclusions> <exclusion> <groupId>javax.servlet</groupId> <artifactId>servlet-api</artifactId> </exclusion> </exclusions> </dependency> </dependencies> </project>
2.連結hive的工具類:
package com.yarm.hadoop.hive.util; import java.sql.Connection; import java.sql.DriverManager; import java.sql.SQLException; import java.sql.Statement; public class HiveUtil { /** * * @Title: getCon * @Description: TODO(獲取HIVE JDBC 連線) * @param @return 設定檔案 * @return Connection 返回型別 * @throws */ public static Connection getCon() { try { Class.forName("org.apache.hive.jdbc.HiveDriver"); Connection conn = DriverManager.getConnection( "jdbc:hive2://127.0.0.1:10000/default", "hive", ""); System.out.println("連線hive"); return conn; } catch (Exception e) { e.printStackTrace(); } return null; } /** * * @Title: getCon * @Description: TODO(獲取HIVE 指定資料庫 JDBC 連線) * @param @param database * @param @return 設定檔案 * @return Connection 返回型別 * @throws */ public static Connection getCon(String database) { try { Class.forName("org.apache.hive.jdbc.HiveDriver"); Connection conn = DriverManager.getConnection( "jdbc:hive2://127.0.0.1:10000/" + database, "hive", ""); System.out.println("連線hive"); return conn; } catch (Exception e) { e.printStackTrace(); } return null; } /** * * @Title: close * @Description: TODO(關閉HIVE JDBC 連線) * @param @param stmt * @param @param conn 設定檔案 * @return void 返回型別 * @throws */ public static void close(Statement stmt, Connection conn){ try { if(stmt != null){ stmt.close(); } if(conn != null){ conn.close(); } } catch (SQLException e) { e.printStackTrace(); } } /** * * @Title: close * @Description: TODO(關閉HIVE JDBC 連線) * @param @param conn 設定檔案 * @return void 返回型別 * @throws */ public static void close(Connection conn){ try { if(conn != null){ conn.close(); } } catch (SQLException e) { e.printStackTrace(); } } /** * * @Title: hex2byte * @Description: 字串轉二進位制 * @param @param str * @param @return 設定檔案 * @return byte[] 返回型別 * @throws */ public static byte[] hex2byte(String str) { // 字串轉二進位制 if (str == null) return null; str = str.trim(); int len = str.length(); if (len == 0 || len % 2 == 1) return null; byte[] b = new byte[len / 2]; try { for (int i = 0; i < str.length(); i += 2) { b[i / 2] = (byte) Integer.decode("0X" + str.substring(i, i + 2)).intValue(); } return b; } catch (Exception e) { return null; } } }
3.呼叫Hive工具類
這一部分我就補貼原始碼了,具體的可以參照我的專案。如需原始碼,可以留言。