1. 程式人生 > >hive和hbase整合,Caused by: java.lang.ClassNotFoundException

hive和hbase整合,Caused by: java.lang.ClassNotFoundException

Hive-1.2.1與HBase-1.1.2的整合
這裡的整合是指,將HBase作為儲存資料的庫,由Hive作為連線橋樑

在Hive中建立表時,出現了

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V

這個問題是版本適配問題,關鍵在 Hive 下 hive-hbase-handler-1.2.1.jar 這個jar需要將HBase的以下jar包放到Hive的lib下重新進行編譯,

如果以下報錯:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/client/Connection
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.hadoop.hive.ql.parse.ParseUtils.ensureClassExists(ParseUtils.java:225)
    at org.apache.hadoop.hive.ql.parse.StorageFormat.fillStorageFormat(StorageFormat.java:64)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10767)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10047)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10128)
    at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
    ... 10 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.client.Connection
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 27 more

粗暴的解決辦法:

將如下 Hbase 安裝目錄lib下面的jar包全部匯入到hive安裝目錄的lib下

hbase-common-1.1.2.jar
hbase-server-1.1.2.jar
hbase-client-1.1.2.jar
hbase-protocol-1.1.2.jar
htrace-core-3.1.0-incubating.jar
netty-all-4.0.23.Final.jar
hbase-hadoop2-compat-1.1.2.jar
hbase-hadoop-compat-1.1.2.jar
metrics-core-2.2.0.jar