hive和hbase整合,Caused by: java.lang.ClassNotFoundException
Hive-1.2.1與HBase-1.1.2的整合
這裡的整合是指,將HBase作為儲存資料的庫,由Hive作為連線橋樑
在Hive中建立表時,出現了
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V
這個問題是版本適配問題,關鍵在 Hive 下 hive-hbase-handler-1.2.1.jar 這個jar需要將HBase的以下jar包放到Hive的lib下重新進行編譯,
如果以下報錯:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/client/Connection
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.hive.ql.parse.ParseUtils.ensureClassExists(ParseUtils.java:225)
at org.apache.hadoop.hive.ql.parse.StorageFormat.fillStorageFormat(StorageFormat.java:64)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10767)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10047)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10128)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
... 10 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.client.Connection
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 27 more
粗暴的解決辦法:
將如下 Hbase 安裝目錄lib下面的jar包全部匯入到hive安裝目錄的lib下
hbase-common-1.1.2.jar
hbase-server-1.1.2.jar
hbase-client-1.1.2.jar
hbase-protocol-1.1.2.jar
htrace-core-3.1.0-incubating.jar
netty-all-4.0.23.Final.jar
hbase-hadoop2-compat-1.1.2.jar
hbase-hadoop-compat-1.1.2.jar
metrics-core-2.2.0.jar