使用hive客戶端java api讀寫hive叢集上的資訊
阿新 • • 發佈:2022-05-02
上文介紹了hdfs叢集資訊的讀取方式,本文說hive
1、先解決依賴
<properties> <hive.version>1.2.1</hive.version> </properties> <dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-exec</artifactId> <version>${hive.version}</version> <scope>provided</scope> </dependency>
2、配置檔案
這裡我們給出一種簡單的配置方法,就是直接將hive-site.xml通過新增檔案的方式載入到配置
例如,hive-site.xml中的配置如下
<configuration> <property> <name>hive.metastore.uris</name> <value>thrift://10.91.64.23:9083,thrift://10.91.64.23:9083,thrift://10.91.64.23:9083</value> </property> </configuration>
3、hive client api
說明:
1、hiveConf.addResource("hive-site.xml") 可以直接把配置檔案載入到配置
2、hive的api很豐富,下面只介紹了其中一部分,如果用到其他再進行封裝即可
package com.xiaoju.dqa.prometheus.client.hive; import org.apache.hadoop.hive.conf.HiveConf; import org.apache.hadoop.hive.metastore.IMetaStoreClient; import org.apache.hadoop.hive.metastore.RetryingMetaStoreClient; import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.metastore.api.FieldSchema; import org.apache.hadoop.hive.metastore.api.MetaException; import org.apache.thrift.TException; import org.slf4j.Logger; import java.util.List; public class HiveClient { protected final Logger logger = org.slf4j.LoggerFactory.getLogger(this.getClass()); IMetaStoreClient client; public HiveClient() { try { HiveConf hiveConf = new HiveConf(); hiveConf.addResource("hive-site.xml"); client = RetryingMetaStoreClient.getProxy(hiveConf); } catch (MetaException ex) { logger.error(ex.getMessage()); } } public List<String> getAllDatabases() { List<String> databases = null; try { databases = client.getAllDatabases(); } catch (TException ex) { logger.error(ex.getMessage()); } return databases; } public Database getDatabase(String db) { Database database = null; try { database = client.getDatabase(db); } catch (TException ex) { logger.error(ex.getMessage()); } return database; } public List<FieldSchema> getSchema(String db, String table) { List<FieldSchema> schema = null; try { schema = client.getSchema(db, table); } catch (TException ex) { logger.error(ex.getMessage()); } return schema; } public List<String> getAllTables(String db) { List<String> tables = null; try { tables = client.getAllTables(db); } catch (TException ex) { logger.error(ex.getMessage()); } return tables; } public String getLocation(String db, String table) { String location = null; try { location = client.getTable(db, table).getSd().getLocation(); }catch (TException ex) { logger.error(ex.getMessage()); } return location; } }