1. 異常問題Permission denied: user [anonymous] does not have [USE] privilege on [default]

com.streamsets.pipeline.api.base.OnRecordErrorException: HIVE_23 - TBL Properties 'com.streamsets.pipeline.stage.lib.hive.exceptions.HiveStageCheckedException: HIVE_20 - Error executing SQL: DESCRIBE DATABASE `default`, Reason:Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [anonymous] does not have [USE] privilege on [default]' Mismatch: Actual: {} , Expected: {}
  at   com.streamsets.pipeline.stage.processor.hive.HiveMetadataProcessor.process(HiveMetadataProcessor.java:585)
	at com.streamsets.pipeline.api.base.RecordProcessor.process(RecordProcessor.java:52)
	at com.streamsets.pipeline.api.base.configurablestage.DProcessor.process(DProcessor.java:35)
	at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:286)
	at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:235)
	at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:298)
	at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:244)


 

解決辦法：

在叢集的ranger中為hive服務新增{user} 群組的許可權。

2.異常問題二

using Storage Format Type org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, but Avro requested instead

原因：因為Steamsets中配置的hive metadata存在格式是AVRO格式，而實際上建立的表格式和AVRO格式不一致。

解決辦法：刪掉之前的表，重新建一個儲存格式是AVRO的表

CREATE TABLE `test3`(
  `id` int,
  `name` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS AVRO
LOCATION
  'hdfs://ns1/apps/hive/warehouse/test3'；