hive 呼叫python腳步報錯
Error: java.lang.RuntimeException: Hive Runtime Error while closing operators: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:587)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:655)
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:279)
... 7 more
在對Hive 進行 select 查詢的時候 我們可以編寫 Python 、PHP 、c++等指令碼來進行相應的資料處理，我們要用到hive 的 TRANSFORM 和 using
這裡我們需要注意的是，一定要先 將map指令碼新增到分散式快取中，否則會報錯
解決辦法，新增指令碼路徑，如
hive>add file oss://LTA.py;
- hive>add file /demo.py;