Hive解析欄位為Json的資料
阿新 • • 發佈:2018-12-10
HIVE直接讀入json的函式有兩個:
(1)get_json_object(string json_string, string path)
返回值: string
說明:解析json的字串json_string,返回path指定的內容。如果輸入的json字串無效,那麼返回NULL。
舉例:
hive> select get_json_object(‘{“store”:{“fruit”:[{"weight":8,"type":"apple"},{"weight":9,"type":"pear"}], “bicycle”:{“price”:19.95,”color”:”red”}}, “email”:” [email protected]_for_json_udf_test.net”, “owner”:”amy” } ‘,’$.owner’) from dual;
結果:amy
這個函式每次只能返回一個數據項。
(2)json_tuple(jsonStr, k1, k2, ...)
引數為一組鍵k1,k2……和JSON字串,返回值的元組。該方法比 get_json_object 高效,因為可以在一次呼叫中輸入多個鍵
select a.timestamp, b.* from log a lateral view json_tuple(a.appevent, 'eventid', 'eventname') b as f1, f2;
處理資料樣例:
{ "GPS_LAT": 39.8965125, "GPS_LONG": 116.3493225, "GPS_SPEED": 20.9993625, "GPS_STATE": "A", "GPS_TIME": "2014-01-02 00:00:16", "IMEI": "508597", "after_oxygen_sensor": 132, "air_condion_state": 3, "bdoneNo_after_mileage": 0, "bdoneNo_zero_mileage": 8044, "db_speed": 22, "direction_angle": 358.2585, "front_oxygen_sensor": 64, "instant_fuel": 233, "speed": 1210, "torque": 33, "total_fuel": 0 }
處理HIVE語句:
create table 2014jrtest as select json_tuple(line,'GPS_LAT','GPS_LONG','GPS_SPEED','GPS_STATE','GPS_TIME','IMEI','after_oxygen_sensor','air_condion_state','bdoneNo_after_mileage','bdoneNo_zero_mileage','db_speed','direction_angle','front_oxygen_sensor','instant_fuel','speed','torque','total_fuel') from 2014test;
作者:石曉文的學習日記 連結:https://www.jianshu.com/p/8a868d2ac307 來源:簡書 簡書著作權歸作者所有,任何形式的轉載都請聯絡作者獲得授權並註明出處。