使用shell將hdfs上的資料匯入到hive表中
阿新 • • 發佈:2018-12-22
days=($(seq 20150515 20150517)) hours=() for (( i=0; i<=23;++i)) do if [ $i -lt 10 ]; then hours[i]="0"$i else hours[i]=$i fi done for day in ${days[*]} do for hour in ${hours[*]} do date_dir=${day}${hour}0000 hadoop fs -test -e hdfs://namenode:8020/test/output/bjlt_bak/${date_dir}/part-r-00000 if [ $? -eq 0 ] ;then hql="LOAD DATA INPATH ""'hdfs://namenode:8020/test/output/bjlt_bak/"${date_dir}"/part-r-00000' INTO TABLE middle_table.game_match_result" hive -e "${hql}" echo ${hql} else echo "${date_dir}/part-r-00000 is not exist Or Zero bytes in size" fi done done
hive -e "${hql}"
上邊一行程式碼中的雙引號一定要加上,否則會報<EOF>的錯誤