hadoop學習(7)—— 使用yarn執行mapreduce一個簡單的wordcount示例
阿新 • • 發佈:2019-01-05
1.hdfs檔案系統目錄要求(建議)
/user
/{username} --使用者名稱
/mr --MapReduce型別應用
/wordcount --應用的名稱
/input --要處理的檔案
/output --輸出結果的資料夾
2.按照目錄格式建立輸入資料夾(不用建立輸出資料夾)
[root@hadoop-yarn hadoop-2.6 .5]# bin/hdfs dfs -mkdir -p /user/root/mr/wordcount/input
3.手動建立測試資料
cyhp supergroup
hadoop hello
hdfs hadoop
mapreduce hadoop
world yarn
4.將輸入資料放入HDFS中
[root@hadoop-yarn hadoop-2.6.5]# bin/hdfs dfs -put test-datas/wc.input /user/root/mr/wordcount/input/
5.使用yarn執行示例
[root@hadoop-yarn hadoop-2.6 .5]# bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar wordcount /user/root/mr/wordcount/input/ /user/root/mr/wordcount/output/
6.檢視輸出
[root@hadoop-yarn hadoop-2.6.5]# bin/hdfs dfs -text /user/root/mr/wordcount/output/par*
[root@hadoop-yarn hadoop-2.6.5]# bin/hdfs dfs -text /user/root/mr/wordcount/output/par*
cyhp 1
hadoop 3
hdfs 1
hello 1
mapreduce 1
supergroup 1
world 1
yarn 1