Yarn命令使用及wordcount解析
阿新 • • 發佈：2018-05-30
Hadoop Yarn 前言：
前面幾篇博客主要介紹了MapReduce與Yarn的架構設計及簡單工作流程，本篇文章將以wordcount程序為例，簡單介紹下Yarn的使用。
1.wordcount示例運行
[root@hadoop000 ~]# su - hadoop [hadoop@hadoop000 ~]$ jps 9201 SecondaryNameNode 9425 ResourceManager 13875 Jps 9540 NodeManager 8852 NameNode 8973 DataNode # 創建wordcount目錄 [hadoop@hadoop000 ~]$ hdfs dfs -mkdir -p /wordcount/input [hadoop@hadoop000 ~]$ vi test.log jepson ruoze hero yimi xjp 123 a b a [hadoop@hadoop000 ~]$ hdfs dfs -put test.log /wordcount/input [hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/input Found 1 items -rw-r--r-- 1 hadoop supergroup 37 2018-05-29 20:38 /wordcount/input/test.log # 執行wordcount示例jar包 [hadoop@hadoop000 ~]$ yarn jar > /opt/software/hadoop-2.8.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar > wordcount > /wordcount/input > /wordcount/output 18/05/29 20:40:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 18/05/29 20:40:59 INFO input.FileInputFormat: Total input files to process : 1 18/05/29 20:41:00 INFO mapreduce.JobSubmitter: number of splits:1 18/05/29 20:41:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1526991305992_0001 18/05/29 20:41:01 INFO impl.YarnClientImpl: Submitted application application_1526991305992_0001 18/05/29 20:41:01 INFO mapreduce.Job: The url to track the job: http://hadoop000:8088/proxy/application_1526991305992_0001/ 18/05/29 20:41:01 INFO mapreduce.Job: Running job: job_1526991305992_0001 18/05/29 20:41:14 INFO mapreduce.Job: Job job_1526991305992_0001 running in uber mode : false 18/05/29 20:41:14 INFO mapreduce.Job: map 0% reduce 0% 18/05/29 20:41:23 INFO mapreduce.Job: map 100% reduce 0% 18/05/29 20:41:29 INFO mapreduce.Job: map 100% reduce 100% 18/05/29 20:41:30 INFO mapreduce.Job: Job job_1526991305992_0001 completed successfully 18/05/29 20:41:30 INFO mapreduce.Job: Counters: 49 # 查看結果 [hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/output Found 2 items -rw-r--r-- 1 hadoop supergroup 0 2018-05-29 20:41 /wordcount/output/_SUCCESS -rw-r--r-- 1 hadoop supergroup 51 2018-05-29 20:41 /wordcount/output/part-r-00000 [hadoop@hadoop000 ~]$ hdfs dfs -cat /wordcount/output/part-r-00000 123 1 a 2 b 1 hero 1 jepson 1 ruoze 1 xjp 1 yimi 1
登錄網頁查看相關信息：http://192.168.6.217:8088/cluster
2.Yarn常用命令總結
yarn jar <jar> --run a jar file
yarn application -list --列出在跑的job
yarn application -kill application_1526991305992_0001（job的id） --殺掉在跑的job3.wordcount流程詳解
參考：https://blog.csdn.net/yczws1/article/details/21794873
Yarn命令使用及wordcount解析