7.測試hadoop安裝成功與否，並跑mapreduce例項

阿新 • • 發佈：2019-01-09

start-all.sh 2.建立hdfs目錄 hadoop fs -mkdir /input 3.上傳檔案 hadoop fs -put /data/hadoop-2.6.5/README.txt /input/ 4.修改檔名稱 hadoop fs -mv /input/README.txt /input/readme.txt 5.檢視檔案 hadoop fs -ls /input 執行輸出情況見下： [email protected]:~$ hadoop fs -ls /input Found 1 items -rw-r--r-- 3 hadoop supergroup 1366

2017-08-13 19:58 /input/readme.txt 【註解】輸出資料夾為output，無需新建，若已存在需刪除 6.執行hadoop自帶例子 hadoop jar /data/hadoop-2.6.5/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar wordcount /input /output 執行輸出情況見下： [email protected]:~$ hadoop jar /data/hadoop-2.6.5/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6

.5.jar wordcount /input /output 17/08/13 20:11:18 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.222.139:8032 17/08/13 20:11:21 INFO input.FileInputFormat: Total input paths to process : 1 17/08/13 20:11:21 INFO mapreduce.JobSubmitter: number of splits:1 17/08/13 20:11:22 INFO mapreduce.JobSubmitter: Submitting tokens for

job: job_1502625091562_0001 17/08/13 20:11:23 INFO impl.YarnClientImpl: Submitted application application_1502625091562_0001 17/08/13 20:11:23 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1502625091562_0001/ 17/08/13 20:11:23 INFO mapreduce.Job: Running job: job_1502625091562_0001 17/08/13 20:11:45 INFO mapreduce.Job: Job job_1502625091562_0001 running in uber mode : false 17/08/13 20:11:45 INFO mapreduce.Job: map 0% reduce 0% 17/08/13 20:11:59 INFO mapreduce.Job: map 100% reduce 0% 17/08/13 20:12:29 INFO mapreduce.Job: map 100% reduce 100% 17/08/13 20:12:30 INFO mapreduce.Job: Job job_1502625091562_0001 completed successfully 17/08/13 20:12:30 INFO mapreduce.Job: Counters: 49 File System Counters FILE: Number of bytes read=1836 FILE: Number of bytes written=218883 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=1466 HDFS: Number of bytes written=1306 HDFS: Number of read operations=6 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 Job Counters Launched map tasks=1 Launched reduce tasks=1 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=11022 Total time spent by all reduces in occupied slots (ms)=26723 Total time spent by all map tasks (ms)=11022 Total time spent by all reduce tasks (ms)=26723 Total vcore-milliseconds taken by all map tasks=11022 Total vcore-milliseconds taken by all reduce tasks=26723 Total megabyte-milliseconds taken by all map tasks=11286528 Total megabyte-milliseconds taken by all reduce tasks=27364352 Map-Reduce Framework Map input records=31 Map output records=179 Map output bytes=2055 Map output materialized bytes=1836 Input split bytes=100 Combine input records=179 Combine output records=131 Reduce input groups=131 Reduce shuffle bytes=1836 Reduce input records=131 Reduce output records=131 Spilled Records=262 Shuffled Maps =1 Failed Shuffles=0 Merged Map outputs=1 GC time elapsed (ms)=245 CPU time spent (ms)=2700 Physical memory (bytes) snapshot=291491840 Virtual memory (bytes) snapshot=3782098944 Total committed heap usage (bytes)=138350592 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=1366 File Output Format Counters Bytes Written=1306 7.檢視檔案輸出結果 hadoop fs -ls /output 執行輸出情況見下： [email protected]:~$ hadoop fs -ls /output Found 2 items -rw-r--r-- 3 hadoop supergroup 0 2017-08-13 20:12 /output/_SUCCESS -rw-r--r-- 3 hadoop supergroup 1306 2017-08-13 20:12 /output/part-r-00000 8.檢視詞頻統計結果 hadoop fs -cat /output/part-r-00000 執行輸出情況見下： [email protected]:~$ hadoop fs -cat /output/part-r-00000 (BIS), 1 (ECCN) 1 (TSU) 1 (see 1 5D002.C.1, 1 740.13) 1 <http://www.wassenaar.org/> 1 Administration 1 Apache 1 BEFORE 1 BIS 1 Bureau 1 Commerce, 1 Commodity 1 Control 1 Core 1 Department 1 ENC 1 Exception 1 Export 2 For 1 Foundation 1 Government 1 Hadoop 1 Hadoop, 1 Industry 1 Jetty 1 License 1 Number 1 Regulations, 1 SSL 1 Section 1 Security 1 See 1 Software 2 Technology 1 The 4 This 1 U.S. 1 Unrestricted 1 about 1 algorithms. 1 and 6 and/or 1 another 1 any 1 as 1 asymmetric 1 at: 2 both 1 by 1 check 1 classified 1 code 1 code. 1 concerning 1 country 1 country's 1 country, 1 cryptographic 3 currently 1 details 1 distribution 2 eligible 1 encryption 3 exception 1 export 1 following 1 for 3 form 1 from 1 functions 1 has 1 have 1 http://hadoop.apache.org/core/ 1 http://wiki.apache.org/hadoop/ 1 if 1 import, 2 in 1 included 1 includes 2 information 2 information. 1 is 1 it 1 latest 1 laws, 1 libraries 1 makes 1 manner 1 may 1 more 2 mortbay.org. 1 object 1 of 5 on 2 or 2 our 2 performing 1 permitted. 1 please 2 policies 1 possession, 2 project 1 provides 1 re-export 2 regulations 1 reside 1 restrictions 1 security 1 see 1 software 2 software, 2 software. 2 software: 1 source 1 the 8 this 3 to 2 under 1 use, 2 uses 1 using 2 visit 1 website 1 which 2 wiki, 1 with 1 written 1 you 1 your 1 9.將hdfs上檔案匯出到本地【註解】先在/home/hadoop/下新建一個/home/hadoop/example目錄用於接受產生的檔案 su hadoop mkdir /home/hadoop/example 再執行： [email protected]:~$ hadoop fs -get /output/part-r-00000 /home/hadoop/example 執行完成後，在/home/hadoop/example目錄下生成part-r-00000檔案，見下圖：此時測試成功，即安裝Hadoop並跑例項成功。

7.測試hadoop安裝成功與否，並跑mapreduce例項

7.測試hadoop安裝成功與否，並跑mapreduce例項

mysql5.7版本之後安裝成功後，服務無法啟動，沒有報告任何錯誤的解決辦法。

Hadoop安裝成功之後，訪問不了web介面的50070埠怎麼解決？

ubuntu16 安裝完畢，測試是否安裝成功的步驟

CentOS 7.4 yum方式安裝LAMP環境，並部署WordPress，使用XCache優化性能

在mac上安裝python-opencv，並在pycharm測試

002，FastDFS 安裝成功後，如何測試是否能正常使用

centos 7 安裝gitlab伺服器，並訪問詳細步驟

在centos7上安裝ClamAV防毒，並防毒（centos隨機英文10字母）成功

CentOS 6.9 yum方式安裝LAMP環境，並部署Discuz

解決app安裝成功後，直接點擊“打開”再按home返回，再次打開app會重新啟動的問題

windows 10安裝hyper-v，並創建虛擬機

ubuntu16.04安裝ssh服務，並實現遠程訪問

Ubuntu安裝Chrome瀏覽器，並解決Chrome瀏覽器無法啟動問題

centos安裝成功後，不能正常使用ifconfig命令

檢測已安裝軟件，並保存在桌面

Ubuntu12.04中的虛擬機安裝Ubuntu16.04，並實現遠程控制16.04

安裝Navicat premium，並激活Navicat premium，並解決Navicat premium連線不上oracle的問題

linux下安裝anaconda教程，並新增清華映象

在CentOS 7 1804 中安裝 Apache HTTP Server 並配置阿里雲 SSL

7.測試hadoop安裝成功與否，並跑mapreduce例項

相關推薦