周海漢 2012.12.26 http://abloz.com

以前寫過一篇《yarn hadoop mapreduce 2.0 編譯》，環境準備和那一篇一樣。 本篇直接從svn下載最新原始碼，進行編譯。

下載原始碼： 從subversion庫check out： [[email protected] hsrc]$ svn co http://svn.apache.org/repos/asf/hadoop/common/trunk

[[email protected] hsrc]$ cd trunk/ [[email protected] trunk]$ ls BUILDING.txt hadoop-assemblies hadoop-common-project hadoop-hdfs-project hadoop-minicluster hadoop-project-dist hadoop-yarn-project dev-support hadoop-client hadoop-dist hadoop-mapreduce-project hadoop-project hadoop-tools pom.xml

Maven 主要模組:

hadoop (Main Hadoop project) - hadoop-project (Parent POM for all Hadoop Maven modules. ) (All plugins & dependencies versions are defined here.) - hadoop-project-dist (Parent POM for modules that generate distributions.) - hadoop-annotations (Generates the Hadoop doclet used to generated the Javadocs) - hadoop-assemblies (Maven assemblies used by the different modules) - hadoop-common-project (Hadoop Common) - hadoop-hdfs-project (Hadoop HDFS) - hadoop-mapreduce-project (Hadoop MapReduce) - hadoop-tools (Hadoop tools like Streaming, Distcp, etc.) - hadoop-dist (Hadoop distribution assembler)

編譯：

[[email protected] trunk]$ mvn compile -DskipTest … [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Main ………………………….. SUCCESS [0.605s] [INFO] Apache Hadoop Project POM ……………………. SUCCESS [0.558s] [INFO] Apache Hadoop Annotations ……………………. SUCCESS [0.288s] [INFO] Apache Hadoop Project Dist POM ……………….. SUCCESS [0.094s] [INFO] Apache Hadoop Assemblies …………………….. SUCCESS [0.088s] [INFO] Apache Hadoop Auth ………………………….. SUCCESS [0.152s] [INFO] Apache Hadoop Auth Examples ………………….. SUCCESS [0.093s] [INFO] Apache Hadoop Common ………………………… SUCCESS [5.188s] [INFO] Apache Hadoop Common Project …………………. SUCCESS [0.049s] [INFO] Apache Hadoop HDFS ………………………….. SUCCESS [12.065s] [INFO] Apache Hadoop HttpFS ………………………… SUCCESS [0.194s] [INFO] Apache Hadoop HDFS BookKeeper Journal …………. SUCCESS [0.616s] [INFO] Apache Hadoop HDFS Project …………………… SUCCESS [0.029s] [INFO] hadoop-yarn ………………………………… SUCCESS [0.157s] [INFO] hadoop-yarn-api …………………………….. SUCCESS [2.951s] [INFO] hadoop-yarn-common ………………………….. SUCCESS [0.752s] [INFO] hadoop-yarn-server ………………………….. SUCCESS [0.124s] [INFO] hadoop-yarn-server-common ……………………. SUCCESS [0.736s] [INFO] hadoop-yarn-server-nodemanager ……………….. SUCCESS [0.592s] [INFO] hadoop-yarn-server-web-proxy …………………. SUCCESS [0.123s] [INFO] hadoop-yarn-server-resourcemanager ……………. SUCCESS [0.200s] [INFO] hadoop-yarn-server-tests …………………….. SUCCESS [0.149s] [INFO] hadoop-yarn-client ………………………….. SUCCESS [0.119s] [INFO] hadoop-yarn-applications …………………….. SUCCESS [0.090s] [INFO] hadoop-yarn-applications-distributedshell ……… SUCCESS [0.167s] [INFO] hadoop-mapreduce-client ……………………… SUCCESS [0.049s] [INFO] hadoop-mapreduce-client-core …………………. SUCCESS [1.103s] [INFO] hadoop-yarn-applications-unmanaged-am-launcher …. SUCCESS [0.142s] [INFO] hadoop-yarn-site ……………………………. SUCCESS [0.082s] [INFO] hadoop-yarn-project …………………………. SUCCESS [0.075s] [INFO] hadoop-mapreduce-client-common ……………….. SUCCESS [1.202s] [INFO] hadoop-mapreduce-client-shuffle ………………. SUCCESS [0.066s] [INFO] hadoop-mapreduce-client-app ………………….. SUCCESS [0.109s] [INFO] hadoop-mapreduce-client-hs …………………… SUCCESS [0.123s] [INFO] hadoop-mapreduce-client-jobclient …………….. SUCCESS [0.114s] [INFO] hadoop-mapreduce-client-hs-plugins ……………. SUCCESS [0.084s] [INFO] Apache Hadoop MapReduce Examples ……………… SUCCESS [0.130s] [INFO] hadoop-mapreduce ……………………………. SUCCESS [0.060s] [INFO] Apache Hadoop MapReduce Streaming …………….. SUCCESS [0.071s] [INFO] Apache Hadoop Distributed Copy ……………….. SUCCESS [0.069s] [INFO] Apache Hadoop Archives ………………………. SUCCESS [0.061s] [INFO] Apache Hadoop Rumen …………………………. SUCCESS [0.135s] [INFO] Apache Hadoop Gridmix ……………………….. SUCCESS [0.082s] [INFO] Apache Hadoop Data Join ……………………… SUCCESS [0.070s] [INFO] Apache Hadoop Extras ………………………… SUCCESS [0.192s] [INFO] Apache Hadoop Pipes …………………………. SUCCESS [0.019s] [INFO] Apache Hadoop Tools Dist …………………….. SUCCESS [0.057s] [INFO] Apache Hadoop Tools …………………………. SUCCESS [0.018s] [INFO] Apache Hadoop Distribution …………………… SUCCESS [0.047s] [INFO] Apache Hadoop Client ………………………… SUCCESS [0.047s] [INFO] Apache Hadoop Mini-Cluster …………………… SUCCESS [0.053s] [INFO] ———————————————————————— [INFO] BUILD SUCCESS [INFO] ———————————————————————— [INFO] Total time: 32.093s [INFO] Finished at: Wed Dec 26 11:00:10 CST 2012 [INFO] Final Memory: 60M/769M

打包命令： 建立二進位制分發版，不帶native code和文件： [[email protected] trunk]$ mvn package -Pdist -DskipTests -Dtar 建立二進位制分發版，帶native code和文件： [[email protected] trunk]$ mvn package -Pdist,native,docs -DskipTests -Dtar 建立原始碼分發版 [[email protected] trunk]$ mvn package -Psrc -DskipTests 建立二進位制帶原始碼分發版，帶native code和文件： [[email protected] trunk]$ mvn package -Pdist,native,docs,src -DskipTests -Dtar 建立本地版web頁面，放在/tmp/hadoop-site [[email protected] trunk]$ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site

參考 http://svn.apache.org/repos/asf/hadoop/common/trunk/BUILDING.txt

如非註明轉載, 均為原創. 本站遵循知識共享CC協議,轉載請註明來源