周海漢 /文

2013.10.17

Hadoop 2.2 是 Hadoop 2 即yarn的第一個穩定版。並且解決單點問題。

maven安裝

解壓後放到/usr/local目錄下。

增加國內maven 開源中國映象

[[email protected] ~]$ sudo vi /usr/local/apache-maven-3.1.1/conf/settings.xml

  <mirror>
<id>nexus-osc</id>
<mirrorOf>*</mirrorOf>
<name>Nexus osc</name>
<url>http://maven.oschina.net/content/groups/public/</url>
</mirror>
 

下載安裝hadoop2.2

[[email protected] ~]$ wget http://mirrors.cnnic.cn/apache/hadoop/common/stable2/hadoop-2.2.0-src.tar.gz

[[email protected] ~]$ cd hadoop-2.2.0-src [[email protected] hadoop-2.2.0-src]$

編譯

[[email protected] hadoop-2.2.0-src]$ mvn clean install -DskipTests

[INFO] — hadoop-maven-plugins:2.2.0:protoc (compile-protoc) @ hadoop-common — [WARNING] [protoc, –version] failed: java.io.IOException: Cannot run program “protoc”: java.io.IOException: error=2, No such file or directory

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: ‘protoc –version’ did not return a version -> [Help 1]

安裝編譯protobuf

[[email protected] ~]$ tar jxvf protobuf-2.5.0.tar.bz2

[[email protected] protobuf-2.5.0]$ ./configure

[[email protected] protobuf-2.5.0]$ make

[[email protected] protobuf-2.5.0]$ protoc –version protoc: error while loading shared libraries: libprotobuf.so.8: cannot open shared object file: No such file or directory

[[email protected] protobuf-2.5.0]# ls /usr/local/lib libhiredis.a        libltdl.a     libltdl.so.3.1.0    libprotobuf-lite.la        libprotobuf.so        libprotoc.la        liby.a libhiredis.so       libltdl.la    libprotobuf.a       libprotobuf-lite.so        libprotobuf.so.8      libprotoc.so        pkgconfig libhiredis.so.0     libltdl.so    libprotobuf.la      libprotobuf-lite.so.8      libprotobuf.so.8.0.0  libprotoc.so.8 libhiredis.so.0.10  libltdl.so.3  libprotobuf-lite.a  libprotobuf-lite.so.8.0.0  libprotoc.a           libprotoc.so.8.0.0

[[email protected] protobuf-2.5.0]# vi /etc/ld.so.conf

include /usr/local/lib

[[email protected] protobuf-2.5.0]$ protoc –version libprotoc 2.5.0

[[email protected] hadoop-2.2.0-src]$ mvn install -DskipTests

[INFO] Apache Hadoop Main ………………………….. SUCCESS [0.947s] [INFO] Apache Hadoop Project POM ……………………. SUCCESS [0.294s] [INFO] Apache Hadoop Annotations ……………………. SUCCESS [0.474s] [INFO] Apache Hadoop Project Dist POM ……………….. SUCCESS [0.287s] [INFO] Apache Hadoop Assemblies …………………….. SUCCESS [0.106s] [INFO] Apache Hadoop Maven Plugins ………………….. SUCCESS [0.937s] [INFO] Apache Hadoop Auth ………………………….. SUCCESS [0.248s] [INFO] Apache Hadoop Auth Examples ………………….. SUCCESS [0.318s] [INFO] Apache Hadoop Common ………………………… SUCCESS [17.582s] [INFO] Apache Hadoop NFS …………………………… SUCCESS [1.364s] [INFO] Apache Hadoop Common Project …………………. SUCCESS [0.016s] [INFO] Apache Hadoop HDFS ………………………….. SUCCESS [39.854s] [INFO] Apache Hadoop HttpFS ………………………… SUCCESS [1.544s] [INFO] Apache Hadoop HDFS BookKeeper Journal …………. SUCCESS [1.494s] [INFO] Apache Hadoop HDFS-NFS ………………………. SUCCESS [0.189s] [INFO] Apache Hadoop HDFS Project …………………… SUCCESS [0.017s] [INFO] hadoop-yarn ………………………………… SUCCESS [5.859s] [INFO] hadoop-yarn-api …………………………….. SUCCESS [2.837s] [INFO] hadoop-yarn-common ………………………….. SUCCESS [1.263s] [INFO] hadoop-yarn-server ………………………….. SUCCESS [0.045s] [INFO] hadoop-yarn-server-common ……………………. SUCCESS [0.458s] [INFO] hadoop-yarn-server-nodemanager ……………….. SUCCESS [0.776s] [INFO] hadoop-yarn-server-web-proxy …………………. SUCCESS [0.192s] [INFO] hadoop-yarn-server-resourcemanager ……………. SUCCESS [0.952s] [INFO] hadoop-yarn-server-tests …………………….. SUCCESS [0.150s] [INFO] hadoop-yarn-client ………………………….. SUCCESS [0.239s] [INFO] hadoop-yarn-applications …………………….. SUCCESS [0.032s] [INFO] hadoop-yarn-applications-distributedshell ……… SUCCESS [0.155s] [INFO] hadoop-mapreduce-client ……………………… SUCCESS [0.028s] [INFO] hadoop-mapreduce-client-core …………………. SUCCESS [1.472s] [INFO] hadoop-yarn-applications-unmanaged-am-launcher …. SUCCESS [0.124s] [INFO] hadoop-yarn-site ……………………………. SUCCESS [0.047s] [INFO] hadoop-yarn-project …………………………. SUCCESS [1.431s] [INFO] hadoop-mapreduce-client-common ……………….. SUCCESS [1.460s] [INFO] hadoop-mapreduce-client-shuffle ………………. SUCCESS [0.140s] [INFO] hadoop-mapreduce-client-app ………………….. SUCCESS [0.718s] [INFO] hadoop-mapreduce-client-hs …………………… SUCCESS [0.320s] [INFO] hadoop-mapreduce-client-jobclient …………….. SUCCESS [1.065s] [INFO] hadoop-mapreduce-client-hs-plugins ……………. SUCCESS [0.104s] [INFO] Apache Hadoop MapReduce Examples ……………… SUCCESS [0.292s] [INFO] hadoop-mapreduce ……………………………. SUCCESS [0.035s] [INFO] Apache Hadoop MapReduce Streaming …………….. SUCCESS [0.243s] [INFO] Apache Hadoop Distributed Copy ……………….. SUCCESS [31.506s] [INFO] Apache Hadoop Archives ………………………. SUCCESS [0.138s] [INFO] Apache Hadoop Rumen …………………………. SUCCESS [0.296s] [INFO] Apache Hadoop Gridmix ……………………….. SUCCESS [0.330s] [INFO] Apache Hadoop Data Join ……………………… SUCCESS [0.132s] [INFO] Apache Hadoop Extras ………………………… SUCCESS [0.182s] [INFO] Apache Hadoop Pipes …………………………. SUCCESS [0.011s] [INFO] Apache Hadoop Tools Dist …………………….. SUCCESS [0.185s] [INFO] Apache Hadoop Tools …………………………. SUCCESS [0.011s] [INFO] Apache Hadoop Distribution …………………… SUCCESS [0.043s] [INFO] Apache Hadoop Client ………………………… SUCCESS [0.106s] [INFO] Apache Hadoop Mini-Cluster …………………… SUCCESS [0.054s] [INFO] ———————————————————————— [INFO] BUILD SUCCESS [INFO] ———————————————————————— [INFO] Total time: 2:00.410s [INFO] Finished at: Thu Oct 17 15:26:18 CST 2013 [INFO] Final Memory: 95M/1548M

如非註明轉載, 均為原創. 本站遵循知識共享CC協議,轉載請註明來源