1. 程式人生 > >mac os x 編譯hadoop

mac os x 編譯hadoop

執行

clean install  package -Pdist -P-cbuild  -DskipTests  -Dtar 

報各種錯誤

1、報錯[ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-7:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]
修改hadoop-common-project/hadoop-common/pom.xml 檔案中,env.JAVA_HOME改為java.home

2、報錯

/hadoop-2.2.0-src/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26: error: invalid operands to binary expression (‘void’ and ‘int’)
[exec] if(setnetgrent(cgroup) == 1) {
[exec] ~~~~~~~~~~~~~~~~~~~ ^ ~
[exec] 1 error generated.
[exec] make[2]: *** [CMakeFiles/hadoop.dir/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c.o] Error 1
[exec] make[1]: *** [CMakeFiles/hadoop.dir/all] Error 2
[exec] make: *** [all] Error 2

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2 -> [Help 1]

修改mvn3的配置檔案:/opt/local/share/java/maven3/settings.xml
在<mirrors>…</mirrors>裡新增國內源:

<mirrors>
<mirror>
<id>nexus-osc</id>
<mirrorOf>*</mirrorOf>
<name>Nexusosc</name>
<url>http://maven.oschina.net/content/groups/public/</url>
</mirror>
</mirrors>

在<profiles>…</profiles>標籤中增加以下內容:
<profile>
<id>jdk-1.7</id>
<activation>
<jdk>1.7<k>
</activation>
<repositories>
<repository>
<id>nexus</id>
<name>local private nexus</name>
<url>http://maven.oschina.net/content/groups/public/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>nexus</id>
<name>local private nexus</name>
<url>http://maven.oschina.net/content/groups/public/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>

注意修改jdk version number

將剛才的maven 配置檔案拷貝到當前使用者的home目錄下:
settings.xml   copy 到  your_hadoop_usr_home/.m2/
cp settings.xml ~/.m2

3、報錯[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project hadoop-hdfs: Fatal error compiling: Error while executing the compiler. InvocationTargetException: Java heap space

分配記憶體不足,參考如下為maven配置JVM引數: export MAVEN_OPTS=”-Xms256m -Xmx512m -Djava.awt.headless=true”

4、報錯 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project hadoop-hdfs: Compilation failure
[ERROR] Failure executing javac, but could not parse the error:

執行maven clean,然後再

export MAVEN_OPTS=”-Xms256m -Xmx512m -Djava.awt.headless=true”
三、最重要的一點,build your code是使用這個command line(Only for Mac OS):
mvn clean install -P-cbuild

編譯之前, 你在hadoop-2.2.0-src目錄(/Users/JuneMAC/hadoop/release-2.2.0)下執行
mvn clean install –DskipTests

上面的成功後,執行下面這個,生成安裝包
mvn clean install  package -Pdist -P-cbuild  -DskipTests  -Dtar

執行完成後,可以在/Users/JuneMAC/hadoop/release-2.2.0/hadoop-dist/target/
下找到
hadoop-2.2.0.tar.gz
將上面這個編譯好的原始碼包解壓到:
/Users/JuneMAC/hadoop/
然後進行相關配置
解壓之後的原始碼包和官網下載下來的原始碼包相對比,沒有lib目錄
相關解釋:
“Here we use the additional options to stop compiling the native code.
this is the key reason why we need use -P-cbuild option”
上面這個是原因,好像不是很重要。實際上如果指定-Pdist,native 生成native lib 不成功,查閱有關官方介紹發現:Hadoop本地庫只支援*nix平臺,已經廣泛使用在GNU/Linux平臺上,但是不支援 Cygwin  和 Mac OS X 。