異常解決:util.NativeCodeLoader: Unable to load native-hadoop library for your platform
剛裝好Hadoop的時候,每次輸入命令執行都會出現:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-Java classes where applicable
提示hadoop不能載入本地庫。
以前官網會提供32位的安裝包,但目前提供的下載包為64位的,在Linux 32位系統下安裝後會一直提示錯誤“WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-
首先下載hadoop-native-64-2.4.0.tar:
http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.4.0.tar
如果你是hadoop2.6的可以下載下面這個:
http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.6.0.tar
tar -xvf hadoop-native-64-2.4.0.tar -C hadoop/lib/native/
大家可以到$HADOOP_HOME/lib/native
下,檢視檔案libhadoop.so.1.0.0,裡面會顯示Hadoop的位數,官網下載好的顯示是64位的,截圖如下:
如果你是64位作業系統,可以直接跳到第二步。
本地編譯原始碼
具體操作看前一篇,這裡
最後編譯完以後,目標在
hadoop-2.7.1-src/hadoop-dist/target/hadoop-2.7.1
下,將此目錄下的lib/native
替換掉$HADOOP_HOME/lib/native
。
配置環境變數
在/etc/profile中,新增下面配置:
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_HOME=/home/hadoop/labc/hadoop-2.7.1
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib:$HADOOP_COMMON_LIB_NATIVE_DIR"
#export HADOOP_ROOT_LOGGER=DEBUG,console #除錯錯誤
- 1
- 2
- 3
- 1
- 2
- 3
最後記得使配置生效:source /etc/profile
並把相同配置新增到hadoop-env.sh檔案末尾。
輸入如下命令,沒有報warn,說明修改成功。
轉:http://blog.csdn.net/young_kim1/article/details/50324345
但是通過上面操作後還是報錯,通過除錯資訊發現是libc版本太低
17/03/04 15:06:01 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/03/04 15:06:01 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /home/grid/soft/hadoop-2.6.0-cdh5.10.0/lib/libhadoop.so: /lib64/libc.so.6: version `GLIBC_2.6' not found (required by /home/grid/soft/hadoop-2.6.0-cdh5.10.0/lib/libhadoop.so)
17/03/04 15:06:01 DEBUG util.NativeCodeLoader: java.library.path=/home/grid/soft/hadoop-2.6.0-cdh5.10.0/lib:/home/grid/soft/hadoop-2.6.0-cdh5.10.0/lib/native
17/03/04 15:06:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/03/04 15:06:01 DEBUG util.PerformanceAdvisory: Falling back to shell based
17/03/04 15:06:01 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
解決辦法參考:
http://blog.csdn.net/cpplang/article/details/8462768/
http://blog.pengduncun.com/?p=1194
附加知識:
tar xvJf ***.tar.xz來解壓