兩個hadoop的問題解決
阿新 • • 發佈：2019-02-10
org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 times出現這個問題多是由於namenode和datanode的資料不一致的問題。最近做了幾個叢集，嘗試了幾種可能出現此問題的情況，大概是因為格式化或者遷移hdfs造成的。解決辦法比較簡單：
${hadoop.tmp.dir}/dfs/name/current下的VERSION檔案內的namespaceID=13837527要與${hadoop.tmp.dir}/dfs/data/current下的VERSION檔案內容一致.停掉hbase和hadoop之後再改這個。發現資料夾時空的話，可以格式化一下hdfs就出現應有的目錄檔案格式了。
java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:485) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 9 more Caused by: java.lang.NoClassDefFoundError: org/apache/zookeeper/KeeperException at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:185) at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:154) at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:132) at com.successfactors.perflog.hadoop.hbase.HBaseUtils.getHBTable(Unknown Source) at com.successfactors.perflog.hadoop.mapred.core.ParseRecordToHB$RecordReducer.configure(Unknown Source) ... 14 more Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.KeeperException at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 19 more
這個問題倒是困擾好久的，試過重置hdfs的路徑（hadoop.tmp.dir）、試過格式化hdfs（hadoop namenode -format）、試過增大系統併發處理檔案的數量（dfs.datanode.max.xcievers）、重新編譯過要執行的MR任務都沒有解決。還是要認真看log啊。
網上出現著個問題的情況千奇百怪，認真看Caused by先！
第一個：
Caused by: java.lang.reflect.InvocationTargetException這個情況可能在很多java的環境下出現，先不考慮；
第二個：
Caused by: java.lang.NoClassDefFoundError: org/apache/zookeeper/KeeperException
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_HOME/hbase-0.94.8-security.jar:$HBASE_HOME/lib/zookeeper-3.4.5.jar:$HBASE_HOME/lib/protobuf-java-2.4.0a.jar:$HBASE_HOME/lib/guava-11.0.2.jar:$HBASE_HOME/conf抱著試試看的心情，新增一個lib的路徑，結果成功了！可是之前zookeeper的路徑沒寫錯啊！用網上的話說真是“莫名其妙地找不到jar包”
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_HOME/hbase-0.94.8-security.jar:$HBASE_HOME/lib/zookeeper-3.4.5.jar:$HBASE_HOME/lib/protobuf-java-2.4.0a.jar:$HBASE_HOME/lib/guava-11.0.2.jar:$HBASE_HOME/lib:$HBASE_HOME/conf
總之認真看錯誤報告再分析錯誤就是了
http://hadoop-server-01:50030/