Hadoop多次格式化導致datanode無法啟動
阿新 • • 發佈:2019-02-15
首先看一下報錯資訊:
017-12-14 05:07:57,636 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
at org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:409)
at org.apache.hadoop.hdfs.server.datanode.DataStorage .addStorageLocations(DataStorage.java:388)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:556)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1566)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool (DataNode.java:1527)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:327)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:266)
at org.apache.hadoop.hdfs.server.datanode .BPServiceActor.run(BPServiceActor.java:746)
at java.lang.Thread.run(Thread.java:745)
2017-12-14 05:07:58,922 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid 63404450-ed85-4636-8eac-ea75dba1d424) service to hadoop/192.168.137.5:9000. Exiting.
java.io.IOException: All specified directories are failed to load.
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:557)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1566)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1527)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:327)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:266)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:746)
at java.lang.Thread.run(Thread.java:745)
這裡面的日誌有一句報錯非常重要
Incompatible clusterIDs in /tmp/hadoop-hadoop/dfs/data: namenode clusterID = CID-c80f243c-4a07-43f3-9eb8-f40d164a4520; datanode clusterID = CID-3e6fcd99-a2fe-42f3-9ccf-bc257a065eb3
這句話告訴我們,namenode的clusterID和datanode的clusterID不同,導致其無法啟動。原因是我們多次格式化namenode導致兩者id不同,無法啟動。
解決方案:
根據日誌,找到存放clusterID的位置i,根據我的日誌位置為/tmp/hadoop-hadoop/dfs/data
hadoop:hadoop:/tmp/hadoop-hadoop/dfs/name:>ll
total 8
drwxrwxr-x 4 hadoop hadoop 4096 Dec 14 05:35 current
-rw-rw-r-- 1 hadoop hadoop 11 Dec 16 05:41 in_use.lock
將current目錄的VERSION中的clusterID複製,覆蓋到cd /tmp/hadoop-hadoop/dfs/data/current/VERSION的clusterID
使兩個的clusterID相同,就可以解決了