Hadoop問題：啟動hadoop 2.6遇到的datanode啟動不了
阿新 • • 發佈：2017-10-05
left hand list common status ctc 解決辦法 總結 aac
問題描述：第一次啟動輸入jps都有，第二次沒有datanode
日誌如下：
查看日誌如下: 2014-12-22 12:08:27,264 INFO org.mortbay.log: Started [email protected]0.0.0.0:50075 2014-12-22 12:08:27,692 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = root 2014-12-22 12:08:27,692 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup2014-12-22 12:08:32,865 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-12-22 12:08:32,889 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020 2014-12-22 12:08:32,931 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020 2014-12-22 12:08:32,945 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null 2014-12-22 12:08:32,968 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default> 2014-12-22 12:08:32,992 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:8020 starting to offer service 2014-12-22 12:08:33,001 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2014-12-22 12:08:33,003 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting 2014-12-22 12:08:33,536 INFO org.apache.hadoop.hdfs.server.common.Storage: DataNode version: -56 and NameNode layout version: -60 2014-12-22 12:08:33,699 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /home/hadoop/tmp/dfs/data/in_use.lock acquired by nodename 17247@henry-ThinkPad-T400 2014-12-22 12:08:33,706 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:8020. Exiting. java.io.IOException: Incompatible clusterIDs in /home/hadoop/tmp/dfs/data: namenode clusterID = CID-19f887ba-2e8d-4c7e-ae01-e38a30581693; datanode clusterID = CID-14aac0b3-3c32-45db-adb8-b5fc494eaa3d at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:646) at org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:320) at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:403) at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:422) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1311) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1276) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:314) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:220) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:828) at java.lang.Thread.run(Thread.java:662) 2014-12-22 12:08:33,716 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to localhost/127.0.0.1:8020 2014-12-22 12:08:33,718 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned) 2014-12-22 12:08:35,718 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode 2014-12-22 12:08:35,720 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0 2014-12-22 12:08:35,722 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at henry-ThinkPad-T400/127.0.0.1 ************************************************************/
問題分析：在第一次格式化dfs後，啟動並使用了hadoop，後來又重新執行了格式化命令（hdfs namenode -format)，
這時namenode的clusterID會重新生成，而datanode的clusterID 保持不變。
問題總結：datanode的clusterID 和 namenode的clusterID 不匹配。
解決辦法：根據日誌中的路徑，cd /home/hadoop/tmp/dfs 能看到 data和name兩個文件夾，
將name/current下的VERSION中的clusterID復制到data/current下的VERSION中，覆蓋掉原來的clusterID，讓兩個保持一致
然後重啟，啟動後執行jps，查看進程
Hadoop問題：啟動hadoop 2.6遇到的datanode啟動不了