原因：DataNode 47系統cpu過高重啟後，出現下述問題。 [[email protected] ~]$ fs -rm 1.txt rm: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /user/zhouhh/1.txt. Name node is in safe mode. 進入了安全模式？ 重啟hadoop也不能解決。只好強制離開安全模式。

 [[email protected] hadoop-1.0.3]$ hadoop dfsadmin -safemode leave
 [
 
[email protected] ~]$ fs -rm 1.txt
 Deleted hdfs://Hadoop48:54310/user/zhouhh/1.txt

刪除檔案成功。

 [[email protected] ~]$ hadoop fs -put 1.txt .
 12/06/18 18:14:53 INFO hdfs.DFSClient: Exception in createBlockOutputStream 192.168.10.46:50010 java.io.IOException: Bad connect ack with firstBadLink as 192.168.10.47:54313
 12/06/18 18:14:53 INFO hdfs.DFSClient: Abandoning block blk_6739152855033658770_1512
 12/06/18 18:14:53 INFO hdfs.DFSClient: Excluding datanode 192.168.10.47:54313
 到47上看，jps 程序都在。
 [
 
[email protected] ~]$ jps
 4067 HQuorumPeer
 3913 TaskTracker
 3803 DataNode
 9951 Jps
 4216 HRegionServer
 檢視埠，也都在監聽：
 [[email protected] ~]$ netstat -antp |grep 543
 (Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
 tcp 0 0 0.0.0.0:54313 0.0.0.0:* LISTEN 3803/java
 tcp 0 0 0.0.0.0:54314 0.0.0.0:* LISTEN 3803/java
 tcp 0 0 0.0.0.0:54315 0.0.0.0:* LISTEN 3803/java
 tcp 0 0 192.168.10.47:48636 192.168.10.48:54310 TIME_WAIT -
 tcp 0 0 192.168.10.47:48639 192.168.10.48:54310 TIME_WAIT -
 tcp 0 0 192.168.10.47:59779 192.168.10.48:54311 ESTABLISHED 3913/java
 tcp 0 0 192.168.10.47:55475 192.168.10.48:54310 ESTABLISHED 3803/java
 

重啟hadoop，沒有解決問題。困惑了很久，忽然想到了防火牆。 有時系統沒有配好防火牆，重啟後導致防火牆遮蔽了hadoop的埠。測試時可以將防火牆全部關閉。

[[email protected] ~]# iptables -L
 Chain INPUT (policy ACCEPT)
 target prot opt source destination
 RH-Firewall-1-INPUT all -- anywhere anywhere




Chain FORWARD (policy ACCEPT)
 target prot opt source destination
 RH-Firewall-1-INPUT all -- anywhere anywhere




Chain OUTPUT (policy ACCEPT)
 target prot opt source destination




Chain RH-Firewall-1-INPUT (2 references)
 target prot opt source destination
 ACCEPT all -- anywhere anywhere
 ACCEPT icmp -- anywhere anywhere icmp any
 ACCEPT esp -- anywhere anywhere
 ACCEPT ah -- anywhere anywhere
 ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns
 ACCEPT udp -- anywhere anywhere udp dpt:ipp
 ACCEPT tcp -- anywhere anywhere tcp dpt:ipp
 ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
 REJECT all -- anywhere anywhere reject-with icmp-host-prohibited

關閉防火牆：

[[email protected] ~]# service iptables stop
 Flushing firewall rules: [ OK ]
 Setting chains to policy ACCEPT: filter [ OK ]
 Unloading iptables modules: [ OK ]
 [[email protected] ~]# chkconfig iptables off

在生產系統中再配置域外的防火牆。

[[email protected] ~]$ fs -put 1.txt . 成功

如非註明轉載, 均為原創. 本站遵循知識共享CC協議,轉載請註明來源