hadoop  fsck

Usage: DFSck <path> [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]
        <path>             檢查這個目錄中的檔案是否完整

        -move               破損的檔案移至/lost+found目錄
        -delete             刪除破損的檔案

        -openforwrite   列印正在開啟寫操作的檔案

        -files                 列印正在check的檔名

        -blocks             列印block報告 （需要和-files引數一起使用）

        -locations         列印每個block的位置資訊（需要和-files引數一起使用）

        -racks               列印位置資訊的網路拓撲圖 （需要和-files引數一起使用）

hadoop  fsck /

用這個命令可以檢查整個檔案系統的健康狀況，但是要注意它不會主動恢復備份缺失的block，這個是由NameNode單獨的執行緒非同步處理的。

....................................................................................................
.................................
/user/distribute-hadoop-boss/tmp/pgv/20090813/1000000103/input/JIFEN.QQ.COM.2009-08-13-18.30:  Replica placement policy is violated for blk_7596595208988121840_5377589. Block should be additionally replicated on 1 more rack(s).
....................................................
/user/distribute-hadoop-boss/tmp/pgv/20090813/1000000310/input/PAY.QQ.COM.2009-08-13-20.30:  Replica placement policy is violated for blk_8146588794511444453_5379501. Block should be additionally replicated on 1 more rack(s).
...............
....................................................................................................
....................................................................................................
.........................................................................................Status: HEALTHY

 Total size:    5042961147529 B (Total open files size: 1610612736 B)
 Total dirs:    723
 Total files:   128089 (Files currently being written: 2)
 Total blocks (validated):      171417 (avg. block size 29419259 B) (Total open file blocks (not validated): 24)
 Minimally replicated blocks:   171417 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         476 (0.2776854 %)
 Default replication factor:    3

  預設的備份引數3
 Average block replication:     3.000146
 Corrupt blocks:                0     破損的block數0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          107
 Number of racks:               4

The filesystem under path '/' is HEALTHY

 hadoop  fsck /user/distribute-hadoop-boss/tmp/pgv/20090813/1000000103/input/JIFEN.QQ.COM.2009-08-13-18.30 -files -blocks -locations  -racks

打印出了這個檔案每個block的詳細資訊包括datanode的機架資訊。

/user/distribute-hadoop-boss/tmp/pgv/20090813/1000000103/input/JIFEN.QQ.COM.2009-08-13-18.30 74110492 bytes, 2 block(s):  Replica placement policy is violated for blk_7596595208988121840_5377589. Block should be additionally replicated on 1 more rack(s).  這個block雖然有三份拷貝，但是都在一個rack裡，應該有一個副本放在不同的機架，詳細見上一節（副本放置策略）

0. blk_-4839761191731553520_5377588 len=67108864 repl=3 [/lg/dminterface0/172.16.236.158:50010, /lg/dminterface1/172.16.218.108:50010, /lg/dminterface1/172.16.236.36:50010]

1. blk_7596595208988121840_5377589 len=7001628 repl=3  [/lg/dminterface2/172.16.236.51:50010, /lg/dminterface2/172.16.218.217:50010, /lg/dminterface2/172.16.218.200:50010]

三份拷貝的datanode資訊,都在/lg/dminterface2裡

Status: HEALTHY
 Total size:    74110492 B
 Total dirs:    0
 Total files:   1
 Total blocks (validated):      2 (avg. block size 37055246 B)
 Minimally replicated blocks:   2 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         1 (50.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          107
 Number of racks:               4

The filesystem under path '/user/distribute-hadoop-boss/tmp/pgv/20090813/1000000103/input/JIFEN.QQ.COM.2009-08-13-18.30' is HEALTHY