2. 程式人生 > 其它 >HDFS 提示 There are 6 missing blocks. The following files may be corrupted:

HDFS 提示 There are 6 missing blocks. The following files may be corrupted:

阿新 • • 發佈：2022-03-05

問題描述

不知怎得最近在啟動測試環境的時候 hdfs 總是進入安全模式，無奈手動離開此模式才能進行正常操作

[hui@hadoop201 ~]$ hdfs dfsadmin -safemode get
Safe mode is ON
[hui@hadoop201 ~]$ hdfs dfsadmin -safemode leave
Safe mode is OFF

今天無意間登入了 hdfs web 頁面發現了有一個警告資訊，才發現 hdfs丟了一點檔案

There are 6 missing blocks. The following files may be corrupted:


blk_1073743362    
 
/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop202_44572
blk_1073743363    /tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop201_34618
blk_1073743364    /tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop203_38826
blk_1073743598    /tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop202_40701
blk_1073743599    
 
/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop201_34111
blk_1073744218    /tmp/logs/hui/logs-tfile/application_1646310503374_0004/hadoop203_35289
Please check the logs or run fsck in order to identify the missing blocks. See the Hadoop FAQ for common causes and potential solutions.

解決方案

1、檢視丟失檔案情況

[hui@hadoop201 job]$ hdfs fsck /
Connecting to namenode via http:
 
//hadoop201:9870/fsck?ugi=hui&path=%2F
FSCK started by hui (auth:SIMPLE) from /192.168.124.141 for path / at Sun Mar 06 00:42:11 CST 2022

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop201_34618: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743363

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop201_34618: CORRUPT 1 blocks of total size 410409 B.
/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop202_44572: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743362

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop202_44572: CORRUPT 1 blocks of total size 251360 B.
/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop203_38826: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743364

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop203_38826: CORRUPT 1 blocks of total size 332373 B.
/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop201_34111: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743599

/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop201_34111: CORRUPT 1 blocks of total size 1072914 B.
/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop202_40701: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743598

/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop202_40701: CORRUPT 1 blocks of total size 90956 B.
/tmp/logs/hui/logs-tfile/application_1646310503374_0004/hadoop203_35289: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073744218

/tmp/logs/hui/logs-tfile/application_1646310503374_0004/hadoop203_35289: CORRUPT 1 blocks of total size 111347 B.

2、刪除丟失檔案的資料

[hui@hadoop201 job]$ hdfs fsck -delete
Connecting to namenode via http://hadoop201:9870/fsck?ugi=hui&delete=1&path=%2F
FSCK started by hui (auth:SIMPLE) from /192.168.124.141 for path / at Sun Mar 06 00:53:45 CST 2022

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop201_34618: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743363

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop201_34618: CORRUPT 1 blocks of total size 410409 B.
/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop202_44572: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743362

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop202_44572: CORRUPT 1 blocks of total size 251360 B.
/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop203_38826: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743364

/tmp/logs/hui/logs-tfile/application_1644582136486_0001/hadoop203_38826: CORRUPT 1 blocks of total size 332373 B.
/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop201_34111: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743599

/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop201_34111: CORRUPT 1 blocks of total size 1072914 B.
/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop202_40701: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073743598

/tmp/logs/hui/logs-tfile/application_1644832307762_0002/hadoop202_40701: CORRUPT 1 blocks of total size 90956 B.
/tmp/logs/hui/logs-tfile/application_1646310503374_0004/hadoop203_35289: CORRUPT blockpool BP-284743301-192.168.124.141-1642979744475 block blk_1073744218

/tmp/logs/hui/logs-tfile/application_1646310503374_0004/hadoop203_35289: CORRUPT 1 blocks of total size 111347 B.
Status: CORRUPT
 Number of data-nodes:  3

重新整理 hdfs web 介面恢復正常

相關推薦

HDFS 提示 There are 6 missing blocks. The following files may be corrupted:

問題描述 不知怎得最近在啟動測試環境的時候 hdfs 總是進入安全模式，無奈手動離開此模式才能進行正常操作

git pull時，提示Your local changes to the following files would be overwritten by merge

問題描述： 本地修改了程式碼後，執行“git pull”命令時，無法更新程式碼，並報錯提示：“Your local changes to the following files would be overwritten by merge”

Git pull 遇到的問題：error: Your local changes to the following files would be overwritten by merge:

技術標籤：gitlabgit 今天 pull 程式碼的時候發現程式碼沒更新，然後git branch 檢視自己分支沒問題，git remote -v 看到連線的地址也沒問題，之後才注意到了這個報錯，

【Git】pull遇到錯誤：error: Your local changes to the following files would be overwritten by merge:

這種情況下，如何保留本地的修改同時又把遠端的合併過來呢？ 首先取決於你是否想要儲存本地修改。（是 /否）

OpenWrt編譯報錯：Package airfly_receiver is missing dependencies for the following libraries

今天在編譯一個OpenWrt測試用例的時候出現報錯 Package airfly_receiver is missing dependencies for the following libraries:

電腦開機提示Default Boot Device Missing or Boot Failed錯誤如何解決

有不少使用者在重灌系統的時候，發現在格式化分割槽或者進行一些操作之後，電腦開機出現提示Default Boot Device Missing or Boot Failed錯誤，該怎麼辦呢，隨小編一起來看看要怎麼解決這樣的問題吧！

解決Sublime Text3 There are no packages available for installation

解決 sublime text 3 使用 Install Package 時出現 There are no packages available for installation 的問題

IDEA搜尋外掛時顯示search results are not loaded check the internet connection的解決辦法

一、錯誤提示： 二、原因分析 IDEA的網路安全機制類似於windows的防火牆，它誤認為你的網路是不安全，所以造成了IDEA的模組系統載入不出來的結果。

C# 引用自定義的專案空間出現“Are you missing a using directive or an assembly reference?”

同一個專案中引用自定義的專案，出現“Are you missing a using directive or an assembly reference?”的解決方法：

springboot 啟動報錯: Multiple Dockets with the same group name are not supported. The following duplicat

報錯詳情 org.springframework.context.ApplicationContextException: Failed to start bean \'documentationPluginsBootstrapper\'; nested exception is java.lang.IllegalStateException: Multiple Dockets with

python pip升級報錯問題：Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max

1 提醒我升級pip WARNING: You are using pip version 20.2.2; however, version 20.2.4 is available.You should consider upgrading via the \'e:\\python\\anz\\python.exe -m pip install --upgrade pip\' comma

解決：java.lang.IllegalStateException: Annotation @EnableCircuitBreaker found, but there are no implementations. Did you forget to include a starter?

在主類新增@EnableCircuitBreaker或@SpringCloudApplication註解 pom.xml裡新增 <dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter-hystrix</

warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 6.0, but the range of supported deployment target versions is 8.0 to 13.2.99.

warning: The iOS Simulator deployment target \'IPHONEOS_DEPLOYMENT_TARGET\' is set to 6.0, but the range of supported deployment target versions is 8.0 to 13.2.99. (in target \'Reachability\' from p

what are the .map files used for in Bootstrap 3.x?

what are the .map files used for in Bootstrap 3.x? There are two files included in the CSS folder with .map file extensions. They are:

Linux，yum錯誤，There are no enabled repos.(學習）

1.yum yum（全稱為 Yellow dog Updater, Modified）是一個在Fedora和RedHat以及SUSE中的Shell前端軟體包管理器。基於RPM包(RPM 是 Red Hat Package Manager 的縮寫，本意是Red Hat 軟體包管理，顧名思義是Red Hat 貢

Too many keys are generated. There are only 1 target objects.

技術標籤：MyBatisSpring Boot Too many keys are generated. There are only 1 target objects. 記一次由spring boot升級導致的問題解決。

Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificat

問題描述 cmd 輸入 pip install {包名} 或者 cmd 輸入python -m pip install --upgrade pip 出現如下錯誤：

【ESP32】The following Python requirements are not satisfied: 問題解決

技術標籤：# ESP32 問題： The following Python requirements are not satisfied: click>=5.0 pyserial>=3.0

【Bug解決】yum提示Another app is currently holding the yum lock； waiting for it to exit...

技術標籤：Bug解決dockerlinuxcentos 問題描述： 今天想在虛擬機器上重新安裝docker 然後使用到yum命令報錯：

org.apache.coyote.http11.HeadersTooLargeException: An attempt was made to write more data to the response headers than there was room available in the buffer. Increase maxHttpHeaderSize

springcloud專案，請求某個服務時報錯如下：org.apache.coyote.http11.HeadersTooLargeException: An attempt was made to write more data to the response headers than there was room available in the buffer.