DataNode退役舊資料節點
倆種方式(1)NameNode通過配製檔案指定DataNode新增白名單
(2)NameNode通過配製檔案指定DataNode新增黑名單,退役會將當前節點中儲存的塊進行轉移!
新增白名單
新增到白名單的主機節點,都允許訪問NameNode,不在白名單的主機節點,都會被退出。
配置白名單的具體步驟如下:
(1)在NameNode的/opt/module/hadoop-2.7.2/etc/hadoop目錄下創建
[[email protected] hadoop]$ pwd
/opt/module/hadoop-2.7.2/etc/hadoop
[[email protected] hadoop]$ touch dfs.hosts
[[email protected] hadoop]$ vi dfs.hosts
新增如下主機名稱(不新增hadoop105)
hadoop102
hadoop103
hadoop104
(2)在NameNode的hdfs-site.xml配置檔案中增加dfs.hosts屬性
<property>
<name>dfs.hosts</name>
<value>/opt/module/hadoop-2.7.2/etc/hadoop/dfs.hosts</value>
</property>
(3)配置檔案分發
[[email protected] hadoop]$ xsync hdfs-site.xml
(4)重新整理NameNode
[[email protected] hadoop-2.7.2]$ hdfs dfsadmin -refreshNodes
Refresh nodes successful
(5)更新ResourceManager節點
[[email protected] hadoop-2.7.2]$ yarn rmadmin -refreshNodes
17/06/24 14:17:11 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.1.103:8033
(6)在web瀏覽器上檢視
4. 如果資料不均衡,可以用命令實現叢集的再平衡
[[email protected] sbin]$ ./start-balancer.sh
starting balancer, logging to /opt/module/hadoop-2.7.2/logs/hadoop-atguigu-balancer-hadoop102.out
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move Bytes Being Moved
黑名單退役
在黑名單上面的主機都會被強制退出。
1.在NameNode的/opt/module/hadoop-2.7.2/etc/hadoop目錄下創建dfs.hosts.exclude檔案
[[email protected] hadoop]$ pwd
/opt/module/hadoop-2.7.2/etc/hadoop
[[email protected] hadoop]$ touch dfs.hosts.exclude
[[email protected] hadoop]$ vi dfs.hosts.exclude
新增如下主機名稱(要退役的節點)
hadoop105
2.在NameNode的hdfs-site.xml配置檔案中增加dfs.hosts.exclude屬性
<property>
<name>dfs.hosts.exclude</name>
<value>/opt/module/hadoop-2.7.2/etc/hadoop/dfs.hosts.exclude</value>
</property>
3.重新整理NameNode、重新整理ResourceManager
[[email protected] hadoop-2.7.2]$ hdfs dfsadmin -refreshNodes
Refresh nodes successful
[[email protected] hadoop-2.7.2]$ yarn rmadmin -refreshNodes
17/06/24 14:55:56 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.1.103:8033
4. 檢查Web瀏覽器,退役節點的狀態為decommission in progress(退役中),說明資料節點正在複製塊到其他節點,如圖3-17所示
5.等待退役節點狀態為decommissioned(所有塊已經複製完成),停止該節點及節點資源管理器。注意:如果副本數是3,服役的節點小於等於3,是不能退役成功的,需要修改副本數後才能退役,如圖3-18所示
[[email protected] hadoop-2.7.2]$ sbin/hadoop-daemon.sh stop datanode
stopping datanode
[[email protected] hadoop-2.7.2]$ sbin/yarn-daemon.sh stop nodemanager
stopping nodemanager
6. 如果資料不均衡,可以用命令實現叢集的再平衡
[[email protected] hadoop-2.7.2]$ sbin/start-balancer.sh
starting balancer, logging to /opt/module/hadoop-2.7.2/logs/hadoop-atguigu-balancer-hadoop102.out
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move Bytes Being Moved
注意:不允許白名單和黑名單中同時出現同一個主機名稱。