1. 程式人生 > >DataNode退役舊資料節點

DataNode退役舊資料節點

倆種方式(1)NameNode通過配製檔案指定DataNode新增白名單     

               (2)NameNode通過配製檔案指定DataNode新增黑名單,退役會將當前節點中儲存的塊進行轉移!

新增白名單

       新增到白名單的主機節點,都允許訪問NameNode,不在白名單的主機節點,都會被退出。

配置白名單的具體步驟如下:

(1)在NameNode的/opt/module/hadoop-2.7.2/etc/hadoop目錄下創

dfs.hosts檔案

[[email protected] hadoop]$ pwd

/opt/module/hadoop-2.7.2/etc/hadoop

[[email protected] hadoop]$ touch dfs.hosts

[[email protected] hadoop]$ vi dfs.hosts

新增如下主機名稱(不新增hadoop105

hadoop102

hadoop103

hadoop104

       (2)在NameNode的hdfs-site.xml配置檔案中增加dfs.hosts屬性

<property>

<name>dfs.hosts</name>

<value>/opt/module/hadoop-2.7.2/etc/hadoop/dfs.hosts</value>

</property>

(3)配置檔案分發

[[email protected] hadoop]$ xsync hdfs-site.xml

       (4)重新整理NameNode

[[email protected] hadoop-2.7.2]$ hdfs dfsadmin -refreshNodes

Refresh nodes successful

       (5)更新ResourceManager節點

[[email protected] hadoop-2.7.2]$ yarn rmadmin -refreshNodes

17/06/24 14:17:11 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.1.103:8033

       (6)在web瀏覽器上檢視

4.    如果資料不均衡,可以用命令實現叢集的再平衡

[[email protected] sbin]$ ./start-balancer.sh

starting balancer, logging to /opt/module/hadoop-2.7.2/logs/hadoop-atguigu-balancer-hadoop102.out

Time Stamp               Iteration#  Bytes Already Moved  Bytes Left To Move  Bytes Being Moved

黑名單退役

在黑名單上面的主機都會被強制退出。

1.在NameNode的/opt/module/hadoop-2.7.2/etc/hadoop目錄下創dfs.hosts.exclude檔案

[[email protected] hadoop]$ pwd

/opt/module/hadoop-2.7.2/etc/hadoop

[[email protected] hadoop]$ touch dfs.hosts.exclude

[[email protected] hadoop]$ vi dfs.hosts.exclude

新增如下主機名稱(要退役的節點)

hadoop105

2.在NameNode的hdfs-site.xml配置檔案中增加dfs.hosts.exclude屬性

<property>

<name>dfs.hosts.exclude</name>

      <value>/opt/module/hadoop-2.7.2/etc/hadoop/dfs.hosts.exclude</value>

</property>

3.重新整理NameNode、重新整理ResourceManager

[[email protected] hadoop-2.7.2]$ hdfs dfsadmin -refreshNodes

Refresh nodes successful

 

[[email protected] hadoop-2.7.2]$ yarn rmadmin -refreshNodes

17/06/24 14:55:56 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.1.103:8033

4.           檢查Web瀏覽器,退役節點的狀態為decommission in progress(退役中),說明資料節點正在複製塊到其他節點,如圖3-17所示

5.等待退役節點狀態為decommissioned(所有塊已經複製完成),停止該節點及節點資源管理器。注意:如果副本數是3,服役的節點小於等於3,是不能退役成功的,需要修改副本數後才能退役,如圖3-18所示

[[email protected] hadoop-2.7.2]$ sbin/hadoop-daemon.sh stop datanode

stopping datanode

[[email protected] hadoop-2.7.2]$ sbin/yarn-daemon.sh stop nodemanager

stopping nodemanager

6.    如果資料不均衡,可以用命令實現叢集的再平衡

[[email protected] hadoop-2.7.2]$ sbin/start-balancer.sh

starting balancer, logging to /opt/module/hadoop-2.7.2/logs/hadoop-atguigu-balancer-hadoop102.out

Time Stamp               Iteration#  Bytes Already Moved  Bytes Left To Move  Bytes Being Moved

       注意:不允許白名單和黑名單中同時出現同一個主機名稱。