Hadoop HDFS 退役旧节点

Bertha 。 2022-02-24 06:46 304阅读 0赞

**注意：如果副本数是3，服役的节点小于等于3，是不能退役成功的，需要修改副本数后才能退役。**

## 配置 dfs.hosts ##

在服务器白名单中删除`hadoop105`

[root@hadoop102 hadoop]# pwd
    /opt/module/hadoop-2.7.2/etc/hadoop
    [root@hadoop102 hadoop]# vim dfs.hosts

## 配置 dfs.hosts.exclude ##

在 namenode 的/opt/module/hadoop-2.7.2/etc/hadoop 目录下创建 dfs.hosts.exclude 文件

[root@hadoop102 hadoop]# pwd
    /opt/module/hadoop-2.7.2/etc/hadoop
    [root@hadoop102 hadoop]# vim dfs.hosts.exclude

添加如下主机名称(要退役的节点)

hadoop105

## 配置 hdfs-site.xml ##

在 namenode 的 hdfs-site.xml 配置文件中增加 dfs.hosts.exclude 属性

<property>
        <name>dfs.hosts.exclude</name> 
        <value>/opt/module/hadoop-2.7.2/etc/hadoop/dfs.hosts.exclude
        </value>
    </property>

## 分发配置 ##

分发白名单`dfs.hosts`

[root@hadoop102 hadoop]# xsync ./dfs.hosts
    fname=dfs.hosts
    pdir=/opt/module/hadoop-2.7.2/etc/hadoop
    --------------------- hadoop102 ----------------
    sending incremental file list
    
    sent 32 bytes  received 12 bytes  88.00 bytes/sec
    total size is 30  speedup is 0.68
    --------------------- hadoop103 ----------------
    sending incremental file list
    dfs.hosts
    
    sent 105 bytes  received 37 bytes  284.00 bytes/sec
    total size is 30  speedup is 0.21
    --------------------- hadoop104 ----------------
    sending incremental file list
    dfs.hosts
    
    sent 105 bytes  received 37 bytes  284.00 bytes/sec
    total size is 30  speedup is 0.21
    --------------------- hadoop105 ----------------
    sending incremental file list
    dfs.hosts
    
    sent 105 bytes  received 37 bytes  284.00 bytes/sec
    total size is 30  speedup is 0.21

分发黑名单`dfs.hosts.exclude`

[root@hadoop102 hadoop]# xsync ./dfs.hosts.exclude 
    fname=dfs.hosts.exclude
    pdir=/opt/module/hadoop-2.7.2/etc/hadoop
    --------------------- hadoop102 ----------------
    sending incremental file list
    
    sent 40 bytes  received 12 bytes  104.00 bytes/sec
    total size is 10  speedup is 0.19
    --------------------- hadoop103 ----------------
    sending incremental file list
    dfs.hosts.exclude
    
    sent 93 bytes  received 31 bytes  82.67 bytes/sec
    total size is 10  speedup is 0.08
    --------------------- hadoop104 ----------------
    sending incremental file list
    dfs.hosts.exclude
    
    sent 93 bytes  received 31 bytes  248.00 bytes/sec
    total size is 10  speedup is 0.08
    --------------------- hadoop105 ----------------
    sending incremental file list
    dfs.hosts.exclude
    
    sent 93 bytes  received 31 bytes  82.67 bytes/sec
    total size is 10  speedup is 0.08

分发`hdfs-site.xml`

[root@hadoop102 hadoop]# xsync ./hdfs-site.xml 
    fname=hdfs-site.xml
    pdir=/opt/module/hadoop-2.7.2/etc/hadoop
    --------------------- hadoop102 ----------------
    sending incremental file list
    
    sent 36 bytes  received 12 bytes  96.00 bytes/sec
    total size is 1216  speedup is 25.33
    --------------------- hadoop103 ----------------
    sending incremental file list
    hdfs-site.xml
    
    sent 599 bytes  received 43 bytes  1284.00 bytes/sec
    total size is 1216  speedup is 1.89
    --------------------- hadoop104 ----------------
    sending incremental file list
    hdfs-site.xml
    
    sent 599 bytes  received 43 bytes  1284.00 bytes/sec
    total size is 1216  speedup is 1.89
    --------------------- hadoop105 ----------------
    sending incremental file list
    hdfs-site.xml
    
    sent 599 bytes  received 43 bytes  1284.00 bytes/sec
    total size is 1216  speedup is 1.89

## 刷新 namenode 和 resourcemanager ##

[root@hadoop102 hadoop]# hdfs dfsadmin -refreshNodes
    Refresh nodes successful

[root@hadoop103 ~]# yarn rmadmin -refreshNodes
    19/04/02 18:54:36 INFO client.RMProxy: Connecting to ResourceManager at hadoop103/192.168.1.103:8033

注意：如果出现下面提示，请检查hdfs-site.xml配置

[root@hadoop102 hadoop]# hdfs dfsadmin -refreshNodes
    refreshNodes: /opt/module/hadoop-2.7.2/etc/hadoop/dfs.hosts.exclude

## 检查 ##

退役节点的状态为 **decommission in progress(退役中)**，说明数据节点正在复制块到其他节点。

hadoop105已经下线  
![2019-04-03-15542672389077.jpg][]

## 停止该节点及节点资源管理器 ##

[root@hadoop105 hadoop-2.7.2]# hadoop-daemon.sh stop datanode
    stopping datanode
    [root@hadoop105 hadoop-2.7.2]# yarn-daemon.sh stop nodemanager
    stopping nodemanager

## 数据均衡 ##

如果数据不均衡，可以用命令实现集群的再平衡

![2019-04-03-15542670852343.jpg][]

![2019-04-03-15542674557641.jpg][]

[2019-04-03-15542672389077.jpg]: /images/20220224/1104fa68c0db4435afb5bc52e100d2cd.png
[2019-04-03-15542670852343.jpg]: /images/20220224/26540629b0964dbeb6e3c214a71731d1.png
[2019-04-03-15542674557641.jpg]: /images/20220224/1fa9242095e44aed818c1a688f39c28f.png