hadoop2.x Yarn HA搭建

爱被打了一巴掌 2022-05-20 10:37 215阅读 0赞

1.在worker1和worker2上配置resourcemanager和nodemanager，resourcemanager一主一备，所以配置这两台机器免密钥登录，前面已经配置过，不再赘述。  
2.先拷贝一份mapred-site.xml，配置mapreduce使用yarn框架

[hadoop@master hadoop]$ vim mapred-site.xml

<configuration>
      <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
      </property>
    </configuration>

3.配置yarn-site.xml

[hadoop@master hadoop]$ vim yarn-site.xml

<configuration>
      <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
      </property>
      <property>
        <name>yarn.nodemanager.log-dirs</name>
        <value>/data/hadoopha/yarn/log/userlogs</value>
      </property>
      <property>
        <name>yarn.resourcemanager.ha.enabled</name>
        <value>true</value>
      </property>
      <property>
        <name>yarn.resourcemanager.cluster-id</name>
        <value>cluster1</value>
      </property>
      <property>
        <name>yarn.resourcemanager.ha.rm-ids</name>
        <value>rm1,rm2</value>
      </property>
      <property>
        <name>yarn.resourcemanager.hostname.rm1</name>
        <value>worker1</value>
      </property>
      <property>
        <name>yarn.resourcemanager.hostname.rm2</name>
        <value>worker2</value>
      </property>
      <property>
        <name>yarn.resourcemanager.zk-address</name>
        <value>master:2181,worker1:2181,worker2:2181</value>
      </property>
    </configuration>

4.启动yarn

[hadoop@master hadoop]$ start-yarn.sh 
    starting yarn daemons
    starting resourcemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-resourcemanager-master.out
    worker1: starting nodemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-nodemanager-worker1.out
    worker2: starting nodemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-nodemanager-worker2.out
    [hadoop@master hadoop]$ jps
    7920 NameNode
    5393 JournalNode
    15651 Jps
    5093 QuorumPeerMain
    10429 DFSZKFailoverController

查看worker1，worker2节点，jps是没有resourcemanager进程的，需要手动启动  
5.在worker1和worker2上启动

[hadoop@worker1 ~]$ yarn-daemon.sh start resourcemanager
    starting resourcemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-resourcemanager-worker1.out
    [hadoop@worker1 ~]$ jps
    5024 JournalNode
    14802 Jps
    14580 ResourceManager
    8868 DFSZKFailoverController
    14438 NodeManager
    4824 QuorumPeerMain
    5436 DataNode
    5373 NameNode

[hadoop@worker2 logs]$ yarn-daemon.sh start resourcemanager
    starting resourcemanager, logging to /app/hadoop/hadoop-2.7.3/logs/yarn-hadoop-resourcemanager-worker2.out
    [hadoop@worker2 logs]$ jps
    4978 JournalNode
    9474 Jps
    5331 DataNode
    9414 ResourceManager
    9272 NodeManager
    4783 QuorumPeerMain
    [hadoop@worker2 logs]$

查看jps，ResourceManager和Nodemanager都已经启动。  
6.在webUI界面查看  
worker1：8088  
![这里写图片描述][70]  
但在worker2：8088查看时直接跳到了worker1，worker2处于standby。  
![这里写图片描述][70 1]

[70]: /images/20220520/3ad25ff4e7824628a2e2343b29f08213.png
[70 1]: /images/20220520/88f55b2396f5473f98b61d5ef038b5d7.png