Hadoop伪分布式 => 启动YARN并运行MR程序

Bertha 。 2023-07-22 06:59 27阅读 0赞

**配置集群：**

cd /opt/module/hadoop-2.9.2/etc/hadoop

sudo vim yarn-env.sh

将 \# export JAVA\_HOME=/home/y/libexec/jdk1.6.0/ 注释打开修改为

export JAVA\_HOME=/opt/module/jdk1.8.0\_241

sudo vim yarn-site.xml

在<configuration></configuration>中间加入下面内容

<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    
    
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>hadoop101</value>
    </property>

sudo vim mapred-env.sh

将 \# export JAVA\_HOME=/home/y/libexec/jdk1.6.0/ 注释打开修改为

export JAVA\_HOME=/opt/module/jdk1.8.0\_241

修改mapred-site.xml.template文件名为mapred-site.xml

mv mapred-site.xml.template mapred-site.xml

sudo vim mapred-site.xml

在<configuration></configuration>中间加入下面内容

<property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>

**启动集群：**

cd /opt/module/hadoop-2.9.2/

sbin/hadoop-daemon.sh start namenode

sbin/hadoop-daemon.sh start datanode

启动ResourceManager

sbin/yarn-daemon.sh start resourcemanager

启动NodeManager

sbin/yarn-daemon.sh start nodemanager

查看状态：jps

![20200404214953611.png][]

打开网址：[http://hadoop101:8088/][http_hadoop101_8088]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODg4MzMzOA_size_16_color_FFFFFF_t_70][]

执行：

hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.2.jar wordcount /user/atguigu/input /user/atguigu/output

![20200404220220878.png][]

[20200404214953611.png]: /images/20230528/78bc3100d7c54a44851c4b5cdd32a7a2.png
[http_hadoop101_8088]: http://hadoop101:8088/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODg4MzMzOA_size_16_color_FFFFFF_t_70]: /images/20230528/20ce9e197a81481d8faa3779751ab642.png
[20200404220220878.png]: /images/20230528/bc335c45684a4311932534fce125483b.png