二十九、日志的聚集

小鱼儿 2022-05-06 06:58 179阅读 0赞

将每次Hadoop运行功能都放到HDFS上

### 1、开启日志聚集功能 ###

[admin@hadoop13 hadoop-2.7.2]$ vim etc/hadoop/yarn-site.xml

添加如下配置：

<property>
    <name>yarn.log-aggregation-enable</name>
    <value>true</value>
    </property>
    
    <property>
    <name>yarn.log-aggregation.retain-seconds</name>
    <value>604800</value>
    </property>

### 2、关闭nodemanager 、resourcemanager和historymanager ###

[admin@hadoop13 hadoop-2.7.2]$ sbin/yarn-daemon.sh stop resourcemanager
    [admin@hadoop13 hadoop-2.7.2]$ sbin/yarn-daemon.sh stop nodemanager
    [admin@hadoop13 hadoop-2.7.2]$ sbin/mr-jobhistory-daemon.sh stop historyserver

### 3、启动nodemanager 、resourcemanager和historymanager ###

[admin@hadoop13 hadoop-2.7.2]$ sbin/yarn-daemon.sh start resourcemanager
    [admin@hadoop13 hadoop-2.7.2]$ sbin/yarn-daemon.sh start nodemanager
    [admin@hadoop13 hadoop-2.7.2]$ sbin/mr-jobhistory-daemon.sh start historyserver

### 5、删除hdfs上已经存在的hdfs文件 ###

[admin@hadoop13 hadoop-2.7.2]$ hadoop fs -rm -R /user/admin/mapreduce/wordcount/output

### 6、执行wordcount程序 ###

[admin@hadoop13 hadoop-2.7.2]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar wordcount /user/admin/mapreduce/wordcount/input /user/admin/mapreduce/wordcount/output

### 7、查看日志 ###

http://192.168.1.113:19888/jobhistory

![在这里插入图片描述][70]

![在这里插入图片描述][70 1]

![在这里插入图片描述][70 2]

### 8、停止服务 ###

[admin@hadoop13 hadoop-2.7.2]$ sbin/yarn-daemon.sh stop nodemanager 
    [admin@hadoop13 hadoop-2.7.2]$ sbin/yarn-daemon.sh stop resourcemanager 
    [admin@hadoop13 hadoop-2.7.2]$ sbin/mr-jobhistory-daemon.sh stop historyserver 
    [admin@hadoop13 hadoop-2.7.2]$ sbin/hadoop-daemon.sh stop datanode 
    [admin@hadoop13 hadoop-2.7.2]$ sbin/hadoop-daemon.sh stop namenode

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L25ld2JpZV85MDc0ODY4NTI_size_16_color_FFFFFF_t_70_pic_center][]

[70]: /images/20220506/12c2227a01c04934a55c3b8701d17a20.png
[70 1]: /images/20220506/742f1656b71a4c1981ca184aa0299b9a.png
[70 2]: /images/20220506/558f22a88d744c879e2f3c6833bf4aa3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L25ld2JpZV85MDc0ODY4NTI_size_16_color_FFFFFF_t_70_pic_center]: /images/20220506/55c325209d7a41f5977175f64df8132d.png