执行wordcount例子报异常

阳光穿透心脏的1/2处 2022-05-21 09:20 421阅读 0赞

异常一：  
2018-01-09 02:50:43,320 INFO mapreduce.Job: Job job\_1515466125577\_0002 failed with state FAILED due to: Application application\_1515466125577\_0002 failed 2 times due to AM Container for appattempt\_1515466125577\_0002\_000002 exited with exitCode: 127  
Failing this attempt.Diagnostics: \[2018-01-09 02:50:42.706\]Exception from container-launch.  
Container id: container\_1515466125577\_0002\_02\_000001  
Exit code: 127  
  
  
\[2018-01-09 02:50:42.750\]Container exited with a non-zero exit code 127. Error file: prelaunch.err.  
Last 4096 bytes of prelaunch.err :  
Last 4096 bytes of stderr :  
/bin/bash: /bin/java: No such file or directory  
  
  
解决方法：在中添加adoop-3.0.0/libexec/hadoop-config.sh文件中添加JAVA\_HOME环境变量  
  
  
  
  
异常二：  
2018-01-09 03:25:37,250 INFO mapreduce.Job: Job job\_1515468264727\_0001 failed with state FAILED due to: Application application\_1515468264727\_0001 failed 2 times due to AM Container for appattempt\_1515468264727\_0001\_000002 exited with exitCode: 1  
Failing this attempt.Diagnostics: \[2018-01-09 03:25:36.447\]Exception from container-launch.  
Container id: container\_1515468264727\_0001\_02\_000001  
Exit code: 1  
  
  
\[2018-01-09 03:25:36.499\]Container exited with a non-zero exit code 1. Error file: prelaunch.err.  
Last 4096 bytes of prelaunch.err :  
Last 4096 bytes of stderr :  
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster  
  
  
Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:  
<property>  
<name>yarn.app.mapreduce.am.env</name>  
<value>HADOOP\_MAPRED\_HOME=$\{full path of your hadoop distribution directory\}</value>  
</property>  
<property>  
<name>mapreduce.map.env</name>  
<value>HADOOP\_MAPRED\_HOME=$\{full path of your hadoop distribution directory\}</value>  
</property>  
<property>  
<name>mapreduce.reduce.env</name>  
<value>HADOOP\_MAPRED\_HOME=$\{full path of your hadoop distribution directory\}</value>  
</property>  
解决方法：按错误提示，在mapred-site.xml配置文件中添加hadoop根目录  
  
  
异常三：  
2018-01-09 03:55:18,863 INFO mapreduce.Job: Job job\_1515470038765\_0001 failed with state FAILED due to: Application application\_1515470038765\_0001 failed 2 times due to AM Container for appattempt\_1515470038765\_0001\_000002 exited with exitCode: -103  
Failing this attempt.Diagnostics: \[2018-01-09 03:55:17.480\]Container \[pid=728,containerID=container\_1515470038765\_0001\_02\_000001\] is running beyond virtual memory limits. Current usage: 146.5 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used. Killing container.  
Dump of the process-tree for container\_1515470038765\_0001\_02\_000001 :  
|- PID PPID PGRPID SESSID CMD\_NAME USER\_MODE\_TIME(MILLIS) SYSTEM\_TIME(MILLIS) VMEM\_USAGE(BYTES) RSSMEM\_USAGE(PAGES) FULL\_CMD\_LINE  
|- 741 728 728 728 (java) 634 19 2852925440 37258 /usr/local/jdk1.8.0\_131/bin/java -Djava.io.tmpdir=/home/hadoop/tmp/nm-local-dir/usercache/hadoop/appcache/application\_1515470038765\_0001/container\_1515470038765\_0001\_02\_000001/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/home/hadoop/hadoop-3.0.0/logs/userlogs/application\_1515470038765\_0001/container\_1515470038765\_0001\_02\_000001 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster  
解决方法：在yarn-site.xml中添加以下配置  
<property>  
<name>yarn.nodemanager.vmem-pmem-ratio</name>  
<value>5</value>  
<description>Ratio between virtual memory to physical memory when setting memory limits  
for containers. Container allocations are expressed in terms of physical memory, and  
virtual memory usage is allowed to exceed this allocation by this ratio.</description>  
</property>