is running 236038656B beyond the 'VIRTUAL' memory limit. Current usage: 52.4 MB of 1 GB physical memory used; 2.3 GB of 2.1 GB virtual memory used.
阿新 • • 發佈:2022-03-18
[2022-03-18 18:26:57.187]Container [pid=92959,containerID=container_1647598528227_0003_01_000002]
is running 236038656B beyond the 'VIRTUAL' memory limit. Current usage: 52.4 MB of 1 GB physical
memory used; 2.3 GB of 2.1 GB virtual memory used. Killing container.
輸入下列程式碼時報以上錯誤
hadoop jar wc.jar com.ltj.wordcount.WordCount -Dmapreduce.job.queuename=default /input /output
問題原因是container在申請多餘的記憶體時,被resouremanager殺掉了
此處可修改hadoop/etc/mapred-site.xml
新增配置資訊
<property> <name>mapreduce.map.memory.mb</name> <value>1536</value> </property> <property> <name>mapreduce.map.java.opts</name> <value>-Xmx1024M</value> </property> <property> <name>mapreduce.reduce.memory.mb</name> <value>3072</value> </property> <property> <name>mapreduce.reduce.java.opts</name> <value>-Xmx2560M</value> </property>
然後重啟hadoop
重新提交任務
找了一天-----555終於解決了