Hadoop完全分散式配置問題
阿新 • • 發佈:2018-11-09
關於搭建Hadoop完全分散式時配置的問題
配置hadoop的配置檔案core-site.xml, hdfs-site.xml, mapred-site.xml,yarn-site.xml,slaves(workers)(都在Hadoop安裝目錄/etc/hadoop資料夾下)
1、core-site.xml
<configuration> <property> <name>fs.default.name</name> <value>hdfs://主機的hostname:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>file:你的Hadoop安裝目錄/tmp</value> </property> </configuration>
2.hdfs-site.xml
<configuration> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:Hadoop安裝目錄/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:Hadoop安裝目錄/tmp/dfs/data</value> </property> <property> <name>dfs.namenode.secondary.http-address</name> <value>主機的hostname:9001</value> </property> </configuration>
3.mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
4.yarn-site.xml
<configuration> <property> <name>yarn.resourcemanager.hostname</name> <value>主機的hostname</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.log-aggregation.retain-seconds</name> <value>604800</value> </property> <property> <name>yarn.application.classpath</name> <value>在終端輸入hadoop classpath獲取的值</value> </property> </configuration>
5.slaves(在Hadoop3.x是workers)
slave1
slave2
這裡是你的從節點的hostname