環境準備:

三臺虛擬機器 master（8）、slave1（9）、slave2（10)

centos 7.1、jdk-8u171-linux-x64.tar.gz、hadoop-2.7.3.tar.gz

0x1環境準備

首先先在三臺虛擬機器中建立hadoop資料夾

mdkir /usr/hadoop

在master中將hadoop解壓到master的Hadoop資料夾中

tar -zxvf hadoop-2.7.3.tar.gz -C /usr/hadoop/

0x2編輯配置檔案

修改配置檔案 vi /usr/hadoop/hadoop-2.7.3/etc/hadoop/hadoop-env.sh 

新增

export JAVA_HOME=/usr/java/jdk1.8.0_171

修改配置檔案vi core-site.xml

<configuration>
        <property>
                <name>fs.default.name</name>
                <value>http://master:9000</value>
        </property>
        <property>
                <name>hadoop.mp.dir</name>
                <value>/usr/hadoop/hadoop-2.7.3/hdfs/tmp</value>
        </property>
        <property>
                <name>io.file.buffer.size</name>
                <value>131072</value>
        </property>
        <property>
                <name>fs.checkpoint.period</name>
                <value>60</value>
        </property>
        <property>
                <name>fs.checkpoint.size</name>
                <value>67108864</value>
        </property>
</configuration>
 

修改配置檔案vi yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
        <property>
                <name>yarn.resourcemanager.address</name>
                <value>masetr:18040</value>
        </property>
        <property>
                <name>yarn.resourcemanager.scheduler.address</name>
                <value>masetr:18030</value>
        </property>
        <property>
                <name>yarn.resourcemanager.webapp.address</name>
                <value>masetr:18088</value>
        </property>
        <property>
                <name>yarn.resourcemanager.resource-tracker.address</name>
                <value>masetr:18025</value>
        </property>
        <property>
                <name>yarn.resourcemanager.admin.address</name>
                <value>masetr:18141</value>
        </property>
        <property>
                <name>yarn.resourcemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
        <property>
                <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
                <value>org.apache.hadoop.mapred.ShuffleHandeler</value>
        </property>
</configuration>
 

修改配置檔案 vi slaves

slave1
slave2

修改配置檔案master

master

修改配置檔案hdfs-site.xml 

<configuration>
        <property>
                <name>dfs.replication</name>
                <value>2</value>
        </property>
        <property>
                <name>dfs.namenode.name.dir</name>
                <value>file:/usr/hadoop/hadoop-2.7.3/hdfs/name</value>
                <final>true</final>
        </property>
        <property>
                <name>dfs.datanode.data.dir</name>
                <value>file:/usr/hadoop/hadoop-2.7.3/hdfs/data</value>
                <final>true</final>
        </property>
        <property>
                <name>dfs.namenode.secondary.http-address</name>
                <value>slave1:9001</value>
        </property>
        <property>
                <name>dfs.webhdfs.enabled</name>
                <value>true</value>
        </property>
        <property>
                <name>dfs.permissions</name>
                <value>false</value>
        </property>
</configuration>

建立配置檔案mapred-site.xml

cp -rvf mapred-site.xml.template mapred-site.xml

然後編輯

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
</configuration>

分發hadoop

scp -r /usr/hadoop/ slave1:/usr/

修改環境變數

vi /etc/profile

export HADOOP_HOME=/usr/hadoop/hadoop-2.7.3/
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

然後在master格式化hadoop

hadoop namenode -format

然後啟動

/usr/hadoop/hadoop-2.7.3/sbin/start-all.sh