一、前期的準備
1、安裝環境的準備
ip地址的配置等參考地點
2、檢視centos的位數

[root@CDHnode1 ~]# file /bin/ls
/bin/ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=3d705971a4c4544545cb78fd890d27bf792af6d4, stripped

3、解壓hadoop並設定軟鏈

[root@CDHnode1 hadoop-2.6
 
.0-cdh5.4.5]# ln -sf /home/hadoopcdh/soft/hadoop-2.6.0-cdh5.4.5 /opt/hadoop

4、hosts的對映檔案

[root@CDHnode1 ~]# vi /etc/hosts
192.168.146.189 CDHnode1

5、準備Hadoop專用使用者和組

[root@CDHnode1 ~]# groupadd hadoop
[root@CDHnode1 ~]# useradd -g hadoop hadoop
[root@CDHnode1 ~]# passwd hadoop
Changing password for user hadoop.
New password
 
: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.

二、ssh的安裝
1、切換到hadoop下

[root@CDHnode1 ~]# su hadoop
[hadoop@CDHnode1 root]$ cd 
[hadoop@CDHnode1 ~]$ pwd
/home/hadoop

2、開始建立並生成目錄

建立.ssh目錄，生成金鑰
[hadoop@CDHnode
 
1 ~]$ mkdir .ssh
[hadoop@CDHnode1 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
f0:fe:1c:b4:f8:66:ec:14:69:b2:4a:bf:6c:f0:fa:11 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|      .          |
|       o   .     |
|        E =      |
|      .. B o     |
|      .o=.+      |
|     . +o*+.     |
|      o+===      |
+-----------------+
切換到.ssh目錄下,進行檢視公鑰和私鑰
[hadoop@CDHnode1 ~]$ cd .ssh
[hadoop@CDHnode1 .ssh]$ ls
id_rsa  id_rsa.pub
將公鑰複製到日誌檔案裡
[hadoop@CDHnode1 .ssh]$ cp id_rsa.pub authorized_keys
[hadoop@CDHnode1 .ssh]$ ls
authorized_keys  id_rsa  id_rsa.pub
退回到/home/hadoop/，來賦予許可權
[hadoop@CDHnode1 .ssh]$ cd ..
[hadoop@CDHnode1 ~]$ pwd
/home/hadoop
[hadoop@CDHnode1 ~]$ chmod 700 .ssh
[hadoop@CDHnode1 ~]$ chmod 600 .ssh/*

3、切換到root使用者下，安裝ssh外掛(openssh)

[hadoop@CDHnode1 ~]$ su root
Password: 
[root@CDHnode1 hadoop]# yum -y install openssh-clients
切換到/home/hadoop/，測試ssh無密碼訪問
[root@CDHnode1 hadoop]# su hadoop
[hadoop@CDHnode1 ~]$ ssh cdhhadoop
The authenticity of host 'cdhhadoop (192.168.146.189)' can't be established.
ECDSA key fingerprint is dc:c3:a8:6a:ac:10:63:15:43:52:51:ce:c9:9b:40:7d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'cdhhadoop,192.168.146.189' (ECDSA) to the list of known hosts.
Last login: Sat May 27 23:08:14 2017

[[email protected] ~]$ 

三、hadoop的環境變數的設定
1、將剛改名的hadoop檔案，許可權賦給hadoop使用者

[root@CDHnode1 opt]# chown -R hadoop:hadoop hadoop
[root@CDHnode1 opt]# ls -l
total 0
lrwxrwxrwx. 1 hadoop hadoop 42 May 27 22:49 hadoop -> /home/hadoopcdh/soft/hadoop-2.6.0-cdh5.4.5
lrwxrwxrwx. 1 root   root   32 May  5 08:58 jdk1.8 -> /home/hadoopcdh/soft/jdk1.8.0_60

2、先建立hadoop資料目錄 ，將整個data目錄許可權賦予給hadoop使用者

[[email protected]CDHnode1 opt]# mkdir -p /data/dfs/name
[[email protected]CDHnode1 opt]# mkdir -p /data/dfs/data
[[email protected]CDHnode1 opt]# mkdir -p /data/tmp
[[email protected]CDHnode1 opt]# chown -R hadoop:hadoop hadoop /data
[[email protected]CDHnode1 opt]# ls -l /data
total 0
drwxr-xr-x. 4 hadoop hadoop 30 May 27 23:13 dfs
drwxr-xr-x. 2 hadoop hadoop  6 May 27 23:13 tmp

3、修改hadoop對應的配置檔案，切換到hadoop使用者，切換到hadoop目錄

[root@CDHnode1 opt]# su hadoop
[hadoop@CDHnode1 opt]$ ls
hadoop  jdk1.8
[hadoop@CDHnode1 opt]$ cd hadoop/
[hadoop@CDHnode1 hadoop]$ 

4、修改etc/hadoop/core-site.xml配置檔案，新增如下資訊
(1)、配置的是HDFS(hadoop)的分散式檔案系統的地址及埠號

<property>
                <name>fs.defaultFS</name>
                <value>hdfs://cdhhadoop:9000</value>
</property>

(2)、HDFS路徑的存放資料的公共目錄

<property>
                <name>hadoop.tmp.dir</name>
                <value>file:/data/tmp</value>
 </property>

(3)、下面配置的是，因為在hadoop1.0中引入了安全機制，所以從客戶端發出的作業提交者全變成了hadoop，不管原始提交者是哪個使用者，為了解決該問題，引入了安全違章功能，允許一個超級使用者來代替其他使用者來提交作業或者執行命令，而對外來看，執行者仍然是普通使用者。所以

配置設為任意客戶端
 <property>
                <name>hadoop.proxyuser.hadoop.hosts</name>
                <value>*</value>
 </property>
配置設為任意使用者組
 <property>
                <name>hadoop.proxyuser.hadoop.groups</name>
                <value>*</value>
</property>

5、修改etc/hadoop/hdfs-site.xml配置檔案，新增如下資訊

配置的是namenode檔案目錄
<property>
                <name>dfs.namenode.name.dir</name>
                <value>/data/dfs/name</value>
                <final>true</final>
</property>
配置的是datanode檔案目錄
<property>
                <name>dfs.datanode.data.dir</name>
                <value>/data/dfs/data</value>
                <final>true</final>
</property>
配置的是資料塊副本和HDFS許可權
<property>
                <name>dfs.replication</name>
                <value>1</value>
 </property>
 <property>
                <name>dfs.permissions</name>
                <value>false</value>
 </property>

6、修改etc/hadoop/mapred-site.xml配置檔案，新增如下資訊。

[root@CDHnode1 hadoop]# cp mapred-site.xml.template mapred-site.xml

與hadoop1.0不同的是，使用的是yarn

 <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
 </property>

7、修改etc/hadoop/yarn-site.xml配置檔案，新增如下資訊。
為了能夠執行mapreduce程式，我們需要讓.nodemanger在啟動時載入shuffle。

<property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>

</property>

8、修改etc/hadoop/slaves，新增如下資訊。

[root@CDHnode1 hadoop]# vi slaves
cdhhadoop

9、設定Hadoop環境變數
vi /etc/profile

export HADOOP_HOME=/opt/hadoop
export PATH=$HADOOP_HOME/bin:$PATH

10、格式化namenode

hadoop namenode  -format

11、啟動叢集

[hadoop@cdhhadoop hadoop]$ sbin/start-all.sh