hadoop 叢集執行

分別有3臺主機
10.10.10.128 master
10.10.10.129 slave1
10.10.10.130 slave2

1、 步驟一：NameNode 格式化

只要執行過 HDFS，Hadoop 的工作目錄（usr/local/src/hadoop/tmp）就會有資料
如果需要重新格式化，則在格式化之前一定要先刪
除工作目錄下的資料，否則格式化時會出問題。

[root@master~]# su – hadoop 
[root@master~]# cd /usr/local/src/hadoop/ 
[hadoop@masterhadoop]$ bin/hdfs namenode –format 
 

2、 步驟二：啟動 NameNode

[hadoop@master hadoop]$ hadoop-daemon.sh start namenode 
 
starting namenode, logging to /opt/module/hadoop2.7.1/logs/hadoop-hadoop-namenode-master.out

1.4.2. 實驗任務二：檢視 Java 程序

顯示當前所有 Java 程序 pid 的命令。

[hadoop@master hadoop]$ jps 
 
3557 NameNode 
3624 Jps

1.4.2.1. 步驟一：slave 啟動 DataNode

啟動 DataNode：

[hadoop@slave1 hadoop]$ hadoop-daemon.sh start datanode 
 
starting datanode, logging to /opt/module/hadoop2.7.1/logs/hadoop-hadoop-datanode-master.out 
 
[hadoop@slave2 hadoop]$ hadoop-daemon.sh start datanode 
 
starting datanode, logging to /opt/module/hadoop2.7.1/logs/hadoop-hadoop-datanode-master.out 
 
[hadoop@slave1 hadoop]$ jps 
 
3557 DataNode 3725 Jps 
 
[hadoop@slave2 hadoop]$ jps 
 
3557 DataNode 3725 Jps
 

1.4.2.2. 步驟二：啟動 SecondaryNameNode

[hadoop@master hadoop]$ hadoop-daemon.sh start secondarynamenode 
 
starting secondarynamenode, logging to /opt/module/hadoop2.7.1/logs/hadoop-hadoop-secondarynamenode-master.out 
 
[hadoop@master hadoop]$ jps 
 
34257 NameNode 
34449 SecondaryNameNode 
34494 Jps

1.4.2.3. 步驟三：檢視 HDFS 資料存放位置：

[hadoop@master hadoop]$ ll dfs/ 
 
總用量 0 drwx------ 3 hadoop hadoop 21 8 月  14 15:26 data drwxr-xr-x 3 hadoop hadoop 40 8 月  14 14:57 name  

[hadoop@master hadoop]$ ll ./tmp/dfs 
總用量 0 drwxrwxr-x. 3 hadoop hadoop 21 5 月   2 16:34 namesecondary 

1.4.3. 實驗任務三：檢視 HDFS 的報告

[hadoop@master sbin]$ hdfs dfsadmin -report 
 
Configured Capacity: 8202977280 (7.64 GB) Present Capacity: 4421812224 (4.12 GB) DFS Remaining: 4046110720 (3.77 GB) DFS Used: 375701504 (358.30 MB) DFS Used%: 8.50% Under replicated blocks: 88 Blocks with corrupt replicas: 0 Missing blocks: 0 
 
------------------------------------------------- Live datanodes (2): 
 
Name: 192.168.1.7:50010 (slave1) Hostname: slave1 Decommission Status : Normal Configured Capacity: 4101488640 (3.82 GB) DFS Used: 187850752 (179.15 MB) Non DFS Used: 2109939712 (1.97 GB) DFS Remaining: 1803698176 (1.68 GB) DFS Used%: 4.58% DFS Remaining%: 43.98% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) 


 
Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 1 Last contact: Mon May 04 18:32:32 CST 2020 
 
 
Name: 192.168.1.8:50010 (slave2) Hostname: slave2 Decommission Status : Normal Configured Capacity: 4101488640 (3.82 GB) DFS Used: 187850752 (179.15 MB) Non DFS Used: 1671225344 (1.56 GB) DFS Remaining: 2242412544 (2.09 GB) DFS Used%: 4.58% DFS Remaining%: 54.67% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 1 Last contact: Mon May 04 18:32:32 CST 2020 

1.4.4. 實驗任務四：使用瀏覽器檢視節點狀態

http://master:50070