2.4 Hadoop單機執行
阿新 • • 發佈：2018-12-04
第2章 Hadoop快速入門
2.4 Hadoop單機執行
緊接上一節內容，首先切換到Hadoop根目錄
或者cd /opt/hadoop-2.7.3進入Hadoop根目錄
通過pwd命令可以知道當前所在目錄
[root@node1 hadoop-2.7.3]# pwd
- 1
注意：本節命令都將在/opt/hadoop-2.7.3目錄下執行。
2.4.1 namenode格式化
執行bin/hadoop namenode -format
[root@node1 hadoop-2.7.3]# bin/hadoop namenode -format
- 1
輸出資訊(下半部分)：
-
17/
05/
12
05:
59:
11 INFO namenode
.NameNode
-
17/
05/
12
05:
59:
11 INFO namenode
.NameNode: createNameNode [-format]
-
Formatting using clusterid: CID-db9a34c9-
661e-4
-
17/
05/
12
05:
59:
12 INFO namenode
.FSNamesystem: No KeyProvider found.
-
17/
05/
12
05:
59:
12 INFO namenode
.FSNamesystem: fsLock is fair:true
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.DatanodeManager: dfs
.block
.invalidate
.limit=
1000
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.DatanodeManager: dfs
.namenode
.datanode
.registration
.ip-hostname-
check=
true
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: dfs
.namenode
.startup
.delay
.block
.deletion
.sec
is
set
to
000:
00:
00:
00.000
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: The
block deletion will
start around
2017 May
12
05:
59:
12
-
17/
05/
12
05:
59:
12 INFO util
.GSet: Computing
capacity
for
map BlocksMap
-
17/
05/
12
05:
59:
12 INFO util
.GSet: VM
type =
64-
bit
-
17/
05/
12
05:
59:
12 INFO util
.GSet:
2.0%
max
memory
966.7 MB =
19.3 MB
-
17/
05/
12
05:
59:
12 INFO util
.GSet:
capacity =
2^
21 =
2097152 entries
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: dfs
.block
.access
.token
.enable=
false
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: defaultReplication =
1
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: maxReplication =
512
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: minReplication =
1
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: maxReplicationStreams =
2
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: replicationRecheckInterval =
3000
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: encryptDataTransfer =
false
-
17/
05/
12
05:
59:
12 INFO blockmanagement
.BlockManager: maxNumBlocksToLog =
1000
-
17/
05/
12
05:
59:
12 INFO namenode
.FSNamesystem: fsOwner = root (auth:SIMPLE)
-
17/
05/
12
05:
59:
12 INFO namenode
.FSNamesystem: supergroup = supergroup
-
17/
05/
12
05:
59:
12 INFO namenode
.FSNamesystem: isPermissionEnabled =
true
-
17/
05/
12
05:
59:
12 INFO namenode
.FSNamesystem: HA Enabled:
false
-
17/
05/
12
05:
59:
12 INFO namenode
.FSNamesystem: Append Enabled:
true
-
17/
05/
12
05:
59:
13 INFO util
.GSet: Computing
capacity
for
map INodeMap
-
17/
05/
12
05:
59:
13 INFO util
.GSet: VM
type =
64-
bit
-
17/
05/
12
05:
59:
13 INFO util
.GSet:
1.0%
max
memory
966.7 MB =
9.7 MB
-
17/
05/
12
05:
59:
13 INFO util
.GSet:
capacity =
2^
20 =
1048576 entries
-
17/
05/
12
05:
59:
13 INFO namenode
.FSDirectory: ACLs enabled?
false
-
17/
05/
12
05:
59:
13 INFO namenode
.FSDirectory: XAttrs enabled?
true
-
17/
05/
12
05:
59:
13 INFO namenode
.FSDirectory: Maximum
size
of an xattr:
16384
-
17/
05/
12
05:
59:
13 INFO namenode
.NameNode:
Caching
file
names occuring more
than
10 times
-
17/
05/
12
05:
59:
13 INFO util
.GSet: Computing
capacity
for
map cachedBlocks
-
17/
05/
12
05:
59:
13 INFO util
.GSet: VM
type =
64-
bit
-
17/
05/
12
05:
59:
13 INFO util
.GSet:
0.25%
max
memory
966.7 MB =
2.4 MB
-
17/
05/
12
05:
59:
13 INFO util
.GSet:
capacity =
2^
18 =
262144 entries
-
17/
05/
12
05:
59:
13 INFO namenode
.FSNamesystem: dfs
.namenode
.safemode
.threshold-pct =
0.9990000128746033
-
17/
05/
12
05:
59:
13 INFO namenode
.FSNamesystem: dfs
.namenode
.safemode
.min
.datanodes =
0
-
17/
05/
12
05:
59:
13 INFO namenode
.FSNamesystem: dfs
.namenode
.safemode
.extension =
30000
-
17/
05/
12
05:
59:
13 INFO metrics
.TopMetrics: NNTop conf: dfs
.namenode
.top
.window
.num
.buckets =
10
-
17/
05/
12
05:
59:
13 INFO metrics
.TopMetrics: NNTop conf: dfs
.namenode
.top
.num
.users =
10
-
17/
05/
12
05:
59:
13 INFO metrics
.TopMetrics: NNTop conf: dfs
.namenode
.top
.windows
.minutes =
1,
5,
25
-
17/
05/
12
05:
59:
13 INFO namenode
.FSNamesystem: Retry
cache
on namenode
is enabled
-
17/
05/
12
05:
59:
13 INFO namenode
.FSNamesystem: Retry
cache will
use
0.03
of total
heap
and retry
cache entry expiry
time
is
600000 millis
-
17/
05/
12
05:
59:
13 INFO util
.GSet: Computing
capacity
for
map NameNodeRetryCache
-
17/
05/
12
05:
59:
13 INFO util
.GSet: VM
type =
64-
bit
-
17/
05/
12
05:
59:
13 INFO util
.GSet:
0.029999999329447746%
max
memory
966.7 MB =
297.0 KB
-
17/
05/
12
05:
59:
13 INFO util
.GSet:
capacity =
2^
15 =
32768 entries
-
17/
05/
12
05:
59:
13 INFO namenode
.FSImage: Allocated
new BlockPoolId: BP
-
2055065474
-
192.168
.80
.131
-
1494583153319
-
17/
05/
12
05:
59:
13 INFO common
.Storage:
Storage
directory /tmp/hadoop-root/dfs/
name has been successfully formatted.
-
17/
05/
12
05:
59:
13 INFO namenode
.FSImageFormatProtobuf: Saving image
file /tmp/hadoop-root/dfs/
name/
current/fsimage
.ckpt_0000000000000000000
using
no compression
-
17/
05/
12
05:
59:
13 INFO namenode
.FSImageFormatProtobuf: Image
file /tmp/hadoop-root/dfs/
name/
current/fsimage
.ckpt_0000000000000000000
of
size
351
bytes saved
in
0 seconds.
-
17/
05/
12
05:
59:
13 INFO namenode
.NNStorageRetentionManager: Going
to retain
1 images
with txid >=
0
-
17/
05/
12
05:
59:
13 INFO util
.ExitUtil: Exiting
with
status
0
-
17/
05/
12
05:
59:
13 INFO namenode
.NameNode: SHUTDOWN_MSG:
-
/************************************************************
-
SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.80.131
-
************************************************************/
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
在執行結果中可以找到has been successfully formatted，說明namenode格式化成功了！
2.4.2 啟動HDFS
執行sbin/start-dfs.sh命令啟動HDFS
-
[[email protected] hadoop
-
2.7
.3]
# sbin/start-dfs.sh
-
Starting namenodes
on [node1]
-
node1: starting namenode, logging
to /opt/hadoop
-
2.7
.3/logs/hadoop-root-namenode-node1.out
-
localhost: starting datanode, logging
to /opt/hadoop
-
2.7
.3/logs/hadoop-root-datanode-node1.out
-
Starting secondary namenodes [
0.0
.0
.0]
-
The authenticity
of host
'
0.0
.0
.0
(
0.0
.0
.0