Hadoop系列二:啟動HDFS和YARN過程日誌
阿新 • • 發佈:2019-02-08
以下記錄為啟動HDFS和YARN環境時的過程中的中間過程輸出,節點註冊 和 HDFS檔案系統的資料變化日誌等。
(1)執行HDFS檔案系統格式化
[[email protected] hadoop]$ $HADOOP_PREFIX/bin/hdfs namenode -format
15/11/19 13:55:39 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = iRobot/192.168.100.200
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 2.0.0-cdh4.5.0
STARTUP_MSG: classpath = /home/bruce/hadoop-install/hadoop/etc/hadoop:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-api-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/activation-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-json-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/kfs-0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-auth-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/cloudera-jets3t-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-math-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-site-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG: build = file:///var/lib/jenkins/workspace/CDH4.5.0-Packaging-Hadoop/build/cdh4/hadoop/2.0.0-cdh4.5.0/source/hadoop-common-project/hadoop-common -r 30821ec616ee7a21ee8447949b7c6208a8f1e7d8; compiled by 'jenkins' on Wed Nov 20 14:35:49 PST 2013
STARTUP_MSG: java = 1.6.0_21
************************************************************/
15/11/19 13:55:39 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/11/19 13:55:39 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Formatting using clusterid: CID-4fb81ac3-7e61-42c2-a707-46f2af89a6e2
15/11/19 13:55:39 INFO blockmanagement.HeartbeatManager: Setting heartbeat recheck interval to 30000 since dfs.namenode.stale.datanode.interval is less than dfs.namenode.heartbeat.recheck-interval
15/11/19 13:55:39 INFO namenode.HostFileManager: read includes:
HostSet(
)
15/11/19 13:55:39 INFO namenode.HostFileManager: read excludes:
HostSet(
)
15/11/19 13:55:39 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
15/11/19 13:55:39 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
15/11/19 13:55:39 INFO blockmanagement.BlockManager: defaultReplication = 1
15/11/19 13:55:39 INFO blockmanagement.BlockManager: maxReplication = 512
15/11/19 13:55:39 INFO blockmanagement.BlockManager: minReplication = 1
15/11/19 13:55:39 INFO blockmanagement.BlockManager: maxReplicationStreams = 2
15/11/19 13:55:39 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false
15/11/19 13:55:39 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
15/11/19 13:55:39 INFO blockmanagement.BlockManager: encryptDataTransfer = false
15/11/19 13:55:39 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000
15/11/19 13:55:39 INFO namenode.FSNamesystem: fsOwner = bruce (auth:SIMPLE)
15/11/19 13:55:39 INFO namenode.FSNamesystem: supergroup = oinstall
15/11/19 13:55:39 INFO namenode.FSNamesystem: isPermissionEnabled = false
15/11/19 13:55:39 INFO namenode.FSNamesystem: HA Enabled: false
15/11/19 13:55:39 INFO namenode.FSNamesystem: Append Enabled: true
15/11/19 13:55:40 INFO namenode.NameNode: Caching file names occuring more than 10 times
15/11/19 13:55:40 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
15/11/19 13:55:40 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
15/11/19 13:55:40 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000
15/11/19 13:55:40 INFO namenode.NNStorage: Storage directory /u01/hadoopdata/hadoop-bruce/dfs/name has been successfully formatted.
15/11/19 13:55:40 INFO namenode.FSImage: Saving image file /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
15/11/19 13:55:40 INFO namenode.FSImage: Image file of size 116 saved in 0 seconds.
15/11/19 13:55:40 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
15/11/19 13:55:40 INFO util.ExitUtil: Exiting with status 0
15/11/19 13:55:40 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at iRobot/192.168.100.200
************************************************************/
[ [email protected] hadoop]$
(2)執行HDFS檔案系統格式化後,相應的HDFS下的naemode fsimage產生
[[email protected] hadoop]$ ls -al /u01/hadoopdata/hadoop-bruce/dfs/name/current/
總用量 24
drwxr-xr-x 2 bruce oinstall 4096 11月 19 13:55 .
drwxr-xr-x 3 bruce oinstall 4096 11月 19 13:55 ..
-rw-r--r-- 1 bruce oinstall 116 11月 19 13:55 fsimage_0000000000000000000
-rw-r--r-- 1 bruce oinstall 62 11月 19 13:55 fsimage_0000000000000000000.md5
-rw-r--r-- 1 bruce oinstall 2 11月 19 13:55 seen_txid
-rw-r--r-- 1 bruce oinstall 206 11月 19 13:55 VERSION
[ [email protected] hadoop]$ cat /u01/hadoopdata/hadoop-bruce/dfs/name/current/VERSION
#Thu Nov 19 13:55:40 CST 2015
namespaceID=337807999
clusterID=CID-4fb81ac3-7e61-42c2-a707-46f2af89a6e2
cTime=0
storageType=NAME_NODE
blockpoolID=BP-981411196-192.168.100.200-1447912540337
layoutVersion=-40
[[email protected] hadoop]$ cat /u01/hadoopdata/hadoop-bruce/dfs/name/current/seen_txid
0
[ [email protected] hadoop]$
[[email protected] hadoop]$
[[email protected] hadoop]$ cat /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage_0000000000000000000.md5
2014cc5c2a85e352064adb7bc4555fe7 *fsimage_0000000000000000000
[[email protected] hadoop]$
(3)啟動NameNode: 產生hadoop-bruce-namenode-iRobot.log, hadoop-bruce-namenode-iRobot.out 和 SecurityAuth-bruce.audit
[[email protected] hadoop]$ $HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script hdfs start namenode
starting namenode, logging to /home/bruce/hadoop-install/hadoop/logs/hadoop-bruce-namenode-iRobot.out
[[email protected] hadoop]$ jps
22880 Bootstrap
24986 Main
6251 Jps
5355 Launcher
5748 Main
6158 NameNode
[[email protected] hadoop]$ ls logs
hadoop-bruce-namenode-iRobot.log hadoop-bruce-namenode-iRobot.out SecurityAuth-bruce.audit
[[email protected] hadoop]$ cat logs/hadoop-bruce-namenode-iRobot.log
2015-11-19 14:02:21,206 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = iRobot/192.168.100.200
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.0.0-cdh4.5.0
STARTUP_MSG: classpath = /home/bruce/hadoop-install/hadoop/etc/hadoop:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-api-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/activation-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-json-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/kfs-0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-auth-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/cloudera-jets3t-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-math-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-site-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG: build = file:///var/lib/jenkins/workspace/CDH4.5.0-Packaging-Hadoop/build/cdh4/hadoop/2.0.0-cdh4.5.0/source/hadoop-common-project/hadoop-common -r 30821ec616ee7a21ee8447949b7c6208a8f1e7d8; compiled by 'jenkins' on Wed Nov 20 14:35:49 PST 2013
STARTUP_MSG: java = 1.6.0_21
************************************************************/
2015-11-19 14:02:21,211 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2015-11-19 14:02:21,464 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-11-19 14:02:21,543 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-11-19 14:02:21,543 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started
2015-11-19 14:02:21,559 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-11-19 14:02:21,907 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of dataloss due to lack of redundant storage directories!
2015-11-19 14:02:21,907 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of dataloss due to lack of redundant storage directories!
2015-11-19 14:02:21,982 INFO org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager: Setting heartbeat recheck interval to 30000 since dfs.namenode.stale.datanode.interval is less than dfs.namenode.heartbeat.recheck-interval
2015-11-19 14:02:21,983 INFO org.apache.hadoop.hdfs.server.namenode.HostFileManager: read includes:
HostSet(
)
2015-11-19 14:02:21,983 INFO org.apache.hadoop.hdfs.server.namenode.HostFileManager: read excludes:
HostSet(
)
2015-11-19 14:02:21,986 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.block.access.token.enable=false
2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: defaultReplication = 1
2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication = 512
2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication = 1
2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplicationStreams = 2
2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: shouldCheckForEnoughRacks = false
2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: replicationRecheckInterval = 3000
2015-11-19 14:02:21,999 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: encryptDataTransfer = false
2015-11-19 14:02:21,999 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxNumBlocksToLog = 1000
2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner = bruce (auth:SIMPLE)
2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup = oinstall
2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled = false
2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false
2015-11-19 14:02:22,008 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled: true
2015-11-19 14:02:22,406 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times
2015-11-19 14:02:22,408 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2015-11-19 14:02:22,408 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
2015-11-19 14:02:22,408 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000
2015-11-19 14:02:22,436 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /u01/hadoopdata/hadoop-bruce/dfs/name/in_use.lock acquired by nodename [email protected]
2015-11-19 14:02:22,513 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Recovering unfinalized segments in /u01/hadoopdata/hadoop-bruce/dfs/name/current
2015-11-19 14:02:22,516 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: No edit log streams selected.
2015-11-19 14:02:22,522 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loading image file /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage_0000000000000000000 using no compression
2015-11-19 14:02:22,522 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Number of files = 1
2015-11-19 14:02:22,523 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Number of files under construction = 0
2015-11-19 14:02:22,523 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Image file of size 116 loaded in 0 seconds.
2015-11-19 14:02:22,524 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loaded image for txid 0 from /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage_0000000000000000000
2015-11-19 14:02:22,528 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 1
2015-11-19 14:02:22,640 INFO org.apache.hadoop.hdfs.server.namenode.NameCache: initialized with 0 entries 0 lookups
2015-11-19 14:02:22,641 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading FSImage in 231 msecs
2015-11-19 14:02:22,796 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 9000
2015-11-19 14:02:22,821 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemState MBean
2015-11-19 14:02:22,838 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of blocks under construction: 0
2015-11-19 14:02:22,839 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: initializing replication queues
2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Total number of blocks = 0
2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of invalid blocks = 0
2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of under-replicated blocks = 0
2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of over-replicated blocks = 0
2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of blocks being written = 0
2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.StateChange: STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 14 msec
2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.StateChange: STATE* Leaving safe mode after 0 secs
2015-11-19 14:02:22,854 INFO org.apache.hadoop.hdfs.StateChange: STATE* Network topology has 0 racks and 0 datanodes
2015-11-19 14:02:22,854 INFO org.apache.hadoop.hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks
2015-11-19 14:02:22,915 INFO org.apache.hadoop.http.HttpServer: SSL is enabled on HttpServer at http://0.0.0.0:-1/ - not live
2015-11-19 14:02:22,920 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2015-11-19 14:02:22,979 INFO org.apache.hadoop.http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2015-11-19 14:02:22,982 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
2015-11-19 14:02:22,982 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2015-11-19 14:02:22,982 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2015-11-19 14:02:22,994 INFO org.apache.hadoop.http.HttpServer: dfs.webhdfs.enabled = true
2015-11-19 14:02:22,995 INFO org.apache.hadoop.http.HttpServer: Added filter 'SPNEGO' (class=org.apache.hadoop.hdfs.web.AuthFilter)
2015-11-19 14:02:22,997 INFO org.apache.hadoop.http.HttpServer: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
2015-11-19 14:02:23,010 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50070
2015-11-19 14:02:23,010 INFO org.mortbay.log: jetty-6.1.26.cloudera.2
2015-11-19 14:02:23,159 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: 'signature.secret' configuration not set, using a random value as secret
2015-11-19 14:02:23,200 INFO org.mortbay.log: Started [email protected]:50070
2015-11-19 14:02:23,200 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: 0.0.0.0:50070
2015-11-19 14:02:23,201 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-11-19 14:02:23,201 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9000: starting
2015-11-19 14:02:23,205 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode up at: iRobot/192.168.100.200:9000
2015-11-19 14:02:23,205 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state
[[email protected] hadoop]$ cat logs/hadoop-bruce-namenode-iRobot.out
ulimit -a for user bruce
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 62702
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
(4)啟動DataNode:
[[email protected] hadoop]$ $HADOOP_PREFIX/sbin/hadoop-daemons.sh --config $HADOOP_CONF_DIR --script hdfs start datanode
iRobot: starting datanode, logging to /home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/logs/hadoop-bruce-datanode-iRobot.out
[[email protected] hadoop]$ jps
22880 Bootstrap
6328 DataNode
24986 Main
6408 Jps
5355 Launcher
5748 Main
6158 NameNode
[[email protected] hadoop]$ ls logs
hadoop-bruce-datanode-iRobot.log hadoop-bruce-datanode-iRobot.out hadoop-bruce-namenode-iRobot.log hadoop-bruce-namenode-iRobot.out SecurityAuth-bruce.audit
[[email protected] hadoop]$
[[email protected] hadoop]$ cat logs/hadoop-bruce-datanode-iRobot.log
2015-11-19 14:06:19,201 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG: host = iRobot/192.168.100.200
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.0.0-cdh4.5.0
STARTUP_MSG: classpath = /home/bruce/hadoop-install/hadoop/etc/hadoop:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-digester-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/slf4j-api-1.6.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/activation-1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jersey-json-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jettison-1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/kfs-0.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/junit-4.8.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jsch-0.1.42.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-net-3.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/hadoop-auth-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/cloudera-jets3t-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-math-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-daemon-1.0.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-client-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-site-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-api-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar
STARTUP_MSG: build = file:///var/lib/jenkins/workspace/CDH4.5.0-Packaging-Hadoop/build/cdh4/hadoop/2.0.0-cdh4.5.0/source/hadoop-common-project/hadoop-common -r 30821ec616ee7a21ee8447949b7c6208a8f1e7d8; compiled by 'jenkins' on Wed Nov 20 14:35:49 PST 2013
STARTUP_MSG: java = 1.6.0_21
************************************************************/
2015-11-19 14:06:19,206 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT]
2015-11-19 14:06:19,432 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-11-19 14:06:19,906 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-11-19 14:06:19,961 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-11-19 14:06:19,961 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2015-11-19 14:06:19,964 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is iRobot
2015-11-19 14:06:19,998 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /192.168.100.200:50010
2015-11-19 14:06:20,001 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s
2015-11-19 14:06:20,037 INFO org.apache.hadoop.http.HttpServer: SSL is enabled on HttpServer at http://0.0.0.0:-1/ - not live
2015-11-19 14:06:20,042 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2015-11-19 14:06:20,094 INFO org.apache.hadoop.http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2015-11-19 14:06:20,096 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
2015-11-19 14:06:20,096 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2015-11-19 14:06:20,096 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2015-11-19 14:06:20,100 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 0.0.0.0:50075
2015-11-19 14:06:20,103 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dfs.webhdfs.enabled = true
2015-11-19 14:06:20,104 INFO org.apache.hadoop.http.HttpServer: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
2015-11-19 14:06:20,108 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50075
2015-11-19 14:06:20,108 INFO org.mortbay.log: jetty-6.1.26.cloudera.2
2015-11-19 14:06:20,300 INFO org.mortbay.log: Started [email protected]:50075
2015-11-19 14:06:20,677 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020
2015-11-19 14:06:20,700 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020
2015-11-19 14:06:20,711 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null
2015-11-19 14:06:20,732 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
2015-11-19 14:06:20,740 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (storage id unknown) service to iRobot/192.168.100.200:9000 starting to offer service
2015-11-19 14:06:20,745 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-11-19 14:06:20,745 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2015-11-19 14:06:21,306 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /u01/hadoopdata/hadoop-bruce/dfs/data/in_use.lock acquired by nodename [email protected]
2015-11-19 14:06:21,308 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /u01/hadoopdata/hadoop-bruce/dfs/data is not formatted
2015-11-19 14:06:21,308 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2015-11-19 14:06:21,420 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled
2015-11-19 14:06:21,422 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /u01/hadoopdata/hadoop-bruce/dfs/data/current/BP-981411196-192.168.100.200-1447912540337 is not formatted.
2015-11-19 14:06:21,422 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2015-11-19 14:06:21,422 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting block pool BP-981411196-192.168.100.200-1447912540337 directory /u01/hadoopdata/hadoop-bruce/dfs/data/current/BP-981411196-192.168.100.200-1447912540337/current
2015-11-19 14:06:21,446 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: nsid=337807999;bpid=BP-981411196-192.168.100.200-1447912540337;lv=-40;nsInfo=lv=-40;cid=CID-4fb81ac3-7e61-42c2-a707-46f2af89a6e2;nsid=337807999;c=0;bpid=BP-981411196-192.168.100.200-1447912540337
2015-11-19 14:06:21,460 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.