1. 程式人生 > >Hadoop系列二:啟動HDFS和YARN過程日誌

Hadoop系列二:啟動HDFS和YARN過程日誌

以下記錄為啟動HDFS和YARN環境時的過程中的中間過程輸出,節點註冊 和 HDFS檔案系統的資料變化日誌等。


(1)執行HDFS檔案系統格式化
[[email protected] hadoop]$ $HADOOP_PREFIX/bin/hdfs namenode -format
15/11/19 13:55:39 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = iRobot/192.168.100.200
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 2.0.0-cdh4.5.0
STARTUP_MSG:   classpath = /home/bruce/hadoop-install/hadoop/etc/hadoop:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-api-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/activation-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-json-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/kfs-0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-auth-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/cloudera-jets3t-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-math-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-site-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = file:///var/lib/jenkins/workspace/CDH4.5.0-Packaging-Hadoop/build/cdh4/hadoop/2.0.0-cdh4.5.0/source/hadoop-common-project/hadoop-common -r 30821ec616ee7a21ee8447949b7c6208a8f1e7d8; compiled by 'jenkins' on Wed Nov 20 14:35:49 PST 2013
STARTUP_MSG:   java = 1.6.0_21
************************************************************/
15/11/19 13:55:39 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/11/19 13:55:39 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Formatting using clusterid: CID-4fb81ac3-7e61-42c2-a707-46f2af89a6e2
15/11/19 13:55:39 INFO blockmanagement.HeartbeatManager: Setting heartbeat recheck interval to 30000 since dfs.namenode.stale.datanode.interval is less than dfs.namenode.heartbeat.recheck-interval
15/11/19 13:55:39 INFO namenode.HostFileManager: read includes:
HostSet(
)
15/11/19 13:55:39 INFO namenode.HostFileManager: read excludes:
HostSet(
)
15/11/19 13:55:39 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
15/11/19 13:55:39 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
15/11/19 13:55:39 INFO blockmanagement.BlockManager: defaultReplication         = 1
15/11/19 13:55:39 INFO blockmanagement.BlockManager: maxReplication             = 512
15/11/19 13:55:39 INFO blockmanagement.BlockManager: minReplication             = 1
15/11/19 13:55:39 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
15/11/19 13:55:39 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false
15/11/19 13:55:39 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
15/11/19 13:55:39 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
15/11/19 13:55:39 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
15/11/19 13:55:39 INFO namenode.FSNamesystem: fsOwner             = bruce (auth:SIMPLE)
15/11/19 13:55:39 INFO namenode.FSNamesystem: supergroup          = oinstall
15/11/19 13:55:39 INFO namenode.FSNamesystem: isPermissionEnabled = false
15/11/19 13:55:39 INFO namenode.FSNamesystem: HA Enabled: false
15/11/19 13:55:39 INFO namenode.FSNamesystem: Append Enabled: true
15/11/19 13:55:40 INFO namenode.NameNode: Caching file names occuring more than 10 times
15/11/19 13:55:40 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
15/11/19 13:55:40 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
15/11/19 13:55:40 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000
15/11/19 13:55:40 INFO namenode.NNStorage: Storage directory /u01/hadoopdata/hadoop-bruce/dfs/name has been successfully formatted.
15/11/19 13:55:40 INFO namenode.FSImage: Saving image file /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
15/11/19 13:55:40 INFO namenode.FSImage: Image file of size 116 saved in 0 seconds.
15/11/19 13:55:40 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
15/11/19 13:55:40 INFO util.ExitUtil: Exiting with status 0
15/11/19 13:55:40 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at iRobot/192.168.100.200
************************************************************/
[
[email protected]
hadoop]$ (2)執行HDFS檔案系統格式化後,相應的HDFS下的naemode fsimage產生 [[email protected] hadoop]$ ls -al /u01/hadoopdata/hadoop-bruce/dfs/name/current/ 總用量 24 drwxr-xr-x 2 bruce oinstall 4096 11月 19 13:55 . drwxr-xr-x 3 bruce oinstall 4096 11月 19 13:55 .. -rw-r--r-- 1 bruce oinstall 116 11月 19 13:55 fsimage_0000000000000000000 -rw-r--r-- 1 bruce oinstall 62 11月 19 13:55 fsimage_0000000000000000000.md5 -rw-r--r-- 1 bruce oinstall 2 11月 19 13:55 seen_txid -rw-r--r-- 1 bruce oinstall 206 11月 19 13:55 VERSION [
[email protected]
hadoop]$ cat /u01/hadoopdata/hadoop-bruce/dfs/name/current/VERSION #Thu Nov 19 13:55:40 CST 2015 namespaceID=337807999 clusterID=CID-4fb81ac3-7e61-42c2-a707-46f2af89a6e2 cTime=0 storageType=NAME_NODE blockpoolID=BP-981411196-192.168.100.200-1447912540337 layoutVersion=-40 [[email protected] hadoop]$ cat /u01/hadoopdata/hadoop-bruce/dfs/name/current/seen_txid 0 [
[email protected]
hadoop]$ [[email protected] hadoop]$ [[email protected] hadoop]$ cat /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage_0000000000000000000.md5 2014cc5c2a85e352064adb7bc4555fe7 *fsimage_0000000000000000000 [[email protected] hadoop]$ (3)啟動NameNode: 產生hadoop-bruce-namenode-iRobot.log, hadoop-bruce-namenode-iRobot.out 和 SecurityAuth-bruce.audit [[email protected] hadoop]$ $HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script hdfs start namenode starting namenode, logging to /home/bruce/hadoop-install/hadoop/logs/hadoop-bruce-namenode-iRobot.out [[email protected] hadoop]$ jps 22880 Bootstrap 24986 Main 6251 Jps 5355 Launcher 5748 Main 6158 NameNode [[email protected] hadoop]$ ls logs hadoop-bruce-namenode-iRobot.log hadoop-bruce-namenode-iRobot.out SecurityAuth-bruce.audit [[email protected] hadoop]$ cat logs/hadoop-bruce-namenode-iRobot.log 2015-11-19 14:02:21,206 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = iRobot/192.168.100.200 STARTUP_MSG: args = [] STARTUP_MSG: version = 2.0.0-cdh4.5.0 STARTUP_MSG: classpath = /home/bruce/hadoop-install/hadoop/etc/hadoop:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/slf4j-api-1.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/activation-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-json-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/kfs-0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-auth-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/cloudera-jets3t-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-math-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-site-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar:/home/bruce/hadoop-install/hadoop/contrib/capacity-scheduler/*.jar STARTUP_MSG: build = file:///var/lib/jenkins/workspace/CDH4.5.0-Packaging-Hadoop/build/cdh4/hadoop/2.0.0-cdh4.5.0/source/hadoop-common-project/hadoop-common -r 30821ec616ee7a21ee8447949b7c6208a8f1e7d8; compiled by 'jenkins' on Wed Nov 20 14:35:49 PST 2013 STARTUP_MSG: java = 1.6.0_21 ************************************************************/ 2015-11-19 14:02:21,211 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 2015-11-19 14:02:21,464 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2015-11-19 14:02:21,543 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2015-11-19 14:02:21,543 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started 2015-11-19 14:02:21,559 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2015-11-19 14:02:21,907 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of dataloss due to lack of redundant storage directories! 2015-11-19 14:02:21,907 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of dataloss due to lack of redundant storage directories! 2015-11-19 14:02:21,982 INFO org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager: Setting heartbeat recheck interval to 30000 since dfs.namenode.stale.datanode.interval is less than dfs.namenode.heartbeat.recheck-interval 2015-11-19 14:02:21,983 INFO org.apache.hadoop.hdfs.server.namenode.HostFileManager: read includes: HostSet( ) 2015-11-19 14:02:21,983 INFO org.apache.hadoop.hdfs.server.namenode.HostFileManager: read excludes: HostSet( ) 2015-11-19 14:02:21,986 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000 2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.block.access.token.enable=false 2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: defaultReplication = 1 2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication = 512 2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication = 1 2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplicationStreams = 2 2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: shouldCheckForEnoughRacks = false 2015-11-19 14:02:21,998 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: replicationRecheckInterval = 3000 2015-11-19 14:02:21,999 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: encryptDataTransfer = false 2015-11-19 14:02:21,999 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxNumBlocksToLog = 1000 2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner = bruce (auth:SIMPLE) 2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup = oinstall 2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled = false 2015-11-19 14:02:22,004 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false 2015-11-19 14:02:22,008 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled: true 2015-11-19 14:02:22,406 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times 2015-11-19 14:02:22,408 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 2015-11-19 14:02:22,408 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0 2015-11-19 14:02:22,408 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000 2015-11-19 14:02:22,436 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /u01/hadoopdata/hadoop-bruce/dfs/name/in_use.lock acquired by nodename [email protected] 2015-11-19 14:02:22,513 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Recovering unfinalized segments in /u01/hadoopdata/hadoop-bruce/dfs/name/current 2015-11-19 14:02:22,516 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: No edit log streams selected. 2015-11-19 14:02:22,522 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loading image file /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage_0000000000000000000 using no compression 2015-11-19 14:02:22,522 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Number of files = 1 2015-11-19 14:02:22,523 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Number of files under construction = 0 2015-11-19 14:02:22,523 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Image file of size 116 loaded in 0 seconds. 2015-11-19 14:02:22,524 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loaded image for txid 0 from /u01/hadoopdata/hadoop-bruce/dfs/name/current/fsimage_0000000000000000000 2015-11-19 14:02:22,528 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 1 2015-11-19 14:02:22,640 INFO org.apache.hadoop.hdfs.server.namenode.NameCache: initialized with 0 entries 0 lookups 2015-11-19 14:02:22,641 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading FSImage in 231 msecs 2015-11-19 14:02:22,796 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 9000 2015-11-19 14:02:22,821 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemState MBean 2015-11-19 14:02:22,838 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of blocks under construction: 0 2015-11-19 14:02:22,839 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: initializing replication queues 2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Total number of blocks = 0 2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of invalid blocks = 0 2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of under-replicated blocks = 0 2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of over-replicated blocks = 0 2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of blocks being written = 0 2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.StateChange: STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 14 msec 2015-11-19 14:02:22,853 INFO org.apache.hadoop.hdfs.StateChange: STATE* Leaving safe mode after 0 secs 2015-11-19 14:02:22,854 INFO org.apache.hadoop.hdfs.StateChange: STATE* Network topology has 0 racks and 0 datanodes 2015-11-19 14:02:22,854 INFO org.apache.hadoop.hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks 2015-11-19 14:02:22,915 INFO org.apache.hadoop.http.HttpServer: SSL is enabled on HttpServer at http://0.0.0.0:-1/ - not live 2015-11-19 14:02:22,920 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2015-11-19 14:02:22,979 INFO org.apache.hadoop.http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2015-11-19 14:02:22,982 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs 2015-11-19 14:02:22,982 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-19 14:02:22,982 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-19 14:02:22,994 INFO org.apache.hadoop.http.HttpServer: dfs.webhdfs.enabled = true 2015-11-19 14:02:22,995 INFO org.apache.hadoop.http.HttpServer: Added filter 'SPNEGO' (class=org.apache.hadoop.hdfs.web.AuthFilter) 2015-11-19 14:02:22,997 INFO org.apache.hadoop.http.HttpServer: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/* 2015-11-19 14:02:23,010 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50070 2015-11-19 14:02:23,010 INFO org.mortbay.log: jetty-6.1.26.cloudera.2 2015-11-19 14:02:23,159 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: 'signature.secret' configuration not set, using a random value as secret 2015-11-19 14:02:23,200 INFO org.mortbay.log: Started [email protected]:50070 2015-11-19 14:02:23,200 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: 0.0.0.0:50070 2015-11-19 14:02:23,201 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2015-11-19 14:02:23,201 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9000: starting 2015-11-19 14:02:23,205 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode up at: iRobot/192.168.100.200:9000 2015-11-19 14:02:23,205 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state [[email protected] hadoop]$ cat logs/hadoop-bruce-namenode-iRobot.out ulimit -a for user bruce core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 62702 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited (4)啟動DataNode: [[email protected] hadoop]$ $HADOOP_PREFIX/sbin/hadoop-daemons.sh --config $HADOOP_CONF_DIR --script hdfs start datanode iRobot: starting datanode, logging to /home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/logs/hadoop-bruce-datanode-iRobot.out [[email protected] hadoop]$ jps 22880 Bootstrap 6328 DataNode 24986 Main 6408 Jps 5355 Launcher 5748 Main 6158 NameNode [[email protected] hadoop]$ ls logs hadoop-bruce-datanode-iRobot.log hadoop-bruce-datanode-iRobot.out hadoop-bruce-namenode-iRobot.log hadoop-bruce-namenode-iRobot.out SecurityAuth-bruce.audit [[email protected] hadoop]$ [[email protected] hadoop]$ cat logs/hadoop-bruce-datanode-iRobot.log 2015-11-19 14:06:19,201 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = iRobot/192.168.100.200 STARTUP_MSG: args = [] STARTUP_MSG: version = 2.0.0-cdh4.5.0 STARTUP_MSG: classpath = /home/bruce/hadoop-install/hadoop/etc/hadoop:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-digester-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/slf4j-api-1.6.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/activation-1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jersey-json-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jettison-1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/kfs-0.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/junit-4.8.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jsch-0.1.42.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-net-3.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/hadoop-auth-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/cloudera-jets3t-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-math-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/common/hadoop-common-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/zookeeper-3.4.5-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jline-0.9.94.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/commons-daemon-1.0.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/hdfs/hadoop-hdfs-2.0.0-cdh4.5.0-test-sources.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-client-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-site-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-api-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/protobuf-java-2.4.0a.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jersey-core-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jersey-guice-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/netty-3.2.4.Final.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/jersey-server-1.8.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0-tests.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.0.0-cdh4.5.0.jar:/home/bruce/hadoop-install/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar STARTUP_MSG: build = file:///var/lib/jenkins/workspace/CDH4.5.0-Packaging-Hadoop/build/cdh4/hadoop/2.0.0-cdh4.5.0/source/hadoop-common-project/hadoop-common -r 30821ec616ee7a21ee8447949b7c6208a8f1e7d8; compiled by 'jenkins' on Wed Nov 20 14:35:49 PST 2013 STARTUP_MSG: java = 1.6.0_21 ************************************************************/ 2015-11-19 14:06:19,206 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT] 2015-11-19 14:06:19,432 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2015-11-19 14:06:19,906 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2015-11-19 14:06:19,961 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2015-11-19 14:06:19,961 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2015-11-19 14:06:19,964 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is iRobot 2015-11-19 14:06:19,998 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /192.168.100.200:50010 2015-11-19 14:06:20,001 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s 2015-11-19 14:06:20,037 INFO org.apache.hadoop.http.HttpServer: SSL is enabled on HttpServer at http://0.0.0.0:-1/ - not live 2015-11-19 14:06:20,042 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2015-11-19 14:06:20,094 INFO org.apache.hadoop.http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2015-11-19 14:06:20,096 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode 2015-11-19 14:06:20,096 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2015-11-19 14:06:20,096 INFO org.apache.hadoop.http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2015-11-19 14:06:20,100 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 0.0.0.0:50075 2015-11-19 14:06:20,103 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dfs.webhdfs.enabled = true 2015-11-19 14:06:20,104 INFO org.apache.hadoop.http.HttpServer: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/* 2015-11-19 14:06:20,108 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50075 2015-11-19 14:06:20,108 INFO org.mortbay.log: jetty-6.1.26.cloudera.2 2015-11-19 14:06:20,300 INFO org.mortbay.log: Started [email protected]:50075 2015-11-19 14:06:20,677 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020 2015-11-19 14:06:20,700 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020 2015-11-19 14:06:20,711 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null 2015-11-19 14:06:20,732 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default> 2015-11-19 14:06:20,740 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (storage id unknown) service to iRobot/192.168.100.200:9000 starting to offer service 2015-11-19 14:06:20,745 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2015-11-19 14:06:20,745 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting 2015-11-19 14:06:21,306 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /u01/hadoopdata/hadoop-bruce/dfs/data/in_use.lock acquired by nodename [email protected] 2015-11-19 14:06:21,308 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /u01/hadoopdata/hadoop-bruce/dfs/data is not formatted 2015-11-19 14:06:21,308 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ... 2015-11-19 14:06:21,420 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled 2015-11-19 14:06:21,422 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /u01/hadoopdata/hadoop-bruce/dfs/data/current/BP-981411196-192.168.100.200-1447912540337 is not formatted. 2015-11-19 14:06:21,422 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ... 2015-11-19 14:06:21,422 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting block pool BP-981411196-192.168.100.200-1447912540337 directory /u01/hadoopdata/hadoop-bruce/dfs/data/current/BP-981411196-192.168.100.200-1447912540337/current 2015-11-19 14:06:21,446 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: nsid=337807999;bpid=BP-981411196-192.168.100.200-1447912540337;lv=-40;nsInfo=lv=-40;cid=CID-4fb81ac3-7e61-42c2-a707-46f2af89a6e2;nsid=337807999;c=0;bpid=BP-981411196-192.168.100.200-1447912540337 2015-11-19 14:06:21,460 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.