[Sqoop02]Sqoop常用操作命令
阿新 • • 發佈：2019-02-16
18/01/06 15:18:54 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.7.0
18/01/06 15:18:54 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/01/06 15:18:55 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/01/06 15:18:55 INFO tool.CodeGenTool: Beginning code generation
18/01/06 15:18:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user` AS t LIMIT 1
18/01/06 15:18:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user` AS t LIMIT 1
18/01/06 15:18:57 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/apps/hadoop
Note: /tmp/sqoop-hadoop/compile/fdf0c2f391ba20be1190c670d7b71bb5/user.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/01/06 15:19:05 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/fdf0c2f391ba20be1190c670d7b71bb5/user.jar
18/01/06 15:19:05 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/01/06 15:19:05 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/01/06 15:19:05 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/01/06 15:19:05 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/01/06 15:19:05 WARN manager.CatalogQueryManager: The table user contains a multi-column primary key. Sqoop will default to the column Host only for this job.
18/01/06 15:19:05 WARN manager.CatalogQueryManager: The table user contains a multi-column primary key. Sqoop will default to the column Host only for this job.
18/01/06 15:19:05 INFO mapreduce.ImportJobBase: Beginning import of user
18/01/06 15:19:07 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/01/06 15:19:10 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/01/06 15:19:11 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
18/01/06 15:19:20 INFO db.DBInputFormat: Using read commited transaction isolation
18/01/06 15:19:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`Host`), MAX(`Host`) FROM `user`
18/01/06 15:19:20 WARN db.TextSplitter: Generating splits for a textual index column.
18/01/06 15:19:20 WARN db.TextSplitter: If your database sorts in a case-insensitive order, this may result in a partial import or duplicate records.
18/01/06 15:19:20 WARN db.TextSplitter: You are strongly encouraged to choose an integral split column.
18/01/06 15:19:20 INFO mapreduce.JobSubmitter: number of splits:5
18/01/06 15:19:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1515112781641_0004
18/01/06 15:19:24 INFO impl.YarnClientImpl: Submitted application application_1515112781641_0004
18/01/06 15:19:25 INFO mapreduce.Job: The url to track the job: http://hadoop001:8088/proxy/application_1515112781641_0004/
18/01/06 15:19:25 INFO mapreduce.Job: Running job: job_1515112781641_0004
18/01/06 15:20:07 INFO mapreduce.Job: Job job_1515112781641_0004 running in uber mode : false
18/01/06 15:20:07 INFO mapreduce.Job: map 0% reduce 0%
18/01/06 15:22:44 INFO mapreduce.Job: map 20% reduce 0%
18/01/06 15:22:50 INFO mapreduce.Job: map 40% reduce 0%
18/01/06 15:22:58 INFO mapreduce.Job: map 60% reduce 0%
18/01/06 15:22:59 INFO mapreduce.Job: map 80% reduce 0%
18/01/06 15:23:01 INFO mapreduce.Job: map 100% reduce 0%
18/01/06 15:23:02 INFO mapreduce.Job: Job job_1515112781641_0004 completed successfully
18/01/06 15:23:03 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=690395
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=655
HDFS: Number of bytes written=492
HDFS: Number of read operations=20
HDFS: Number of large read operations=0
HDFS: Number of write operations=10
Job Counters
Killed map tasks=1
Launched map tasks=6
Other local map tasks=6
Total time spent by all maps in occupied slots (ms)=824619
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=824619
Total vcore-seconds taken by all map tasks=824619
Total megabyte-seconds taken by all map tasks=844409856
Map-Reduce Framework
Map input records=4
Map output records=4
Input split bytes=655
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=34641
CPU time spent (ms)=18840
Physical memory (bytes) snapshot=551452672
Virtual memory (bytes) snapshot=7501139968
Total committed heap usage (bytes)=239206400
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=492
18/01/06 15:23:03 INFO mapreduce.ImportJobBase: Transferred 492 bytes in 232.3861 seconds (2.1172 bytes/sec)
18/01/06 15:23:03 INFO mapreduce.ImportJobBase: Retrieved 4 records.
18/01/06 15:18:54 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/01/06 15:18:55 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/01/06 15:18:55 INFO tool.CodeGenTool: Beginning code generation
18/01/06 15:18:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user` AS t LIMIT 1
18/01/06 15:18:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user` AS t LIMIT 1
18/01/06 15:18:57 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/apps/hadoop
Note: /tmp/sqoop-hadoop/compile/fdf0c2f391ba20be1190c670d7b71bb5/user.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/01/06 15:19:05 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/fdf0c2f391ba20be1190c670d7b71bb5/user.jar
18/01/06 15:19:05 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/01/06 15:19:05 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/01/06 15:19:05 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/01/06 15:19:05 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/01/06 15:19:05 WARN manager.CatalogQueryManager: The table user contains a multi-column primary key. Sqoop will default to the column Host only for this job.
18/01/06 15:19:05 WARN manager.CatalogQueryManager: The table user contains a multi-column primary key. Sqoop will default to the column Host only for this job.
18/01/06 15:19:05 INFO mapreduce.ImportJobBase: Beginning import of user
18/01/06 15:19:07 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/01/06 15:19:10 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/01/06 15:19:11 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
18/01/06 15:19:20 INFO db.DBInputFormat: Using read commited transaction isolation
18/01/06 15:19:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`Host`), MAX(`Host`) FROM `user`
18/01/06 15:19:20 WARN db.TextSplitter: Generating splits for a textual index column.
18/01/06 15:19:20 WARN db.TextSplitter: If your database sorts in a case-insensitive order, this may result in a partial import or duplicate records.
18/01/06 15:19:20 WARN db.TextSplitter: You are strongly encouraged to choose an integral split column.
18/01/06 15:19:20 INFO mapreduce.JobSubmitter: number of splits:5
18/01/06 15:19:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1515112781641_0004
18/01/06 15:19:24 INFO impl.YarnClientImpl: Submitted application application_1515112781641_0004
18/01/06 15:19:25 INFO mapreduce.Job: The url to track the job: http://hadoop001:8088/proxy/application_1515112781641_0004/
18/01/06 15:19:25 INFO mapreduce.Job: Running job: job_1515112781641_0004
18/01/06 15:20:07 INFO mapreduce.Job: Job job_1515112781641_0004 running in uber mode : false
18/01/06 15:20:07 INFO mapreduce.Job: map 0% reduce 0%
18/01/06 15:22:44 INFO mapreduce.Job: map 20% reduce 0%
18/01/06 15:22:50 INFO mapreduce.Job: map 40% reduce 0%
18/01/06 15:22:58 INFO mapreduce.Job: map 60% reduce 0%
18/01/06 15:22:59 INFO mapreduce.Job: map 80% reduce 0%
18/01/06 15:23:01 INFO mapreduce.Job: map 100% reduce 0%
18/01/06 15:23:02 INFO mapreduce.Job: Job job_1515112781641_0004 completed successfully
18/01/06 15:23:03 INFO mapreduce.Job: Counters: 31
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=690395
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=655
HDFS: Number of bytes written=492
HDFS: Number of read operations=20
HDFS: Number of large read operations=0
HDFS: Number of write operations=10
Job Counters
Killed map tasks=1
Launched map tasks=6
Other local map tasks=6
Total time spent by all maps in occupied slots (ms)=824619
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=824619
Total vcore-seconds taken by all map tasks=824619
Total megabyte-seconds taken by all map tasks=844409856
Map-Reduce Framework
Map input records=4
Map output records=4
Input split bytes=655
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=34641
CPU time spent (ms)=18840
Physical memory (bytes) snapshot=551452672
Virtual memory (bytes) snapshot=7501139968
Total committed heap usage (bytes)=239206400
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=492
18/01/06 15:23:03 INFO mapreduce.ImportJobBase: Transferred 492 bytes in 232.3861 seconds (2.1172 bytes/sec)
18/01/06 15:23:03 INFO mapreduce.ImportJobBase: Retrieved 4 records.