sqoop  mysql 匯入，匯出
1.
安裝（前提hadoop啟動）

[[email protected] ~]$ tar -zxvf sqoop-1.3.0-cdh3u5.tar.gz
[[email protected] hadoop-0.20.2-cdh3u5]$ cp hadoop-core-0.20.2-cdh3u5.jar /home/hadoop/sqoop-1.3.0-cdh3u5/lib/

[[email protected] ~]$ cp ojdbc6.jar sqoop-1.3.0-cdh3u5/lib/

[[email protected] ~]$ vi .bash_profile
新增
export SQOOP_HOME=/home/hadoop/sqoop-1.3.0-cdh3u5

[

[email protected] ~]$ source .bash_profile

2.

[[email protected] ~]$ cd sqoop-1.3.0-cdh3u5/bin/

[[email protected] bin]$ vi configure-sqoop
註釋掉hbase和zookeeper檢查
## Moved to be a runtime check in sqoop.
#if [ ! -d "${HBASE_HOME}" ]; then
#  echo "Warning: $HBASE_HOME does not exist! HBase imports will fail."
#  echo 'Please set $HBASE_HOME to the root of your HBase installation.'
#fi


3.mysql 授權

mysql> insert into mysql.user(Host,User,Password) values("localhost","sqoop",password("sqoop"));
mysql> flush privileges;
mysql> grant all privileges on *.* to 'sqoop'@'%' identified by 'sqoop' with grant option;
mysql> flush privileges;

mysql> use test;
mysql> create table sss (id int,name varchar(10));

mysql> insert into sss values(1,'zs');
mysql> insert into sss values(2,'ls');




4.測試sqoop能否連線上mysql
[

[email protected] mysql-connector-java-5.0.7]$ cp mysql-connector-java-5.0.7-bin.jar /home/hadoop/sqoop-1.3.0-cdh3u5/lib/

[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop list-tables --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456
（顯示有sss表）

5.將mysql中的sqoop使用者下sss 匯入到HDFS中
[

[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop import --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456 --table sss -m 1
（-m 為並行  預設並行度為4）

[[email protected] hadoop-0.20.2-cdh3u5]$ bin/hadoop fs -ls
多出個sss目錄
[[email protected] hadoop-0.20.2-cdh3u5]$ bin/hadoop fs -ls /user/hadoop/sss
[[email protected] hadoop-0.20.2-cdh3u5]$ bin/hadoop fs -cat /user/hadoop/sss/part-m-00000
看到 sss表內容
1,zs
2,ls

6.從HDFS匯入到mysql中

mysql> delete from sss;

[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop export --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456 --table sss --export-dir hdfs://h1:9000/user/hadoop/sss/part-m-00000

[[email protected] ~]# mysql -usqoop -p
mysql> use test
mysql> select * from sss;
表中的資料 又 回來了


---------------------------------------------------------------
增量抽取 mysql資料 到HDFS中
型別1（欄位增長）
bin/sqoop import --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456 --table sss -m 1 --target-dir /user/hadoop/a --check-column id --incremental append --last-value 3


型別2（時間增長）
mysql> create table s2 (id int,sj timestamp not null default current_timestamp);
mysql> insert into s2 (id)values(123);
mysql> insert into s2 (id)values(321);
mysql> select * from s2;
+------+---------------------+
| id   | sj                  |
+------+---------------------+
|  123 | 2015-11-20 22:34:09 |
|  321 | 2015-11-20 22:34:23 |
+------+---------------------+

bin/sqoop import --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456 --table s2 -m 1 --target-dir /user/hadoop/abc --incremental lastmodified --check-column sj --last-value '2015-11-20 22:34:15'
結果：只有id為321的行更新了
---------------------------------------------------------
配置 sqoop匯入的
[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop export --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456 --table qqq --export-dir hdfs://h1:9000/user/hadoop/qqq.txt --input-fields-terminated-by '\t'

****--input-fields-terminated-by '\t'    
 宣告分隔符*******


----------------------------------------------------------
sqoop eval工具：
sqoop下 使用sql語句對 關係型資料庫進行操作


[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop eval --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456 --query "select * from sss"

[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop eval --connect jdbc:mysql://192.168.18.111:3306/test --username sqoop --password 123456 --query "insert into sss values(3,'ww')"

==============================================================
sqoop hive
mysql 匯入到 hive中

mysql> select * from sss;
+------+------+
| id   | name |
+------+------+
|    1 | zs   |
|    2 | ls   |
|    3 | ww   |
|   10 | haha |
+------+------+


hive> create table tb1(id int,name string)
     row format delimited
     fields terminated by ',';

**** sqoop 預設的分隔符為"," ***********


[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop import --connect jdbc:mysql://192.168.18.111/test --username sqoop --password 123456 --table sss  --hive-import -m 1 --hive-table tb1 --fields-terminated-by ','

hive> select * from tb1;





============================================================
sqoop oracle

1.

[[email protected] ~]$ cp ojdbc6.jar sqoop-1.3.0-cdh3u5/lib/

 測試sqoop 連線oracle
[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop list-tables --connect jdbc:oracle:thin:@192.168.8.222:1521:TEST --username scott --password abc

2.
匯出到HDFS中
[[email protected] sqoop-1.3.0-cdh3u5]$ bin/sqoop import --connect jdbc:oracle:thin:@192.168.8.222:1521:TEST --username SCOTT --password abc --verbose -m 1 --table S1
（表名字 和 使用者名稱要大寫 ）

HDFS 中檢視
[[email protected] hadoop-0.20.2-cdh3u5]$ bin/hadoop fs -cat /user/hadoop/S1/part-m-00000
101,zhangsan
102,lisi


================================================================
1.mysql （t2表  必須id為主鍵）
mysql> select * From t2;
+----+-------+-------+-------+
| id | name1 | name2 | name3 |
+----+-------+-------+-------+
|  1 | aa    | bb    | cc    |
|  2 | aaa   | bbb   | ccc   |
+----+-------+-------+-------+

2.vi .bash_profile
export HBASE_HOME=/home/hadoop/hbase-0.90.6-cdh3u5

3.
 bin/sqoop import --connect jdbc:mysql://192.168.8.101:3306/test --username sqoop --password sqoop  --table t3 --hbase-table qqq  --column-family ccf --hbase-row-key id --hbase-create-table