1. 程式人生 > >異構資料處理(1).Canal入門示例

異構資料處理(1).Canal入門示例

Canal

最近有使用到Alibaba的mysql資料庫binlog的增量訂閱&消費元件“Canal”,抱著試試看的心情開啟簡(jian)約(lou)的官網,下了原始碼,並安裝除錯。1.0.25和1.0.26兩個Canal版本反覆除錯引數發現跑不起來!按照之前遇(bei)到(keng)的經驗,直接丟個issue發到Canal官網,然後該幹嘛幹嘛去了。
終於收到回覆給了一個下載地址,還是canal.deployer-1.0.26-SNAPSHOT.tar.gz,md5sum確認是新的tar檔案,下載重新配置啟動了下,成功啟動,demo程式碼也除錯成功!

資料庫配置

[mysqld]
log_bin=mysql-bin binlog_format='ROW' server_id=1 datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock symbolic-links=0 sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid

mysql canal使用者的許可權配置(可以看到許可權沒有問題)

mysql> select host,user,password from mysql.user  where user like '%canal%';
+-----------+-------+-------------------------------------------+
| host | user | password | +-----------+-------+-------------------------------------------+ | % | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 | | 127.0.0.1 | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 | | localhost | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 | +-----------+-------+-------------------------------------------+
3 rows in set (0.05 sec)

canal 配置

## mysql serverId
canal.instance.mysql.slaveId=0
# position info
canal.instance.master.address=127.0.0.1:3306
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=


# table meta tsdb info
canal.instance.tsdb.enable=true
canal.instance.tsdb.dir=${canal.file.data.dir:../conf}/${canal.instance.destination:}
canal.instance.tsdb.url=jdbc:h2:${canal.instance.tsdb.dir}/h2;CACHE_SIZE=1000;MODE=MYSQL;
#canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
canal.instance.tsdb.dbUsername=canal
canal.instance.tsdb.dbPassword=canal


#canal.instance.standby.address =
#canal.instance.standby.journal.name =
#canal.instance.standby.position = 
#canal.instance.standby.timestamp = 
# username/password
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.defaultDatabaseName=canal_test
canal.instance.connectionCharset=UTF-8
# table regex
canal.instance.filter.regex=.*\\..*
# table black regex
canal.instance.filter.black.regex=