異構資料處理(1).Canal入門示例
阿新 • • 發佈:2019-02-18
Canal
最近有使用到Alibaba的mysql資料庫binlog的增量訂閱&消費元件“Canal”,抱著試試看的心情開啟簡(jian)約(lou)的官網,下了原始碼,並安裝除錯。1.0.25和1.0.26兩個Canal版本反覆除錯引數發現跑不起來!按照之前遇(bei)到(keng)的經驗,直接丟個issue發到Canal官網,然後該幹嘛幹嘛去了。
終於收到回覆給了一個下載地址,還是canal.deployer-1.0.26-SNAPSHOT.tar.gz,md5sum確認是新的tar檔案,下載重新配置啟動了下,成功啟動,demo程式碼也除錯成功!
資料庫配置
[mysqld]
log_bin=mysql-bin
binlog_format='ROW'
server_id=1
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
mysql canal使用者的許可權配置(可以看到許可權沒有問題)
mysql> select host,user,password from mysql.user where user like '%canal%';
+-----------+-------+-------------------------------------------+
| host | user | password |
+-----------+-------+-------------------------------------------+
| % | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 |
| 127.0.0.1 | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 |
| localhost | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 |
+-----------+-------+-------------------------------------------+
3 rows in set (0.05 sec)
canal 配置
## mysql serverId
canal.instance.mysql.slaveId=0
# position info
canal.instance.master.address=127.0.0.1:3306
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=
# table meta tsdb info
canal.instance.tsdb.enable=true
canal.instance.tsdb.dir=${canal.file.data.dir:../conf}/${canal.instance.destination:}
canal.instance.tsdb.url=jdbc:h2:${canal.instance.tsdb.dir}/h2;CACHE_SIZE=1000;MODE=MYSQL;
#canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
canal.instance.tsdb.dbUsername=canal
canal.instance.tsdb.dbPassword=canal
#canal.instance.standby.address =
#canal.instance.standby.journal.name =
#canal.instance.standby.position =
#canal.instance.standby.timestamp =
# username/password
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.defaultDatabaseName=canal_test
canal.instance.connectionCharset=UTF-8
# table regex
canal.instance.filter.regex=.*\\..*
# table black regex
canal.instance.filter.black.regex=