Cannal單節點部署
官網:https://codechina.csdn.net/mirrors/alibaba/canal?utm_source=csdn_github_accelerator
下載地址:https://github.com/alibaba/canal/releases
參考部署地址:https://blog.csdn.net/qq_29116427/article/details/106498040
文件:https://github.com/alibaba/canal/wiki/QuickStart
Cannal單節點部署
Mysql配置
Mysql開啟binlog寫入功能,my.cnf 中配置如下:
[mysqld] log-bin=mysql-bin # 開啟 binlog binlog-format=ROW # 選擇 ROW 模式 server_id=1 # 配置 MySQL replaction 需要定義,不要和 canal 的 slaveId 重複 max_binlog_size=500m #每個binlog日誌檔案大小
建立mysql使用者,並且賦權:
CREATE USER canal IDENTIFIED BY 'canal'; GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%'; -- GRANT ALL PRIVILEGES ON *.* TO 'canal'@'%' ; FLUSH PRIVILEGES;
重啟mysql,檢視binlog配置是否生效,如下表示生效:
mysql> show variables like 'binlog_format'; mysql> show variables like 'log_bin'; mysql> show master status;
Canal配置
下載canal.deployer-1.1.5.tar.gz 包
下載地址:https://github.com/alibaba/canal/releases
由於cannal解壓,無根目錄,故新建cannal_deployer目錄,
Tar -zvxf ./ canal.deployer-1.1.5.tar.gz -C ../softwore/ cannal_deployer
進入cannal_deployer 目錄:
Cd/root/software/cannal_deployer
修改canal配置檔案
vi ./conf/example/instance.properties
修改內容如下:
# position info canal.instance.master.address=10.18.35.155:3309 # username/password canal.instance.dbUsername=root canal.instance.dbPassword=123456 # table regex *\\..*表示監聽所有資料庫 canal.instance.filter.regex=.*\\..* # mq config 表示生產資料到kafka時,建立的topic的名稱 canal.mq.topic=canal_topic 修改canal.properties 檔案 vi ./conf/canal.properties 修改內容如下: #預設為TCP,也就是你通過官方的example可以在終端檢視資料,我們修改為kafka # tcp, kafka, rocketMQ, rabbitMQ canal.serverMode = kafka canal.destinations = example #kafka地址配置 kafka.bootstrap.servers = 10.18.35.155:9092,10.18.35.156:9092,10.18.35.157:9092 # 針對庫名或者表名傳送動態topic #canal.mq.dynamicTopic=mytest,.*,mytest.user,mytest\\..*,.*\\..*
Cannal配置檔案說明:https://blog.csdn.net/weixin_35852328/article/details/87600871
啟動cannal-server
./bin/startup.sh
檢視日誌:
tail ./logs/example/example.log
cannal測試:
檢視當前kafka中topic
./kafka-topics.sh -zookeeper 192.168.0.20:2181 -list
查詢canal建立的topic的消費情況
./kafka-console-consumer.sh --bootstrap-server 192.168.0.20:9092 --topic canal_topic --from-beginning