1. 程式人生 > 實用技巧 >使用zookeeper管理遠端Mycat配置檔案、Mycat監控、Mycat資料遷移(擴容)

使用zookeeper管理遠端Mycat配置檔案、Mycat監控、Mycat資料遷移(擴容)

一、使用zookeeper管理遠端Mycat配置檔案

環境準備:

虛擬機器192.168.152.130:

zookeeper,具體參考前面文章搭建dubbo+zookeeper+dubboadmin分散式服務框架(windows平臺下)

虛擬機器192.168.152.128:

安裝好Mycat,具體參考前面文章資料庫分庫分表中介軟體mycat的安裝和mycat配置詳解

本機:

搭建好zookeeper的客戶端工具ZooInspector ,具體參考這個人的文章Zookeeper資料檢視工具ZooInspector簡介

1.啟動ZK

./zkServer.sh start

2. 修改/usr/local/mycat/conf/myid.properties

vim /usr/local/mycat/conf/myid.properties
loadZk=true
zkURL=192.168.152.130:2181
clusterId=mycat-cluster-1
myid=mycat_fz_01
clusterNodes=mycat_fz_01,mycat_fz_02,mycat_fz_04
#server  booster  ;   booster install on db same server,will reset all minCon to 1
type=server
boosterDataHosts=dn2,dn3

myid.properties配置說明:

loadZk:預設值false。代表mycat叢集是否使用ZK,true表示使用

zkURLzk叢集的地址

clusterIdmycat叢集名字

myid:當前的mycat伺服器名稱

clusterNodes:把所有叢集中的所有mycat伺服器羅列進行以逗號隔開,比如:clusterNodes=mycat_fz_01,mycat_fz_02,mycat_fz_03

3.使用Mycat指令碼初始化mycat在ZK中的節點資料

/usr/local/mycat/bin/init_zk_data.sh

4. 啟動mycat

/usr/local/mycat/bin/mycat start

5. 連線mycat,檢視當前資料庫中表的情況

mysql -uroot -pdigdeep -P8066 -h192.168.152.128
use TESTDB;
show tables;

6. 使用zookeeper的客戶端工具ZooInspector連線zookeeper修改mycat配置檔案

6.1進入目錄E:\software\devtools\ZK\ZooInspector\build,執行zookeeper-dev-ZooInspector.jar;

 java -jar zookeeper-dev-ZooInspector.jar

連線遠端zookeeper

連線成功以後可以看到mycat的樹節點和mycat的安裝後的conf目錄下的schema.xml和rule.xml檔案是對應的

6.2 在zookeeper管理工具裡面動態修改schema配置檔案新增一張邏輯表travelrecord2

儲存修改,檢視mycat的schema.xml配置檔案,可以看到修改已經同步過來了

重啟mycat,可以看到TESTDB裡面有新配置的travelrecord2表

/usr/local/mycat/bin/mycat restart

二、Mycat監控

1. 下載Mycat-web上傳到虛擬機器中

http://dl.mycat.io/mycat-web-1.0/

2. 解壓Mycat-web

tar -zxvf Mycat-web-1.0-SNAPSHOT-20170102153329-linux.tar.gz

3 修改mycat-web專案下/WEB-INF/classes/mycat.properties中zookeeper屬性

vim /software/mycat-web/mycat-web/WEB-INF/classes/mycat.properties
#Mon Jan 16 15:37:36 CST 2012
show.period=3000000
zookeeper=192.168.152.130:2181

mycat_warn_mail=[{"cc"\:"[email protected]","index"\:1,"mangerPort"\:"465","smtpHost"\:"smtp.139.com","smtpPassword"\:"123456","smtpProtocol"\:"smtp","smtpUser"\:"[email protected]","to"\:"[email protected]"}]
##sql\u4E0A\u7EBF\u76F8\u5173\u914D\u7F6E
sqlonline.server=192.168.80.128
sqlonline.user=root
sqlonline.passwd=123456

4. 啟動mycat-web

/software/mycat-web/start.sh

5. 訪問mycat-web

訪問地址:http://192.168.152.128:8082/mycat/

6. 在Mycat管理介面手動初始化Mycat配置

6.1 Mycat-配置管理。增加mycat伺服器配置

儲存,新增成功

6.2 mycat-VM管理。配置JMX服務項

儲存新增成功

6.3 MySQL配置管理

分別儲存,新增成功

初始化完前面的三步,就可以對Mycat進行監控了

三、Mycat資料遷移(擴容)

以employee表作資料遷移

1. 準備工作:

1)修改employee表的分片規則為一致性hash,vim schema.xml

<table name="employee" dataNode="dn1,dn2" rule="sharding-by-murmur" primaryKey="ID"/>

2)修改分散式全域性唯一id方式為本地檔案方式,vim server.xml

<property name="sequnceHandlerType">0</property>

3)關掉ZK配置,vim myid.properties

loadZk=false
zkURL=192.168.152.130:2181
clusterId=mycat-cluster-1
myid=mycat_fz_01
clusterNodes=mycat_fz_01,mycat_fz_02,mycat_fz_04
#server  booster  ;   booster install on db same server,will reset all minCon to 1
type=server
boosterDataHosts=dn2,dn3

4) 重啟mycat

../bin/mycat restart

5) 連線mycat,並建立employee表,並插入資料

mysql -uroot -pdigdeep -P8066 -h192.168.152.128
use TESTDB;
create table employee(
 id bigint(20) not null primary key auto_increment,
 name varchar(20),
 sharding_id bigint(20),
 company_id bigint(20)
);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack001', 10010, 1);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack002', 10000, 2);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack003', 10010, 3);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack004', 10010, 4);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack005', 10010, 5);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack006', 10000, 1);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack007', 10000, 1);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack008', 10000, 2);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack009', 10010, 2);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack010', 10010, 1);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack011', 10010, 3);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack012', 10010, 4);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack013', 10000, 5);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack014', 10000, 5);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack015', 10010, 2);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack016', 10000, 2);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack017', 10010, 3);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack018', 10000, 3);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack019', 10010, 1);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack021', 10000, 1);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack022', 10010, 2);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack023', 10000, 5);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack024', 10000, 4);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack025', 10000, 5);
insert into employee(id, name, sharding_id, company_id) values(next value for MYCATSEQ_GLOBAL, 'Jack026', 10010, 3);

可以看到插入了25條資料到employee裡面

然後在主庫(192.168.152.130)的兩個分片dn1(db1),dn2(db2)上檢視employee的資料

可以看到dn1上有15條資料,dn2上有10條資料

2. 開始做資料遷移

當前employee表的資料分佈在dn1,dn2,遷移後要分佈在dn1,dn2,dn3

1)生成一個新的schema檔案,檔名newSchema.xml,同時修改dataNode="dn1,dn2,dn3"(新的分片定義)

cp schema.xml newSchema.xml
vim newSchema.xml

以前的schema.xml檔案:

新的newSchema.xml:

2)生成一個新的rule檔案,檔名newRule.xml,同時修改分片數為3(新的分片規則)

cp rule.xml newRule.xml
vim newRule.xml

以前的rule.xml:

新的newRule.xml:

3)修改/usr/local/mycat/conf/migrateTables.properties檔案。寫明要遷移的schema和表,多張表用逗號隔開

vim migrateTables.properties
#schema1=tb1,tb2,...
#schema2=all(寫all或者不寫將對此schema下拆分節點變化的拆分表全部進行重新路由)
#...

#sample
#TESTDB=travelrecord,company,goods
TESTDB=employee

4)停止mycat(也可以不停止mycat)

5)執行dataMigrate.sh開始遷移(擴容)。

[root@centos1 bin]# ./dataMigrate.sh 

遇到的錯誤1:

開始執行./dataMigrate.sh 的時候報沒有適合的驅動

No suitable driver found for jdbc:mysql://192.168.152.130:3306/db1 java.sql.SQLException: No suitable driver found for jdbc:mysql://192.168.152.130:3306/db1

解決方法1:

下載jar包mysql-connector-java-5.1.6.jar,然後上傳到/usr/local/mycat/lib

檢視主庫(192.168.152.130)三個分片dn1(db1),dn2(db2),dn3(db3)上employee表的資料

可以看到資料遷移成功

6)修改newRule.xml成為rule.xml,修改newSchema.xml為schema.xml。重啟mycat

./bin/mycat restart

連線mycat,檢視employee表的資料

mysql -uroot -pdigdeep -P8066 -h192.168.152.128