MYSQL+Amoeba讀寫分離
阿新 • • 發佈：2018-12-17
一、部署環境： amoeba:192.168.0.235 masterDB：192.168.0.225 slaveDB：192.168.0.213 以上系統全為centos7 安裝好JDK1.5版本以上的，masterDB和slaveDB配置好主從複製關係。 二、安裝Amoeba 1.解壓即可使用，這裡將Amoeba解壓到/usr/local/amoeba目錄下
wget http://jaist.dl.sourceforge.net/project/amoeba/Amoeba%20for%20mysql/3.x/amoeba-mysql-3.0.5-RC-distribution.zip unzip amoeba-mysql-3.0.5-RC-distribution.zip mv amoeba-mysql-3.0.5-RC /usr/local/amoeba
2.配置Amoeba Amoeba的配置檔案在本環境下位於/usr/local/amoeba/conf目錄下。配置檔案比較多，但是僅僅使用讀寫分離功能，只需配置兩個檔案即可，分別是dbServers.xml和amoeba.xml，如果需要配置ip訪問控制，還需要修改access_list.conf檔案，下面首先介紹dbServers.xml
[[email protected] amoeba]# cat conf/dbServers.xml <?xml version="1.0" encoding="gbk"?> <!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd"> <amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/"> <!-- Each dbServer needs to be configured into a Pool, If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration: add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig such as 'multiPool' dbServer --> <dbServer name="abstractServer" abstractive="true"> <factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory"> <property name="connectionManager">${defaultManager}</property> <property name="sendBufferSize">64</property> <property name="receiveBufferSize">128</property> <!-- mysql port --> <property name="port">3306</property> #設定Amoeba要連線的mysql資料庫的埠，預設是3306 <!-- mysql schema --> <property name="schema">testdb</property> #設定預設的資料庫，當連線amoeba時，操作表必須顯式的指定資料庫名，即採用dbname.tablename的方式，不支援 use dbname指定預設庫，因為操作會排程到各個後端dbserver <!-- mysql user --> <property name="user">root</property> #設定amoeba連線後端資料庫伺服器的賬號和密碼，因此需要在所有後端資料庫上建立該使用者，並授權amoeba伺服器可連線 <property name="password">220316</property> </factoryConfig> <poolConfig class="com.meidusa.toolkit.common.poolable.PoolableObjectPool"> <property name="maxActive">500</property> #最大連線數，預設500 <property name="maxIdle">500</property> #最大空閒連線數 <property name="minIdle">1</property> #最新空閒連線數 <property name="minEvictableIdleTimeMillis">600000</property> <property name="timeBetweenEvictionRunsMillis">600000</property> <property name="testOnBorrow">true</property> <property name="testOnReturn">true</property> <property name="testWhileIdle">true</property> </poolConfig> </dbServer> <dbServer name="writedb" parent="abstractServer"> #設定一個後端可寫的dbServer，這裡定義為writedb，這個名字可以任意命名，後面還會用到 <factoryConfig> <!-- mysql ip --> <property name="ipAddress">192.168.2.204</property> #設定後端可寫dbserver </factoryConfig> </dbServer> <dbServer name="slave" parent="abstractServer"> #設定後端可讀dbserver <factoryConfig> <!-- mysql ip --> <property name="ipAddress">192.168.2.205</property> </factoryConfig> </dbServer> <dbServer name="myslave" virtual="true"> #設定定義一個虛擬的dbserver，實際上相當於一個dbserver組，這裡將可讀的資料庫ip統一放到一個組中，將這個組的名字命名為myslave <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool"> <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA--> <property name="loadbalance">1</property> #選擇排程演算法，1表示複製均衡，2表示權重，3表示HA， 這裡選擇1 <!-- Separated by commas,such as: server1,server2,server1 --> <property name="poolNames">slave</property> #myslave組成員 </poolConfig> </dbServer> </amoeba:dbServers>
配置檔案amoeba.xml
[[email protected] amoeba]# cat conf/amoeba.xml <?xml version="1.0" encoding="gbk"?> <!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd"> <amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/"> <proxy> <!-- service class must implements com.meidusa.amoeba.service.Service --> <service name="Amoeba for Mysql" class="com.meidusa.amoeba.mysql.server.MySQLService"> <!-- port --> <property name="port">8066</property> #設定amoeba監聽的埠，預設是8066 <!-- bind ipAddress --> #下面配置監聽的介面，如果不設定，預設監聽所以的IP <!-- <property name="ipAddress">127.0.0.1</property> --> <property name="connectionFactory"> <bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory"> <property name="sendBufferSize">128</property> <property name="receiveBufferSize">64</property> </bean> </property> <property name="authenticateProvider"> <bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator"> # 提供客戶端連線amoeba時需要使用這裡設定的賬號 (這裡的賬號密碼和amoeba連線後端資料庫伺服器的密碼無關) <property name="user">root</property> <property name="password">123456</property> <property name="filter"> <bean class="com.meidusa.toolkit.net.authenticate.server.IPAccessController"> <property name="ipFile">${amoeba.home}/conf/access_list.conf</property> </bean> </property> </bean> </property> </service> <runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext"> <!-- proxy server client process thread size --> <property name="executeThreadSize">128</property> <!-- per connection cache prepared statement size --> <property name="statementCacheSize">500</property> <!-- default charset --> <property name="serverCharset">utf8</property> <!-- query timeout( default: 60 second , TimeUnit:second) --> <property name="queryTimeout">60</property> </runtime> </proxy> <!-- Each ConnectionManager will start as thread manager responsible for the Connection IO read , Death Detection --> <connectionManagerList> <connectionManager name="defaultManager" class="com.meidusa.toolkit.net.MultiConnectionManagerWrapper"> <property name="subManagerClassName">com.meidusa.toolkit.net.AuthingableConnectionManager</property> </connectionManager> </connectionManagerList> <!-- default using file loader --> <dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader"> <property name="configFile">${amoeba.home}/conf/dbServers.xml</property> </dbServerLoader> <queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter"> <property name="ruleLoader"> <bean class="com.meidusa.amoeba.route.TableRuleFileLoader"> <property name="ruleFile">${amoeba.home}/conf/rule.xml</property> <property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property> </bean> </property> <property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property> <property name="LRUMapSize">1500</property> <property name="defaultPool">writedb</property> #設定amoeba預設的池，這裡設定為writedb <property name="writePool">writedb</property> #這兩個選項預設是登出掉的，需要取消註釋，這裡用來指定前面定義好的倆個讀寫池 <property name="readPool">myslave</property> <property name="needParse">true</property> </queryRouter> </amoeba:configuration>
3.在masterdb上建立資料庫testdb
mysql> create database testdb;
Query OK, 1 row affected (0.08 sec)
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mydb |
| mysql |
| performance_schema |
| test |
| testdb |
+--------------------+
6 rows in set (0.00 sec)
檢視slavedb是否複製成功
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mydb |
| mysql |
| performance_schema |
| test |
| testdb |
+--------------------+
6 rows in set (0.00 sec)
4.分別在masterdb和slavedb上為amoedb授權
mysql> GRANT ALL ON testdb.* TO 'test1'@'192.168.2.203' IDENTIFIED BY '111111';
Query OK, 0 rows affected (0.05 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.02 sec)
5.啟動amoeba
[email protected] amoeba]# /usr/local/amoeba/bin/launcher
Error: JAVA_HOME environment variable is not set.
[[email protected] amoeba]# vim /etc/profile
[[email protected] amoeba]# source /etc/profile
[[email protected] amoeba]# /usr/local/amoeba/bin/launcher
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=16m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=96m; support was removed in 8.0
The stack size specified is too small, Specify at least 228k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
報錯:
Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. 從錯誤文字上看，應該是由於stack size太小，導致JVM啟動失敗，要如何修改呢？ 其實Amoeba已經考慮到這個問題，並將JVM引數配置寫在屬性檔案裡。現在，讓我們通過該屬性檔案修改JVM引數。 修改jvm.properties檔案JVM_OPTIONS引數。
[[email protected] amoeba]# vim /usr/local/amoeba/jvm.properties
改成：JVM_OPTIONS="-server -Xms1024m -Xmx1024m -Xss256k -XX:PermSize=16m -XX:MaxPermSize=96m"
原為：JVM_OPTIONS="-server -Xms256m -Xmx1024m -Xss196k -XX:PermSize=16m -XX:MaxPermSize=96m"
再次啟動
[[email protected] ~]# /usr/local/amoeba/bin/launcher
at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:127)
at org.codehaus.classworlds.Launcher.main(Launcher.java:110)
Caused by: com.meidusa.toolkit.common.bean.util.InitialisationException: default pool required!,defaultPool=writedb invalid
at com.meidusa.amoeba.route.AbstractQueryRouter.init(AbstractQueryRouter.java:469)
at com.meidusa.amoeba.context.ProxyRuntimeContext.initAllInitialisableBeans(ProxyRuntimeContext.java:337)
... 11 more
2018-10-24 18:46:37 [INFO] Project Name=Amoeba-MySQL, PID=1577 , System shutdown ....
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=16m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=96m; support was removed in 8.0
2018-10-24 18:50:19 [INFO] Project Name=Amoeba-MySQL, PID=1602 , starting...
log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml
2018-10-24 18:50:21,668 INFO context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-3.0.4-BETA
log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf
2018-10-24 18:50:22,852 INFO net.ServerableConnectionManager - Server listening on 0.0.0.0/0.0.0.0:8066.
6.測試 遠端登陸mysql客戶端通過指定amoeba配置檔案中指定的使用者名稱、密碼、和埠以及amoeba伺服器ip地址連結mysql資料庫
mysql -h192.168.0.235 -uroot -p -P8066
在testdb中建立表test並插入資料，分別登陸masterdb和slavedb檢視資料是否同步。如果同步，關掉masterdb，就會插入失敗。關掉slavedb，就會檢視失敗。只有同時啟動masterdb和slavedb，才可以正常插入和檢視。 注意：mysql主從同步配置可以檢視博主的其他筆記
其他拓展： Amoeba主配置檔案($AMOEBA_HOME/conf/amoeba.xml)，用來配置Amoeba服務的基本引數，如Amoeba主機地址、埠、認證方式、用於連線的使用者名稱、密碼、執行緒數、超時時間、其他配置檔案的位置等。
資料庫伺服器配置檔案($AMOEBA_HOME/conf/dbServers.xml)，用來儲存和配置Amoeba所代理的資料庫伺服器的資訊，如:主機IP、埠、使用者名稱、密碼等。
切分規則配置檔案($AMOEBA_HOME/conf/rule.xml)，用來配置切分規則。
資料庫函式配置檔案($AMOEBA_HOME/conf/functionMap.xml)，用來配置資料庫函式的處理方法，Amoeba將使用該配置檔案中的方法解析資料庫函式。
切分規則函式配置檔案($AMOEBA_HOME/conf/ruleFunctionMap.xml)，用來配置切分規則中使用的使用者自定義函式的處理方法。
訪問規則配置檔案($AMOEBA_HOME/conf/access_list.conf)，用來授權或禁止某些伺服器IP訪問Amoeba。
日誌規格配置檔案($AMOEBA_HOME/conf/log4j.xml)，用來配置Amoeba輸出日誌的級別和方式。