sharding-jdbc結合mybatis實現分庫分表功能
源自:https://www.cnblogs.com/zwt1990/p/6762135.html
最近忙於專案已經好久幾天沒寫部落格了,前2篇文章我給大家介紹了搭建基礎springMvc+mybatis的maven工程,這個簡單框架已經可以對付一般的小型專案。但是我們實際專案中會碰到很多複雜的場景,比如資料量很大的情況下如何保證效能。今天我就給大家介紹資料庫分庫分表的優化,本文介紹mybatis結合噹噹網的sharding-jdbc分庫分表技術(原理這裡不做介紹)
首先在pom檔案中引入需要的依賴
<dependency> <groupId>com.dangdang</groupId> <artifactId>sharding-jdbc-core</artifactId> <version>1.4.2</version> </dependency> <dependency> <groupId>com.dangdang</groupId> <artifactId>sharding-jdbc-config-spring</artifactId> <version>1.4.0</version> </dependency>
二、新建一個sharding-jdbc.xml檔案,實現分庫分表的配置
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"xmlns:tx="http://www.springframework.org/schema/tx" xmlns:rdb="http://www.dangdang.com/schema/ddframe/rdb" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.dangdang.com/schema/ddframe/rdb http://www.dangdang.com/schema/ddframe/rdb/rdb.xsd"> <rdb:strategy id="tableShardingStrategy" sharding-columns="user_id" algorithm-class="com.meiren.member.common.sharding.MemberSingleKeyTableShardingAlgorithm"/> <rdb:data-source id="shardingDataSource"> <rdb:sharding-rule data-sources="dataSource"> <rdb:table-rules> <rdb:table-rule logic-table="member_index" actual-tables="member_index_tbl_${[0,1,2,3,4,5,6,7,8,9]}${0..9}" table-strategy="tableShardingStrategy"/> <rdb:table-rule logic-table="member_details" actual-tables="member_details_tbl_${[0,1,2,3,4,5,6,7,8,9]}${0..9}" table-strategy="tableShardingStrategy"/> </rdb:table-rules> </rdb:sharding-rule> </rdb:data-source> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="shardingDataSource" /> </bean> </beans>
這裡我簡單介紹下一些屬性的含義,
<rdb:strategy id="tableShardingStrategy" sharding-columns="user_id" algorithm-class="com.meiren.member.common.sharding.MemberSingleKeyTableShardingAlgorithm"/> 配置分表規則器 sharding-columns:分表規 則
依賴的名(根據user_id取模分表),algorithm-class:分表規則的實現類
<rdb:sharding-rule data-sources="dataSource"> 這裡填寫關聯資料來源(多個數據源用逗號隔開),
<rdb:table-rule logic-table="member_index" actual-tables="member_index_tbl_${[0,1,2,3,4,5,6,7,8,9]}${0..9}" table-strategy="tableShardingStrategy"/> logic-table:邏輯表名(mybatis中代替的表名)actual-tables:
資料庫實際的表名,這裡支援inline表示式,比如:member_index_tbl_${0..2}會解析成member_index_tbl_0,member_index_tbl_1,member_index_tbl_2;member_index_tbl_${[a,b,c]}會被解析成
member_index_tbl_a,member_index_tbl_b和member_index_tbl_c,兩種表示式一起使用的時候,會採取笛卡爾積的方式:member_index_tbl_${[a,b]}${0..2}解析為member_index_tbl_a0,member_index_tbl_a1 member_index_tbl_a2,member_index_tbl_b0,member_index_tbl_b1,member_index_tbl_b2;table-strategy:前面定義的分表規則器;
三、配置好改檔案後,需要修改之前我們的spring-dataSource的幾個地方,把sqlSessionFactory和transactionManager原來關聯的dataSource統一修改為shardingDataSource(這一步作用就是把資料來源全部託管給sharding去管理)
四、實現分表(分庫)邏輯,我們的分表邏輯類需要實現SingleKeyTableShardingAlgorithm介面的三個方法doBetweenSharding、doEqualSharding、doInSharding
/** * 分表邏輯 * @author zhangwentao * */ public class MemberSingleKeyTableShardingAlgorithm implements SingleKeyTableShardingAlgorithm<Long> { /** * sql between 規則 */ public Collection<String> doBetweenSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) { Collection<String> result = new LinkedHashSet<String>(tableNames.size()); Range<Long> range = (Range<Long>) shardingValue.getValueRange(); for (long i = range.lowerEndpoint(); i <= range.upperEndpoint(); i++) { Long modValue = i % 100; String modStr = modValue < 10 ? "0" + modValue : modValue.toString(); for (String each : tableNames) { if (each.endsWith(modStr)) { result.add(each); } } } return result; } /** * sql == 規則 */ public String doEqualSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) { Long modValue = shardingValue.getValue() % 100; String modStr = modValue < 10 ? "0" + modValue : modValue.toString(); for (String each : tableNames) { if (each.endsWith(modStr)) { return each; } } throw new IllegalArgumentException(); } /** * sql in 規則 */ public Collection<String> doInSharding(Collection<String> tableNames, ShardingValue<Long> shardingValue) { Collection<String> result = new LinkedHashSet<String>(tableNames.size()); for (long value : shardingValue.getValues()) { Long modValue = value % 100; String modStr = modValue < 10 ? "0" + modValue : modValue.toString(); for (String tableName : tableNames) { if (tableName.endsWith(modStr)) { result.add(tableName); } } } return result; } }
五、以上四步,我們就完成了sharding-jdbc的搭建,我們可以寫一個測試demo來檢查我們的成果
<select id="getDetailsById" resultType="com.meiren.member.dataobject.MemberDetailsDO" parameterType="java.lang.Long"> select user_id userId ,qq,email from member_details where user_id =#{userId} limit 1 </select>
123456789101112131415161718192021 | private static final String SERVICE_PROVIDER_XML = "/spring/member-service.xml" ; private static final String BEAN_NAME = "idcacheService" ; private ClassPathXmlApplicationContext context = null ; IdcacheServiceImpl bean = null ; IdcacheDao idcacheDao; @Before public void before() { context= new ClassPathXmlApplicationContext( new String[] {SERVICE_PROVIDER_XML}); idcacheDao=context.getBean( "IdcacheDao" , IdcacheDao. class ); } @Test public void getAllCreditActionTest() { // int id = bean.insertIdcache(); Long s=100l; MemberDetailsDO memberDetailsDO=idcacheDao.getDetailsById(s); System.out.println( "QQ---------------------" +memberDetailsDO.getQq()); } |
列印sql語句,輸出結果:QQ-------------------------------------100,證明成功!
注意點:這次搭建過程中,我有碰到一個小坑,就是執行的時候會報錯:,官方文件是有解決方案:引入 <context:property-placeholder location="classpath:/member_service.properties" ignore-unresolvable="true" /> ,引入這行程式碼的時候,·必須要要把這邊管理配配置檔案的bean刪除,換句話說,即Spring容器僅允許最多定義一個PropertyPlaceholderConfigurer(或<context:property-placeholder/>),其餘的會被Spring忽略掉(當時搞了半天啊)