使用tk.mapper引起的一次框架程式碼學習
背景:
專案上線啟動,發生CPU佔滿的問題
定位問題
- 積累線上日誌排查,發現問題並快速定位問題的能力
按照日誌顯示,發現在專案有依賴使用tk.mybatis二方庫中的tk.mybatis.mapper.mapperhelper.MapperInterceptor, 使用的maven依賴如下:
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
<version>3.3.0</version >
<scope>test</scope>
</dependency>
<dependency>
<groupId>tk.mybatis</groupId>
<artifactId>mapper</artifactId>
<version>3.1.2</version>
<scope>test</scope>
</dependency>
在MapperHelper這個類中,使用瞭如下定義:
/**
* 快取skip結果
*/
private final Map<String, Boolean> msIdSkip = new HashMap<String, Boolean>();
其作用是使用map快取需要攔截處理的mapper方法,但是在併發情況下,存在併發問題,導致cpu佔滿的情況。
找到tk.mybatis.mapper.mapperhelper.MapperIntorceptor.intercept()方法,定位到在mapperHelper.isMapperMethod()中 使用HashMap,併發條件下導致HashMap死迴圈。
HashMap死迴圈的問題
解決方案
- 常規解決方案
- 聯絡作者,詢問是否有針對的fix版本,得到作者將在下一新版本中修復這個併發問題的答覆後,檢視作者更新記錄,最新版本與這一版差別很大,在3.2.x之後,作者在tk中去掉了MapperInterceptor,改用tkxxxConfiguration替代這個功能,不能直接升級到最新版,目前沒有能夠直接使用的修復版本。
- 在作者的3.1.2開發分支拉取一個開發分支,修復成threadsafe,修復此問題;
存在的問題是限定了例如3.1.2.x這樣一個版本,修改版本號之後還存在此問題,同時作者開發的新功能,新版本的功能無法使用,作者的修改對此版本來說是未知的,我們要與作者的新功能保持同步是一件很麻煩的事情。
- 最務實的解決方案
- 檢視使用此框架的地方,確認使用的範圍,去掉此框架的使用;
存在的問題是需要找到使用次框架的範圍,程式碼改造並替換框架的功能,並需要全面測試,成本比較大且風險較高。
- 比較優雅的改造方法
- 利用反射修改框架程式碼,學習使用hack精神,遵循開閉原則,儘可能少的修改程式碼和原有配置,儘可能的降低對框架版本的依賴。
Hack是基於開源的程式的基礎,對其程式碼進行增加、刪除或者修改、優化,使之在功能上符合新的需求。
針對此問題,使用hack的方式解決是極好的,是一個很好的切入點。
/**
* <p>An instance of this class is used to fix tk.mybatis.mapper.mapperhelper.
* MapperInterceptor.mapperHelper.msIdSkip(HashMap<String, boolean> threads not threadSafe)
* through reflect to change msIdSkip.HashMap to msIdSkip.ConcurrentHashMap for threadSafe </p>
*
* <p>The purpose of this class is designed to fix the thread unsafe problems through the way of hack,
* framework to modify the source code for as little as possible, and at the same time as little as possible
* to modify the project configuration, better use of the open-closed principle, reduce dependence on tk</p>
*
* <p>if you want to use this function, you need to add an bean instance of this class at your
* spring.xml,{@link <bean class="TkMapperInterceptorHacker" /> }
* and you don't need to dependency tk.mybatis </p>
*
* @author liuyong
* 2017-08-28
*/
此處第一次提交時沒有增加註釋,後來補上了hack開發增加hack的背景、解決什麼問題、原因、方案和使用方法,方便其他人使用和理解,同時增加自己的被知名度。 框架程式碼使用英文註釋,不要出現中英文混用的情況,同時要去掉警告程式碼
class TkMapperInterceptorHacker extends InstantiationAwareBeanPostProcessorAdapter {
自定義BeanPostProcessor,在bean載入過程中對bean攔截進行操作。 類名使用TkMapperInterceptorHacker比TkMapperInterceptorHack更好一些。
private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperInterceptorHacker.class);
定義Logger,final static變數名需要大寫
// interface method is invoked after the bean is instantiated
最開始這裡使用的是中文註釋,框架程式碼中不要出現中英文混合的註釋
@Override
public Object postProcessAfterInitialization(Object bean, String beanName){
String beanClassName = bean.getClass().getName();
if (beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")) {
此處判斷bean為我們需要攔截處理的bean時,有以下幾點收穫:
- 使用指定的beanName,eg:if("mapperInterceptor".equles(beanName)){}; 存在的問題是beanName是多變的,如果註冊該bean的beanName不是mapperIntercptor則會失效。
- 使用if(bean instanceof MapperInterceptor){}; 存在的問題是需要依賴tk.mybatis二方庫,並且需要在該版本中找到MapperInterceptor類,例如這個具體問題,在3.2.x之後沒有MapperInterceptor類了,那麼升級到3.2.x之後的版本則會導致找不到類的錯誤,導致不能使用,使用3.2.x之下的版本同樣不能使用開發的新功能,對版本依賴嚴重。
- 使用if("tk.mybatis.mapper.mapperhelper.MapperInterceptor".equles(bean.getClass().getName())){}; 存在的問題是如果bean對應的類是代理類,則其全類名是tk.mybatis.mapper.mapperhelper.MapperInterceptor.$xxx.$x..x的情況,這樣會導致匹配失敗。
- 使用if (beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")){},存在的問題是會擴大匹配範圍,例如xxx.tk.mybatis.mapper.mapperhelper.MapperInterceptor.* 都會被匹配上。不過此例可以使用
- 使用if (beanClassName.startWith("tk.mybatis.mapper.mapperhelper.MapperInterceptor")){}, 這樣能夠精確匹配。
try {
LOGGER.info("TkMapperInterceptorHacker invoke postProcessAfterInitialization method reload bean:" + beanName);
Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
//使用setAccessible(true)修改private final不可操作的特性
mapperHelperFiled.setAccessible(true);
Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
//此處使用屬性object反射獲取類,操作類屬性。沒有必要現獲取到具體的物件在反射操作類屬性,這樣對具體的類還有依賴。
msIdSkipField.setAccessible(true);
//修改MapperHelper.msIdSkip屬性的HashMap為ConcurrentHashMap屬性
msIdSkipField.set(mapperHelperFiled.get(bean), new ConcurrentHashMap<>());
} catch (Exception e) {
throw new RuntimeException(e);
//此處應該遵循fast-fail原則,快速失敗,但是不能System.exit(),屬於野蠻的做法。利用spring的異常捕獲,丟擲異常,讓容器啟動失敗即可。
}
}
//父類的操作,需要根據父類方法的實現判斷是否需要呼叫,避免父類方法做一些我們未知的操作
return super.postProcessAfterInitialization(bean, beanName);
}
}
補充單元測試
/**
* <P>test class methods keys
* every testCase need follow next rules:
* 1、begin check :self check
* 2、 post : do test method
* 3、check again : check result
* </P>
* Created by yehao on 2017/8/25.
*/
public class TkMapperHackTest {
private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperHackTest.class);
//TestCase1: not the bean we won't to deal, it should return the bean no change
@Test
public void testBeanPostProcessorNotTheBean() {
Object bean = new Object();
Field mapperHelper ;
try {
mapperHelper = bean.getClass().getDeclaredField("mapperHelper");
Assert.assertNull(mapperHelper);
} catch (Exception e) {
Assert.assertEquals(NoSuchFieldException.class, e.getClass());
LOGGER.info("bean self check ok, it should not to change");
}
List<Map<String, Object>> list ;
try {
list = compareTwoClass(new Object(), new TkMapperInterceptorHacker().postProcessAfterInitialization(bean, "mapperInterceptor"));
Assert.assertEquals(0, list.size());
LOGGER.info("bean ckeck again ok, fields no change");
} catch (IllegalAccessException e) {
Assert.fail("bean check again error"+ e);
}
}
/**
* test the bean which we want to deal by post
*/
@Test
public void testBeanPostProcessor() {
MapperInterceptor bean = new MapperInterceptor();
try {
Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
mapperHelperFiled.setAccessible(true);
Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
msIdSkipField.setAccessible(true);
Assert.assertEquals(HashMap.class, msIdSkipField.get(mapperHelperFiled.get(bean)).getClass());
LOGGER.info("bean self ckeck ok, fields class is we want");
} catch (Exception e) {
Assert.fail("bean self check error"+ e);
}
new TkMapperInterceptorHacker().postProcessAfterInitialization(bean, "mapperInterceptor");
try {
Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
mapperHelperFiled.setAccessible(true);
Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
msIdSkipField.setAccessible(true);
Assert.assertEquals(ConcurrentHashMap.class, msIdSkipField.get(mapperHelperFiled.get(bean)).getClass());
LOGGER.info("bean ckeck again ok, fields class is we want after post deal");
} catch (Exception e) {
Assert.fail("bean self check error"+ e);
}
}
}
針對以上的程式碼框架,可以解決CPU飆升的問題,然而此版本存在另一個問題是如果tk.MapperHelper是使用的代理類,怎會出現並沒有修改msIdSkip的類,為了相容代理類,修改TkMapperInterceptorHacker類程式碼如下:
package com.helijia.framework.hack.tk;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.config.InstantiationAwareBeanPostProcessorAdapter;
import org.springframework.util.StringUtils;
import java.lang.reflect.Field;
import java.util.concurrent.ConcurrentHashMap;
/**
* <p>An instance of this class is used to fix tk.mybatis.mapper.mapperhelper.
* MapperInterceptor.mapperHelper.msIdSkip(HashMap<String, boolean> threads not threadSafe)
* through reflect to change msIdSkip.HashMap to msIdSkip.ConcurrentHashMap for threadSafe </p>
* <p>
* <p>The purpose of this class is designed to fix the thread unsafe problems through the way of hack,
* framework to modify the source code for as little as possible, and at the same time as little as possible
* to modify the project configuration, better use of the open-closed principle, reduce dependence on tk</p>
* <p>
* <p>if you want to use this function, you need to add an bean instance of this class at your
* spring.xml,{@link <bean class="TkMapperInterceptorHacker" /> }
* and you don't need to dependency tk.mybatis </p>
* <p>
* <p>This class don't fit for {@code JDKDynamic} proxy class, if class is the agent of JDKDynamic, this class won't make effects</p>
*
* @author liuyong
* 2017-08-28
*/
class TkMapperInterceptorHacker extends InstantiationAwareBeanPostProcessorAdapter {
private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperInterceptorHacker.class);
/**
* interface method is invoked after the bean is instantiated
*
* @param bean
* @param beanName
* @return
*/
@Override
public Object postProcessAfterInitialization(Object bean, String beanName) {
String beanClassName = bean.getClass().getName();
if (!StringUtils.isEmpty(beanClassName) && beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")) {
try {
LOGGER.info("TkMapperInterceptorHacker invoke postProcessAfterInitialization method reload bean:" + beanName);
exceClass(bean, bean.getClass());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
return super.postProcessAfterInitialization(bean, beanName);
}
private void exceClass(Object bean, Class clazz) throws Exception {
if (null == clazz || clazz == Object.class) {
return;
}
boolean isMatch = false;
Field[] fields = clazz.getDeclaredFields();
for (final Field field : fields) {
if ("mapperHelper".equals(field.getName())) {
field.setAccessible(true);
Field msIdSkipField = field.get(bean).getClass().getDeclaredField("msIdSkip");
msIdSkipField.setAccessible(true);
msIdSkipField.set(field.get(bean), new ConcurrentHashMap<>());
isMatch = true;
break;
}
}
if (!isMatch) {
exceClass(bean, clazz.getSuperclass());
}
}
}
github地址