1. 程式人生 > >使用tk.mapper引起的一次框架程式碼學習

使用tk.mapper引起的一次框架程式碼學習

背景:

專案上線啟動,發生CPU佔滿的問題 image

定位問題

  • 積累線上日誌排查,發現問題並快速定位問題的能力

按照日誌顯示,發現在專案有依賴使用tk.mybatis二方庫中的tk.mybatis.mapper.mapperhelper.MapperInterceptor, 使用的maven依賴如下:

<dependency>
        <groupId>org.mybatis</groupId>
        <artifactId>mybatis</artifactId>
        <version>3.3.0</version
>
<scope>test</scope> </dependency> <dependency> <groupId>tk.mybatis</groupId> <artifactId>mapper</artifactId> <version>3.1.2</version> <scope>test</scope> </dependency>

在MapperHelper這個類中,使用瞭如下定義:

/**
 * 快取skip結果
 */
private final Map<String, Boolean> msIdSkip = new HashMap<String, Boolean>();

其作用是使用map快取需要攔截處理的mapper方法,但是在併發情況下,存在併發問題,導致cpu佔滿的情況。

找到tk.mybatis.mapper.mapperhelper.MapperIntorceptor.intercept()方法,定位到在mapperHelper.isMapperMethod()中 使用HashMap,併發條件下導致HashMap死迴圈。

HashMap死迴圈的問題

解決方案

  •  常規解決方案
  1. 聯絡作者,詢問是否有針對的fix版本,得到作者將在下一新版本中修復這個併發問題的答覆後,檢視作者更新記錄,最新版本與這一版差別很大,在3.2.x之後,作者在tk中去掉了MapperInterceptor,改用tkxxxConfiguration替代這個功能,不能直接升級到最新版,目前沒有能夠直接使用的修復版本。
  2. 在作者的3.1.2開發分支拉取一個開發分支,修復成threadsafe,修復此問題;

存在的問題是限定了例如3.1.2.x這樣一個版本,修改版本號之後還存在此問題,同時作者開發的新功能,新版本的功能無法使用,作者的修改對此版本來說是未知的,我們要與作者的新功能保持同步是一件很麻煩的事情。

  •  最務實的解決方案
  1. 檢視使用此框架的地方,確認使用的範圍,去掉此框架的使用;

存在的問題是需要找到使用次框架的範圍,程式碼改造並替換框架的功能,並需要全面測試,成本比較大且風險較高。

  •  比較優雅的改造方法
  1. 利用反射修改框架程式碼,學習使用hack精神,遵循開閉原則,儘可能少的修改程式碼和原有配置,儘可能的降低對框架版本的依賴。

Hack是基於開源的程式的基礎,對其程式碼進行增加、刪除或者修改、優化,使之在功能上符合新的需求。

針對此問題,使用hack的方式解決是極好的,是一個很好的切入點。

/**
 * <p>An instance of this class is used to fix tk.mybatis.mapper.mapperhelper.
 *  MapperInterceptor.mapperHelper.msIdSkip(HashMap<String, boolean> threads not threadSafe)
 *  through reflect to change msIdSkip.HashMap to msIdSkip.ConcurrentHashMap for threadSafe </p>
 *
 *  <p>The purpose of this class is designed to fix the thread unsafe problems through the way of hack,
 *  framework to modify the source code for as little as possible, and at the same time as little as possible
 *  to modify the project configuration, better use of the open-closed principle, reduce dependence on tk</p>
 *
 *  <p>if you want to use this function, you need to add an bean instance of this class at your
 *  spring.xml,{@link <bean class="TkMapperInterceptorHacker" /> }
 *  and you don't need to dependency tk.mybatis </p>
 *
 * @author liuyong
 * 2017-08-28
 */

此處第一次提交時沒有增加註釋,後來補上了hack開發增加hack的背景、解決什麼問題、原因、方案和使用方法,方便其他人使用和理解,同時增加自己的被知名度。 框架程式碼使用英文註釋,不要出現中英文混用的情況,同時要去掉警告程式碼

class TkMapperInterceptorHacker extends InstantiationAwareBeanPostProcessorAdapter {

自定義BeanPostProcessor,在bean載入過程中對bean攔截進行操作。 類名使用TkMapperInterceptorHacker比TkMapperInterceptorHack更好一些。

private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperInterceptorHacker.class);

定義Logger,final static變數名需要大寫

// interface method is invoked after the bean is instantiated

最開始這裡使用的是中文註釋,框架程式碼中不要出現中英文混合的註釋

@Override
    public Object postProcessAfterInitialization(Object bean, String beanName){
        String beanClassName = bean.getClass().getName();
        if (beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")) {

此處判斷bean為我們需要攔截處理的bean時,有以下幾點收穫:

  1. 使用指定的beanName,eg:if("mapperInterceptor".equles(beanName)){}; 存在的問題是beanName是多變的,如果註冊該bean的beanName不是mapperIntercptor則會失效。
  2. 使用if(bean instanceof MapperInterceptor){}; 存在的問題是需要依賴tk.mybatis二方庫,並且需要在該版本中找到MapperInterceptor類,例如這個具體問題,在3.2.x之後沒有MapperInterceptor類了,那麼升級到3.2.x之後的版本則會導致找不到類的錯誤,導致不能使用,使用3.2.x之下的版本同樣不能使用開發的新功能,對版本依賴嚴重。
  3. 使用if("tk.mybatis.mapper.mapperhelper.MapperInterceptor".equles(bean.getClass().getName())){}; 存在的問題是如果bean對應的類是代理類,則其全類名是tk.mybatis.mapper.mapperhelper.MapperInterceptor.$xxx.$x..x的情況,這樣會導致匹配失敗。
  4. 使用if (beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")){},存在的問題是會擴大匹配範圍,例如xxx.tk.mybatis.mapper.mapperhelper.MapperInterceptor.* 都會被匹配上。不過此例可以使用
  5. 使用if (beanClassName.startWith("tk.mybatis.mapper.mapperhelper.MapperInterceptor")){}, 這樣能夠精確匹配。
try {
                LOGGER.info("TkMapperInterceptorHacker invoke postProcessAfterInitialization method reload bean:" + beanName);
                Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
                //使用setAccessible(true)修改private final不可操作的特性
                mapperHelperFiled.setAccessible(true);
                Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
                //此處使用屬性object反射獲取類,操作類屬性。沒有必要現獲取到具體的物件在反射操作類屬性,這樣對具體的類還有依賴。
                msIdSkipField.setAccessible(true);
                //修改MapperHelper.msIdSkip屬性的HashMap為ConcurrentHashMap屬性
                msIdSkipField.set(mapperHelperFiled.get(bean), new ConcurrentHashMap<>());
            } catch (Exception e) {
                throw new RuntimeException(e);
                //此處應該遵循fast-fail原則,快速失敗,但是不能System.exit(),屬於野蠻的做法。利用spring的異常捕獲,丟擲異常,讓容器啟動失敗即可。
            }
        }
        //父類的操作,需要根據父類方法的實現判斷是否需要呼叫,避免父類方法做一些我們未知的操作
        return super.postProcessAfterInitialization(bean, beanName);
    }
}

補充單元測試

/**
 * <P>test class methods keys
 *  every testCase need follow next rules:
 *  1、begin check :self check
 *  2、 post   : do test method
 *  3、check again : check result
 * </P>
 * Created by yehao on 2017/8/25.
 */
public class TkMapperHackTest {
    private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperHackTest.class);

    //TestCase1: not the bean we won't to deal, it should return the bean no change
    @Test
    public void testBeanPostProcessorNotTheBean() {
        Object bean = new Object();
        Field mapperHelper ;
        try {
            mapperHelper = bean.getClass().getDeclaredField("mapperHelper");
            Assert.assertNull(mapperHelper);
        } catch (Exception e) {
            Assert.assertEquals(NoSuchFieldException.class, e.getClass());
            LOGGER.info("bean self check ok, it should not to change");
        }
        List<Map<String, Object>> list ;
        try {
            list = compareTwoClass(new Object(),  new TkMapperInterceptorHacker().postProcessAfterInitialization(bean, "mapperInterceptor"));
            Assert.assertEquals(0, list.size());
            LOGGER.info("bean ckeck again ok, fields no change");
        } catch (IllegalAccessException e) {
            Assert.fail("bean check again error"+ e);
        }

    }


    /**
     * test the bean which we want to deal by post
     */
    @Test
    public void testBeanPostProcessor() {
        MapperInterceptor bean = new MapperInterceptor();
        try {
            Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
            mapperHelperFiled.setAccessible(true);
            Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
            msIdSkipField.setAccessible(true);
            Assert.assertEquals(HashMap.class, msIdSkipField.get(mapperHelperFiled.get(bean)).getClass());
            LOGGER.info("bean self ckeck ok, fields class is we want");
        } catch (Exception e) {
            Assert.fail("bean self check error"+ e);
        }

        new TkMapperInterceptorHacker().postProcessAfterInitialization(bean, "mapperInterceptor");
        try {
            Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
            mapperHelperFiled.setAccessible(true);
            Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
            msIdSkipField.setAccessible(true);
            Assert.assertEquals(ConcurrentHashMap.class, msIdSkipField.get(mapperHelperFiled.get(bean)).getClass());
            LOGGER.info("bean ckeck again ok, fields class is we want after post deal");
        } catch (Exception e) {
            Assert.fail("bean self check error"+ e);
        }

    }
}

針對以上的程式碼框架,可以解決CPU飆升的問題,然而此版本存在另一個問題是如果tk.MapperHelper是使用的代理類,怎會出現並沒有修改msIdSkip的類,為了相容代理類,修改TkMapperInterceptorHacker類程式碼如下:

package com.helijia.framework.hack.tk;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.config.InstantiationAwareBeanPostProcessorAdapter;
import org.springframework.util.StringUtils;

import java.lang.reflect.Field;
import java.util.concurrent.ConcurrentHashMap;

/**
 * <p>An instance of this class is used to fix tk.mybatis.mapper.mapperhelper.
 * MapperInterceptor.mapperHelper.msIdSkip(HashMap<String, boolean> threads not threadSafe)
 * through reflect to change msIdSkip.HashMap to msIdSkip.ConcurrentHashMap for threadSafe </p>
 * <p>
 * <p>The purpose of this class is designed to fix the thread unsafe problems through the way of hack,
 * framework to modify the source code for as little as possible, and at the same time as little as possible
 * to modify the project configuration, better use of the open-closed principle, reduce dependence on tk</p>
 * <p>
 * <p>if you want to use this function, you need to add an bean instance of this class at your
 * spring.xml,{@link <bean class="TkMapperInterceptorHacker" /> }
 * and you don't need to dependency tk.mybatis </p>
 * <p>
 * <p>This class don't fit for {@code JDKDynamic} proxy class, if class is the agent of JDKDynamic, this class won't make effects</p>
 *
 * @author liuyong
 *         2017-08-28
 */
class TkMapperInterceptorHacker extends InstantiationAwareBeanPostProcessorAdapter {

    private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperInterceptorHacker.class);


    /**
     * interface method is invoked after the bean is instantiated
     *
     * @param bean
     * @param beanName
     * @return
     */
    @Override
    public Object postProcessAfterInitialization(Object bean, String beanName) {
        String beanClassName = bean.getClass().getName();
        if (!StringUtils.isEmpty(beanClassName) && beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")) {
            try {
                LOGGER.info("TkMapperInterceptorHacker invoke postProcessAfterInitialization method reload bean:" + beanName);


                exceClass(bean, bean.getClass());

            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
        return super.postProcessAfterInitialization(bean, beanName);
    }


    private void exceClass(Object bean, Class clazz) throws Exception {
        if (null == clazz || clazz == Object.class) {
            return;
        }
        boolean isMatch = false;
        Field[] fields = clazz.getDeclaredFields();
        for (final Field field : fields) {
            if ("mapperHelper".equals(field.getName())) {
                field.setAccessible(true);
                Field msIdSkipField = field.get(bean).getClass().getDeclaredField("msIdSkip");
                msIdSkipField.setAccessible(true);
                msIdSkipField.set(field.get(bean), new ConcurrentHashMap<>());
                isMatch = true;
                break;
            }
        }
        if (!isMatch) {
            exceClass(bean, clazz.getSuperclass());
        }
    }
}
github地址