多線程處理慢sql查詢以及List（Array）的拆分

系統數據量不大，但是訪問速度特別慢，使用多線程優化一下！！！

優化結果：訪問時間縮短了十幾秒 25s --> 8s

一、List的拆分：Iterables.partition

註意: 引入的包為google名下的

　　 Iterables.partitions(List list , int size); // list表示需要被拆分的集合，size表示拆分後的集合的元素個數

　　 例如：size=3,表示會拆分為2個集合，第一個集合中的元素為{1,2,3},第二個集合中的元素為{4,5}

　　　　　 size=2，表示會拆分為3個集合，分別為{1,2} {3,4} {5} 以此類推

ps: 集合的合並：

List list = new ArrayList();

List list1 = new ArrayList();

list.addAll(list1);

import com.google.common.collect.Iterables;  

List<String> strList = new ArrayList<String>();
strList.add(1);
strList.add(2);
strList.add(3);
strList.add(4);
strList.add(5);
Iterable<List<String>>
 
 splitStringlList = Iterables.partition(strList, 3); //3表示拆分後的集合元素的數量

二、Array的拆分：Arrays.asList(ArrayObj)

先將數組轉換為List,然後再按照第一方法去拆分。

String[] validRiderArray = [1,2,3,4,5];
ArrayList<String> validArrayList = new ArrayList<String>(Arrays.asList(validRiderArray));

三、多線程處理類：

package com.chinalife.proposal.common.util;

 
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import org.springframework.stereotype.Component;

import com.google.common.util.concurrent.ThreadFactoryBuilder;
/**
 * 固定線程個數的線程池工具類
 * */
 
public class SimpleFixedThreadPoolUtil {
    /** 默認線程個數 */
    private static final int DEFAULT_THREAD = Runtime.getRuntime().availableProcessors();//根據系統內核自動獲取
    /** 線程池個數: 默認為： Runtime.getRuntime().availableProcessors() * 2 */
    public static int THREADPOOL_NUM = DEFAULT_THREAD * 2;
    private static ExecutorService executor = Executors.newFixedThreadPool(THREADPOOL_NUM,
            new ThreadFactoryBuilder().setNameFormat("SimpleFixedThreadPool- %d: ").build());

    /**
     * 提交任務,任務執行後無返回值
     * 
     * @param task
     */
    public static void addTask(Runnable task) {
        executor.execute(task);
    }

    /**
     * 提交具有返回值的任務
     * @param task
     * @return
     */
    public static <T> Future<T> addTask(Callable<T> task) {
        Future<T> future = executor.submit(task);
        return future;
    }

    /**
     * 關閉線程池
     */
    /*public void shutdown() {
        executor.shutdown();
    }*/
}

四、多線程處理慢查詢使用示例：

JunitTest:

/**
     * 多線程處理慢sql查詢示例
     * @throws Exception
     */
    @Test
    public void testAddTaskCallables() throws Exception {
        String role = "Agent";
        String nodeid = "2";
        String subnodeid = "3";
        long time = System.currentTimeMillis();
        List<OnlProposalRolePrdRelation> rolePrdlist = webProposalService.getAvailableProductListByRoleAndID(role,nodeid, subnodeid);
        Iterable<List<OnlProposalRolePrdRelation>> result = Iterables.partition(rolePrdlist, 30);
        Iterator<List<OnlProposalRolePrdRelation>> iterator = result.iterator();
        List<Future<List<OnlProposalRolePrdRelation>>> futures = new ArrayList<>();
        while (iterator.hasNext()) {
            List<OnlProposalRolePrdRelation> listTemp = iterator.next();
            Future<List<OnlProposalRolePrdRelation>> future = SimpleFixedThreadPoolUtil
                    .addTask((Callable<List<OnlProposalRolePrdRelation>>) () -> {
                        List<OnlProposalRolePrdRelation> relations = null;
                        try {
                            relations = webProposalService.filterByCriteria(listTemp, 20, "1", "1", "0");
                            return relations;
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                        return relations;
                    });
            futures.add(future);
        }
        List<OnlProposalRolePrdRelation> proposalResultList = new ArrayList<OnlProposalRolePrdRelation>();
        for (Future<List<OnlProposalRolePrdRelation>> future : futures) {
            while (!future.isDone()) {
                Thread.sleep(100);
            }
            List<OnlProposalRolePrdRelation> relations = future.get();
            System.out.println(relations.size());
            proposalResultList.addAll(relations);

        }
        logger.debug("time 3 =" + (System.currentTimeMillis() - time) / 1000);
        System.out.println("stop");
    }

實際應用：

        ArrayList<String> validArrayList = new ArrayList<String>(Arrays.asList(validRiderArray)); 
        
        //step1 method start
        //List<TblProposalRiderAttr> tblProposalRiderAttrList = getValidRiderListStep1(validRiderArray, parameter,tblProposalProductSetting, tPresentrelationForRider );
        Iterable<List<String>> splitValidArrayList = Iterables.partition(validArrayList, 15);
        Iterator<List<String>> iterator = splitValidArrayList.iterator();
        List<Future<List<TblProposalRiderAttr>>> tblRiderFutures = new ArrayList<Future<List<TblProposalRiderAttr>>>();
        while (iterator.hasNext()) {
            List<String> tmpValidArrayList = iterator.next();
            //調用多線程執行  future為單個list的返回結果; tblRiderFutures為所有返回結果的集合
            Future<List<TblProposalRiderAttr>> future = SimpleFixedThreadPoolUtil
                    .addTask((Callable<List<TblProposalRiderAttr>>) () -> {
                        List<TblProposalRiderAttr> tmpTblRiderList = null;
                        try {
                            //調用step1方法進行篩選附加產品信息  進行多線程處理的方法
                            tmpTblRiderList = getValidRiderListStep1(tmpValidArrayList, parameter,tblProposalProductSetting,tPresentrelationForRider);
                            return tmpTblRiderList;
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                        return tmpTblRiderList;
                    });
            tblRiderFutures.add(future);
        }
        //將篩選後的結果添加到集合中
        List<TblProposalRiderAttr> tblProposalRiderAttrList = new ArrayList<TblProposalRiderAttr>();
        for (Future<List<TblProposalRiderAttr>> future : tblRiderFutures) {
            while (!future.isDone()) {
                try {
                    Thread.sleep(100);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            try {
                tblProposalRiderAttrList.addAll(future.get());
            } catch (InterruptedException e) {
                e.printStackTrace();
            } catch (ExecutionException e) {
                e.printStackTrace();
            }
        }
        //step1 method  end

多線程處理慢sql查詢小筆記~