.Net中EF針對大資料量查詢超時的一種優化
阿新 • • 發佈:2019-01-07
舊程式碼:
--receiptIds id集合,每次查1000左右
var mappingList = new List<FinanceSettlementMapping>();
mappingList.AddRange(SettlementMappingRepository.Entities.Include(o => o.ReceiptsTo).Include(d => d.FinanceSettlement).Where(d => receiptIds.Contains(d.ReceiptIdFrom)).ToList());
mappingList.AddRange(SettlementMappingRepository.Entities.Include(o => o.ReceiptsTo).Include(d => d.FinanceSettlement).Where(d => receiptIds.Contains(d.ReceiptIdTo)).ToList());
--FinanceSettlementMapping -500w資料,9個欄位
--ReceiptsTo,外來鍵表,FinanceReceipts --800w資料,64個欄位
--FinanceSettlement --200w資料,15個欄位
//FinanceSettlementMapping join FinanceSettlement join FinanceReceipts 表資料越大,連線越慢,可能會超時
新程式碼:外來鍵表FinanceReceipts資料單獨查詢
var mappingList = new List<FinanceSettlementMapping>();
var settlementmapFromList = SettlementMappingRepository.Entities.Include(d => d.FinanceSettlement).Where(d => receiptIds.Contains(d.ReceiptIdFrom)).ToList();
if (settlementmapFromList.Count > 0)
{
var toreceiptIds = settlementmapFromList.Select(o => o.ReceiptIdTo).ToArray();
var toreceipts = ReceiptsRepository.Entities.Where(o => toreceiptIds.Contains(o.ReceiptId)).ToList();//查詢資料到記憶體
foreach (var map in settlementmapFromList)//迴圈記憶體資料賦值
{
var tempreceipt = toreceipts.Where(o => o.ReceiptId == map.ReceiptIdTo).FirstOrDefault();
map.ReceiptsTo = tempreceipt;
}
}
mappingList.AddRange(settlementmapFromList);
var settlementmapToList = SettlementMappingRepository.Entities.Include(d => d.FinanceSettlement).Where(d => receiptIds.Contains(d.ReceiptIdTo)).ToList();
if (settlementmapToList.Count > 0)
{
var toreceiptIds = settlementmapToList.Select(o => o.ReceiptIdTo).ToArray();
var toreceipts = ReceiptsRepository.Entities.Where(o => toreceiptIds.Contains(o.ReceiptId)).ToList();//查詢資料到記憶體
foreach (var map in settlementmapToList)//迴圈記憶體資料賦值
{
var tempreceipt = toreceipts.Where(o => o.ReceiptId == map.ReceiptIdTo).FirstOrDefault();
map.ReceiptsTo = tempreceipt;
}
}
mappingList.AddRange(settlementmapToList);
//1000條資料,最差的線性查詢時間複雜度 (1+1000)*1000/2 ,本次查詢的是Id,是數字型別,如果系統用的是二叉樹或其他快速查詢方法,就更快,再者,查詢到之後,把toreceipts 值賦給map.ReceiptsTo,只是修改了map.ReceiptsTo的指標地址,沒有賦值操作,當然也是很快的,實測1000資料0.006s。
解決了大資料量表的Join查詢慢的問題,能加快程式總體執行速度,因為,我們網站應用 ,大部分時間是耗在資料庫操作上,所以能減少資料庫查詢時間,就能減少完成一個操作的整體執行時間。