HBase原始碼分析 -- HBase Region 拆分(split)
阿新 • • 發佈:2018-12-31
程式碼版本:hbase-1.2.6
工程:hbase-server
類:org.apache.hadoop.hbase.regionserver.HRegion
需要解決的問題:
1、什麼時候觸發拆分?
2、拆分的策略是什麼?
1、判斷是否需要切分
方法: checkSplit
返回值: splitpoint
做了一些判斷後,其實是呼叫:
byte[] ret = splitPolicy.getSplitPoint();
2、切分策略
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy
/** * @return the key at which the region should be split, or null * if it cannot be split. This will only be called if shouldSplit * previously returned true. */ protected byte[] getSplitPoint() { byte[] explicitSplitPoint = this.region.getExplicitSplitPoint(); if (explicitSplitPoint != null) { return explicitSplitPoint; } List<Store> stores = region.getStores(); byte[] splitPointFromLargestStore = null; long largestStoreSize = 0; for (Store s : stores) { byte[] splitPoint = s.getSplitPoint(); long storeSize = s.getSize(); if (splitPoint != null && largestStoreSize < storeSize) { splitPointFromLargestStore = splitPoint; largestStoreSize = storeSize; } } return splitPointFromLargestStore; }
從上邊程式碼看如果explicitSplitPoint不為空,則使用這個,再往上查是forceSplit賦值的
如果explicitSplitPoint為空,則region.getStores() ,根據storeSize找到splitPoint
呼叫的是HStore的getSplitPoint方法:
@Override public byte[] getSplitPoint() { this.lock.readLock().lock(); try { // Should already be enforced by the split policy! assert !this.getRegionInfo().isMetaRegion(); // Not split-able if we find a reference store file present in the store. if (hasReferences()) { return null; } return this.storeEngine.getStoreFileManager().getSplitPoint(); } catch(IOException e) { LOG.warn("Failed getting store size for " + this, e); } finally { this.lock.readLock().unlock(); } return null; }
DefaultStoreFileManager
@Override
public final byte[] getSplitPoint() throws IOException {
if (this.storefiles.isEmpty()) {
return null;
}
return StoreUtils.getLargestFile(this.storefiles).getFileSplitPoint(this.kvComparator);
}
然後到SotreFile的:
/** * Gets the approximate mid-point of this file that is optimal for use in splitting it. * @param comparator Comparator used to compare KVs. * @return The split point row, or null if splitting is not possible, or reader is null. */ @SuppressWarnings("deprecation") byte[] getFileSplitPoint(KVComparator comparator) throws IOException { if (this.reader == null) { LOG.warn("Storefile " + this + " Reader is null; cannot get split point"); return null; } // Get first, last, and mid keys. Midkey is the key that starts block // in middle of hfile. Has column and timestamp. Need to return just // the row we want to split on as midkey. byte [] midkey = this.reader.midkey(); if (midkey != null) { KeyValue mk = KeyValue.createKeyValueFromKey(midkey, 0, midkey.length); byte [] fk = this.reader.getFirstKey(); KeyValue firstKey = KeyValue.createKeyValueFromKey(fk, 0, fk.length); byte [] lk = this.reader.getLastKey(); KeyValue lastKey = KeyValue.createKeyValueFromKey(lk, 0, lk.length); // if the midkey is the same as the first or last keys, we cannot (ever) split this region. if (comparator.compareRows(mk, firstKey) == 0 || comparator.compareRows(mk, lastKey) == 0) { if (LOG.isDebugEnabled()) { LOG.debug("cannot split because midkey is the same as first or last row"); } return null; } return mk.getRow(); } return null; }