hbase-2.1.0 原始碼閱讀
1.Hmaster啟動
用了這麼久的hbase,今天開始著手hbase方面的原始碼閱讀
2.1.0版本剛釋出不久,是Hbase 2.x系列的第二次版本。旨在提高 HBase 的穩定性和可靠性,主要更新內容如下:
基於 Procedure v2 的複製對等修改
串行復制
最小 Hadoop 版本已更改為 2.7.1
成功完成從 1.4.3 到 2.1.0 的滾動升級,這表明可以從 1.x 滾動升級到 2.x
a.我們如何啟動hbase master
linxu 系統下
bin/hbase-daemon.sh start master
我們來分析這裡發生了什麼,追本溯源
在指令碼hbase-daemon.sh中
case $startStop in (start) check_before_start hbase_rotate_log $HBASE_LOGOUT hbase_rotate_log $HBASE_LOGGC echo running $command, logging to $HBASE_LOGOUT $thiscmd --config "${HBASE_CONF_DIR}" \ foreground_start $command $args < /dev/null > ${HBASE_LOGOUT} 2>&1 & disown -h -r sleep 1; head "${HBASE_LOGOUT}" ;;
這裡發現通過start的時候 呼叫了foreground_start 來啟動
(foreground_start)
trap cleanAfterRun SIGHUP SIGINT SIGTERM EXIT
if [ "$HBASE_NO_REDIRECT_LOG" != "" ]; then
# NO REDIRECT
echo "`date` Starting $command on `hostname`"
echo "`ulimit -a`"
# in case the parent shell gets the kill make sure to trap signals.
# Only one will get called. Either the trap or the flow will go through.
nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
$command " [email protected]" start &
else
echo "`date` Starting $command on `hostname`" >> ${HBASE_LOGLOG}
echo "`ulimit -a`" >> "$HBASE_LOGLOG" 2>&1
# in case the parent shell gets the kill make sure to trap signals.
# Only one will get called. Either the trap or the flow will go through.
nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
$command "[email protected]" start >> ${HBASE_LOGOUT} 2>&1 &
fi
# Add to the command log file vital stats on our environment.
hbase_pid=$!
echo $hbase_pid > ${HBASE_PID}
wait $hbase_pid
;;
可以看到最終呼叫了"$HBASE_HOME"/bin/hbase 指令碼來start,我們繼續檢視hbase指令碼
elif [ "$COMMAND" = "master" ] ; then
CLASS='org.apache.hadoop.hbase.master.HMaster'
if [ "$1" != "stop" ] && [ "$1" != "clear" ] ; then
HBASE_OPTS="$HBASE_OPTS $HBASE_MASTER_OPTS"
fi
匹配COMMAND為master時 CLASS 為 org.apache.hadoop.hbase.master.HMaster
最後通過java 命令啟動
if [ "${HBASE_NOEXEC}" != "" ]; then
"$JAVA" -Dproc_$COMMAND -XX:OnOutOfMemoryError="kill -9 %p" $HEAP_SETTINGS $HBASE_OPTS $CLASS "[email protected]"
else
exec "$JAVA" -Dproc_$COMMAND -XX:OnOutOfMemoryError="kill -9 %p" $HEAP_SETTINGS $HBASE_OPTS $CLASS "[email protected]"
fi
我們通過jps -v -m -l |grep HMaster檢視對應的啟動命令 (這裡是HBase 1.2.4)
8984 org.apache.hadoop.hbase.master.HMaster start -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -Xmx2G -Duser.timezone=Asia/Shanghai -Xss256k -XX:PermSize=128m -XX:MaxPermSize=128m -Xmx24g -Xms24g -Xmn4G -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:ParallelCMSThreads=8 -XX:+CMSClassUnloadingEnabled -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+HeapDumpOnOutOfMemoryError -XX:+UseCMSInitiatingOccupancyOnly -XX:MaxTenuringThreshold=4 -XX:SurvivorRatio=4 -verbose:gc -Xloggc:logs/gc_log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintCommandLineFlags -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1024M -XX:PermSize=256m -XX:MaxPermSize=256m -Dhbase.log.dir=/data/hbase_home/logs -Dhbase.log.file=hbase-wifi-master-hadoop1.log -Dhbase.home.dir=/data/hbase_home -Dhbase.id.str=hadoop -Dhbase.root.logger=INFO,RFA -Djava.library.path=/data/hadoop-2.6.4/lib/native -Dhbase.security.logger=INFO,RFAS
下來我們開啟org.apache.hadoop.hbase.master.HMaster看看
主函式入口
public static void main(String [] args) {
LOG.info("STARTING service " + HMaster.class.getSimpleName());
VersionInfo.logVersion();
new HMasterCommandLine(HMaster.class).doMain(args);
}
可以發現通過呼叫HMasterCommandLine的doMain方法來啟動
解析命令列引數並執行
/**
* Parse and run the given command line. This may exit the JVM if
* a nonzero exit code is returned from <code>run()</code>.
*/
public void doMain(String args[]) {
try {
int ret = ToolRunner.run(HBaseConfiguration.create(), this, args);
if (ret != 0) {
System.exit(ret);
}
} catch (Exception e) {
LOG.error("Failed to run", e);
System.exit(-1);
}
}
進入run方法發現,他呼叫的是org.apache.hadoop.util中的ToolRunner,這裡的this將HMasterCommandLine物件實現的run方法傳遞進去,以達到呼叫的效果
public class ToolRunner {
public ToolRunner() {
}
public static int run(Configuration conf, Tool tool, String[] args) throws Exception {
if (conf == null) {
conf = new Configuration();
}
GenericOptionsParser parser = new GenericOptionsParser(conf, args);
tool.setConf(conf);
String[] toolArgs = parser.getRemainingArgs();
return tool.run(toolArgs);
}
最後呼叫了 tool.run,我們發現 doMain方法內會呼叫ToolRunner的run方法,HMasterCommandLine繼承自ServerCommandLine類,ServerCommandLine類實現了Tool介面。所以最後會呼叫HMasterCommandLine的run方法。下面我們看看HMasterCommandLine的run方法
public int run(String args[]) throws Exception {
Options opt = new Options();
opt.addOption("localRegionServers", true,
"RegionServers to start in master process when running standalone");
opt.addOption("masters", true, "Masters to start in this process");
opt.addOption("minRegionServers", true, "Minimum RegionServers needed to host user tables");
opt.addOption("backup", false, "Do not try to become HMaster until the primary fails");
CommandLine cmd;
try {
cmd = new GnuParser().parse(opt, args);
} catch (ParseException e) {
LOG.error("Could not parse: ", e);
usage(null);
return 1;
}
if (cmd.hasOption("minRegionServers")) {
String val = cmd.getOptionValue("minRegionServers");
getConf().setInt("hbase.regions.server.count.min",
Integer.parseInt(val));
LOG.debug("minRegionServers set to " + val);
}
// minRegionServers used to be minServers. Support it too.
if (cmd.hasOption("minServers")) {
String val = cmd.getOptionValue("minServers");
getConf().setInt("hbase.regions.server.count.min", Integer.parseInt(val));
LOG.debug("minServers set to " + val);
}
// check if we are the backup master - override the conf if so
if (cmd.hasOption("backup")) {
getConf().setBoolean(HConstants.MASTER_TYPE_BACKUP, true);
}
// How many regionservers to startup in this process (we run regionservers in same process as
// master when we are in local/standalone mode. Useful testing)
if (cmd.hasOption("localRegionServers")) {
String val = cmd.getOptionValue("localRegionServers");
getConf().setInt("hbase.regionservers", Integer.parseInt(val));
LOG.debug("localRegionServers set to " + val);
}
// How many masters to startup inside this process; useful testing
if (cmd.hasOption("masters")) {
String val = cmd.getOptionValue("masters");
getConf().setInt("hbase.masters", Integer.parseInt(val));
LOG.debug("masters set to " + val);
}
@SuppressWarnings("unchecked")
List<String> remainingArgs = cmd.getArgList();
if (remainingArgs.size() != 1) {
usage(null);
return 1;
}
String command = remainingArgs.get(0);
if ("start".equals(command)) {
return startMaster();
} else if ("stop".equals(command)) {
return stopMaster();
} else if ("clear".equals(command)) {
return (ZNodeClearer.clear(getConf()) ? 0 : 1);
} else {
usage("Invalid command: " + command);
return 1;
}
}
這裡的run幹了什麼?
一解析引數,二根據引數判斷是start還是stop 執行對應的方法,我們來看一些startMaster
private int startMaster() {
Configuration conf = getConf();
TraceUtil.initTracer(conf);
try {
// If 'local', defer to LocalHBaseCluster instance. Starts master
// and regionserver both in the one JVM.
if (LocalHBaseCluster.isLocal(conf)) {
DefaultMetricsSystem.setMiniClusterMode(true);
final MiniZooKeeperCluster zooKeeperCluster = new MiniZooKeeperCluster(conf);
File zkDataPath = new File(conf.get(HConstants.ZOOKEEPER_DATA_DIR));
// find out the default client port
int zkClientPort = 0;
// If the zookeeper client port is specified in server quorum, use it.
String zkserver = conf.get(HConstants.ZOOKEEPER_QUORUM);
if (zkserver != null) {
String[] zkservers = zkserver.split(",");
if (zkservers.length > 1) {
// In local mode deployment, we have the master + a region server and zookeeper server
// started in the same process. Therefore, we only support one zookeeper server.
String errorMsg = "Could not start ZK with " + zkservers.length +
" ZK servers in local mode deployment. Aborting as clients (e.g. shell) will not "
+ "be able to find this ZK quorum.";
System.err.println(errorMsg);
throw new IOException(errorMsg);
}
String[] parts = zkservers[0].split(":");
if (parts.length == 2) {
// the second part is the client port
zkClientPort = Integer.parseInt(parts [1]);
}
}
// If the client port could not be find in server quorum conf, try another conf
if (zkClientPort == 0) {
zkClientPort = conf.getInt(HConstants.ZOOKEEPER_CLIENT_PORT, 0);
// The client port has to be set by now; if not, throw exception.
if (zkClientPort == 0) {
throw new IOException("No config value for " + HConstants.ZOOKEEPER_CLIENT_PORT);
}
}
zooKeeperCluster.setDefaultClientPort(zkClientPort);
// set the ZK tick time if specified
int zkTickTime = conf.getInt(HConstants.ZOOKEEPER_TICK_TIME, 0);
if (zkTickTime > 0) {
zooKeeperCluster.setTickTime(zkTickTime);
}
// login the zookeeper server principal (if using security)
ZKUtil.loginServer(conf, HConstants.ZK_SERVER_KEYTAB_FILE,
HConstants.ZK_SERVER_KERBEROS_PRINCIPAL, null);
int localZKClusterSessionTimeout =
conf.getInt(HConstants.ZK_SESSION_TIMEOUT + ".localHBaseCluster", 10*1000);
conf.setInt(HConstants.ZK_SESSION_TIMEOUT, localZKClusterSessionTimeout);
LOG.info("Starting a zookeeper cluster");
int clientPort = zooKeeperCluster.startup(zkDataPath);
if (clientPort != zkClientPort) {
String errorMsg = "Could not start ZK at requested port of " +
zkClientPort + ". ZK was started at port: " + clientPort +
". Aborting as clients (e.g. shell) will not be able to find " +
"this ZK quorum.";
System.err.println(errorMsg);
throw new IOException(errorMsg);
}
conf.set(HConstants.ZOOKEEPER_CLIENT_PORT, Integer.toString(clientPort));
// Need to have the zk cluster shutdown when master is shutdown.
// Run a subclass that does the zk cluster shutdown on its way out.
int mastersCount = conf.getInt("hbase.masters", 1);
int regionServersCount = conf.getInt("hbase.regionservers", 1);
// Set start timeout to 5 minutes for cmd line start operations
conf.setIfUnset("hbase.master.start.timeout.localHBaseCluster", "300000");
LOG.info("Starting up instance of localHBaseCluster; master=" + mastersCount +
", regionserversCount=" + regionServersCount);
LocalHBaseCluster cluster = new LocalHBaseCluster(conf, mastersCount, regionServersCount,
LocalHMaster.class, HRegionServer.class);
((LocalHMaster)cluster.getMaster(0)).setZKCluster(zooKeeperCluster);
cluster.startup();
waitOnMasterThreads(cluster);
} else {
logProcessInfo(getConf());
HMaster master = HMaster.constructMaster(masterClass, conf);
if (master.isStopped()) {
LOG.info("Won't bring the Master up as a shutdown is requested");
return 1;
}
master.start();
master.join();
if(master.isAborted())
throw new RuntimeException("HMaster Aborted");
}
} catch (Throwable t) {
LOG.error("Master exiting", t);
return 1;
}
return 0;
}
根據配置判斷啟動那種模式的master,可以看到有兩種模式,本地模式和分散式模式。如果是分散式模式,通過反射呼叫HMaster的構造方法,並呼叫其start和join方法。如果是本地模式,將會執行一個LocalHBaseCluster,包括:LocalHMaster、HRegionServer、MiniZooKeeperCluster (Starts master and regionserver both in the one JVM)
下面看看Hmaster構造
public static HMaster constructMaster(Class<? extends HMaster> masterClass,
final Configuration conf) {
try {
Constructor<? extends HMaster> c = masterClass.getConstructor(Configuration.class);
return c.newInstance(conf);
} catch(Exception e) {
Throwable error = e;
if (e instanceof InvocationTargetException &&
((InvocationTargetException)e).getTargetException() != null) {
error = ((InvocationTargetException)e).getTargetException();
}
throw new RuntimeException("Failed construction of Master: " + masterClass.toString() + ". "
, error);
}
}
通過反射構造Hmaster,java 反射機制 通過 getConstructor獲取有引數建構函式,然後newInstance執行有引數的建構函式
Hmaster 初始化啟動
a.初始化master 初始化
初始化replication資訊
master的壓縮和加密檢查
載入表描述符
預設不開啟服務資訊的傳送
開啟頁面監聽埠
b.啟動本地的HRegionServer
c.啟動 ActiveMasterManager
public HMaster(final Configuration conf)
throws IOException, KeeperException {
//呼叫父類的有參構造方法
super(conf);
TraceUtil.initTracer(conf);
try {
this.rsFatals = new MemoryBoundedLogMessageBuffer(
conf.getLong("hbase.master.buffer.for.rs.fatals", 1 * 1024 * 1024));
LOG.info("hbase.rootdir=" + getRootDir() +
", hbase.cluster.distributed=" + this.conf.getBoolean(HConstants.CLUSTER_DISTRIBUTED, false));
// Disable usage of meta replicas in the master
this.conf.setBoolean(HConstants.USE_META_REPLICAS, false);
decorateMasterConfiguration(this.conf);
// Hack! Maps DFSClient => Master for logs. HDFS made this
// config param for task trackers, but we can piggyback off of it.
if (this.conf.get("mapreduce.task.attempt.id") == null) {
this.conf.set("mapreduce.task.attempt.id", "hb_m_" + this.serverName.toString());
}
// should we check the compression codec type at master side, default true, HBASE-6370
this.masterCheckCompression = conf.getBoolean("hbase.master.check.compression", true);
// should we check encryption settings at master side, default true
this.masterCheckEncryption = conf.getBoolean("hbase.master.check.encryption", true);
this.metricsMaster = new MetricsMaster(new MetricsMasterWrapperImpl(this));
// preload table descriptor at startup
this.preLoadTableDescriptors = conf.getBoolean("hbase.master.preload.tabledescriptors", true);
this.maxBlancingTime = getMaxBalancingTime();
this.maxRitPercent = conf.getDouble(HConstants.HBASE_MASTER_BALANCER_MAX_RIT_PERCENT,
HConstants.DEFAULT_HBASE_MASTER_BALANCER_MAX_RIT_PERCENT);
// Do we publish the status?
boolean shouldPublish = conf.getBoolean(HConstants.STATUS_PUBLISHED,
HConstants.STATUS_PUBLISHED_DEFAULT);
Class<? extends ClusterStatusPublisher.Publisher> publisherClass =
conf.getClass(ClusterStatusPublisher.STATUS_PUBLISHER_CLASS,
ClusterStatusPublisher.DEFAULT_STATUS_PUBLISHER_CLASS,
ClusterStatusPublisher.Publisher.class);
if (shouldPublish) {
if (publisherClass == null) {
LOG.warn(HConstants.STATUS_PUBLISHED + " is true, but " +
ClusterStatusPublisher.DEFAULT_STATUS_PUBLISHER_CLASS +
" is not set - not publishing status");
} else {
clusterStatusPublisherChore = new ClusterStatusPublisher(this, conf, publisherClass);
getChoreService().scheduleChore(clusterStatusPublisherChore);
}
}
// Some unit tests don't need a cluster, so no zookeeper at all
if (!conf.getBoolean("hbase.testing.nocluster", false)) {
this.activeMasterManager = new ActiveMasterManager(zooKeeper, this.serverName, this);
} else {
this.activeMasterManager = null;
}
} catch (Throwable t) {
// Make sure we log the exception. HMaster is often started via reflection and the
// cause of failed startup is lost.
LOG.error("Failed construction of Master", t);
throw t;
}
}
這裡通過super呼叫Hregionserver的有參建構函式
public HRegionServer(Configuration conf) throws IOException {
//設定執行緒名稱
super("RegionServer"); // thread name
TraceUtil.initTracer(conf);
try {
this.startcode = System.currentTimeMillis();
this.conf = conf;
this.fsOk = true;
this.masterless = conf.getBoolean(MASTERLESS_CONFIG_NAME, false);
this.eventLoopGroupConfig = setupNetty(this.conf);
MemorySizeUtil.checkForClusterFreeHeapMemoryLimit(this.conf);
HFile.checkHFileVersion(this.conf);
checkCodecs(this.conf);
this.userProvider = UserProvider.instantiate(conf);
FSUtils.setupShortCircuitRead(this.conf);
decorateRegionServerConfiguration(this.conf);
// Disable usage of meta replicas in the regionserver
this.conf.setBoolean(HConstants.USE_META_REPLICAS, false);
// Config'ed params
this.numRetries = this.conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER,
HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER);
this.threadWakeFrequency = conf.getInt(HConstants.THREAD_WAKE_FREQUENCY, 10 * 1000);
this.msgInterval = conf.getInt("hbase.regionserver.msginterval", 3 * 1000);
this.sleeper = new Sleeper(this.msgInterval, this);
boolean isNoncesEnabled = conf.getBoolean(HConstants.HBASE_RS_NONCES_ENABLED, true);
this.nonceManager = isNoncesEnabled ? new ServerNonceManager(this.conf) : null;
this.numRegionsToReport = conf.getInt("hbase.regionserver.numregionstoreport", 10);
this.operationTimeout = conf.getInt(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT,
HConstants.DEFAULT_HBASE_CLIENT_OPERATION_TIMEOUT);
this.shortOperationTimeout = conf.getInt(HConstants.HBASE_RPC_SHORTOPERATION_TIMEOUT_KEY,
HConstants.DEFAULT_HBASE_RPC_SHORTOPERATION_TIMEOUT);
this.abortRequested = false;
this.stopped = false;
//啟動rpc服務
rpcServices = createRpcServices();
useThisHostnameInstead = getUseThisHostnameInstead(conf);
String hostName =
StringUtils.isBlank(useThisHostnameInstead) ? this.rpcServices.isa.getHostName()
: this.useThisHostnameInstead;
serverName = ServerName.valueOf(hostName, this.rpcServices.isa.getPort(), this.startcode);
rpcControllerFactory = RpcControllerFactory.instantiate(this.conf);
rpcRetryingCallerFactory = RpcRetryingCallerFactory.instantiate(this.conf);
// login the zookeeper client principal (if using security)
ZKUtil.loginClient(this.conf, HConstants.ZK_CLIENT_KEYTAB_FILE,
HConstants.ZK_CLIENT_KERBEROS_PRINCIPAL, hostName);
// login the server principal (if using secure Hadoop)
login(userProvider, hostName);
// init superusers and add the server principal (if using security)
// or process owner as default super user.
//檢查超級使用者 載入超級使用者列表等
Superusers.initialize(conf);
regionServerAccounting = new RegionServerAccounting(conf);
cacheConfig = new CacheConfig(conf);
mobCacheConfig = new MobCacheConfig(conf);
uncaughtExceptionHandler = new UncaughtExceptionHandler() {
@Override
public void uncaughtException(Thread t, Throwable e) {
abort("Uncaught exception in executorService thread " + t.getName(), e);
}
};
//初始化檔案系統
initializeFileSystem();
spanReceiverHost = SpanReceiverHost.getInstance(getConfiguration());
this.configurationManager = new ConfigurationManager();
setupWindows(getConfiguration(), getConfigurationManager());
// Some unit tests don't need a cluster, so no zookeeper at all
if (!conf.getBoolean("hbase.testing.nocluster", false)) {
// Open connection to zookeeper and set primary watcher
zooKeeper = new ZKWatcher(conf, getProcessName() + ":" +
rpcServices.isa.getPort(), this, canCreateBaseZNode());
// If no master in cluster, skip trying to track one or look for a cluster status.
if (!this.masterless) {
this.csm = new ZkCoordinatedStateManager(this);
masterAddressTracker = new MasterAddressTracker(getZooKeeper(), this);
masterAddressTracker.start();
clusterStatusTracker = new ClusterStatusTracker(zooKeeper, this);
clusterStatusTracker.start();
} else {
masterAddressTracker = null;
clusterStatusTracker = null;
}
} else {
zooKeeper = null;
masterAddressTracker = null;
clusterStatusTracker = null;
}
this.rpcServices.start(zooKeeper);
// This violates 'no starting stuff in Constructor' but Master depends on the below chore
// and executor being created and takes a different startup route. Lots of overlap between HRS
// and M (An M IS A HRS now). Need to refactor so less duplication between M and its super
// Master expects Constructor to put up web servers. Ugh.
// class HRS. TODO.
this.choreService = new ChoreService(getName(), true);
this.executorService = new ExecutorService(getName());
putUpWebUI();
} catch (Throwable t) {
// Make sure we log the exception. HRegionServer is often started via reflection and the
// cause of failed startup is lost.
LOG.error("Failed construction RegionServer", t);
throw t;
}
}
下面是Hmaster 的run函式,構造完成 ,開始啟動
// Main run loop. Calls through to the regionserver run loop AFTER becoming active Master; will
// block in here until then.
@Override
public void run() {
try {
if (!conf.getBoolean("hbase.testing.nocluster", false)) {
Threads.setDaemonThreadRunning(new Thread(() -> {
try {
int infoPort = putUpJettyServer();
startActiveMasterManager(infoPort);
} catch (Throwable t) {
// Make sure we log the exception.
String error = "Failed to become Active Master";
LOG.error(error, t);
// Abort should have been called already.
if (!isAborted()) {
abort(error, t);
}
}
}));
}
// Fall in here even if we have been aborted. Need to run the shutdown services and
// the super run call will do this for us.
//一些啟動操作
super.run();
} finally {
if (this.clusterSchemaService != null) {
// If on way out, then we are no longer active master.
this.clusterSchemaService.stopAsync();
try {
this.clusterSchemaService.awaitTerminated(
getConfiguration().getInt(HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS,
DEFAULT_HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS), TimeUnit.SECONDS);
} catch (TimeoutException te) {
LOG.warn("Failed shutdown of clusterSchemaService", te);
}
}
this.activeMaster = false;
}
}
startActiveMasterManager 中master註冊到zk,判斷是否有活躍的master,沒有切換為主master,
在 super.run()中 initializeZooKeeper();setupClusterConnection() 為master建立rpc的通訊客戶端
問題導讀:
- 什麼是HBase?
- 需要熟悉所需環境?
- 原始碼需要準備?
解決方案:
- 什麼是HBase
瞭解HBase的基本概念、架構和使用場景
瞭解HBase的核心知識點和高階特性
瞭解HBase 客戶端API的資料操作
瞭解HBase運維調優的相關內容
- 熟悉所需環境安裝
JDK
Eclipse
maven
VMWare虛擬機器
ubuntu/CentOS系統
- 原始碼準備
建立Eclipse工程
簡單瞭解原始碼組織結構
使用Maven編譯HBase原始碼
將專案匯入Eclipse工程
- 原始碼
分析原始碼的一些入口:
Zookeeper啟動過程
HMaster程式碼結構
HRegionServer程式碼結構
HMaster啟動過程
RegionServer啟動過程
HMaster與RegionServer通訊過程
Compact/Split過程
LSM資料模型
插入資料流程
查詢資料流程
掃描資料流程
刪除資料流程
HFile格式
HLog格式
WAL
Lock (RowLock)
Filter例項,作用範圍
Filter集合,對應類