1. 程式人生 > >hbase-2.1.0 原始碼閱讀

hbase-2.1.0 原始碼閱讀

1.Hmaster啟動
用了這麼久的hbase,今天開始著手hbase方面的原始碼閱讀
2.1.0版本剛釋出不久,是Hbase 2.x系列的第二次版本。旨在提高 HBase 的穩定性和可靠性,主要更新內容如下:
基於 Procedure v2 的複製對等修改
串行復制
最小 Hadoop 版本已更改為 2.7.1
成功完成從 1.4.3 到 2.1.0 的滾動升級,這表明可以從 1.x 滾動升級到 2.x

a.我們如何啟動hbase master
linxu 系統下

bin/hbase-daemon.sh start master

我們來分析這裡發生了什麼,追本溯源
在指令碼hbase-daemon.sh中


case $startStop in

(start)
    check_before_start
    hbase_rotate_log $HBASE_LOGOUT
    hbase_rotate_log $HBASE_LOGGC
    echo running $command, logging to $HBASE_LOGOUT
    $thiscmd --config "${HBASE_CONF_DIR}" \
        foreground_start $command $args < /dev/null > ${HBASE_LOGOUT} 2>&1  &
    disown -h -r
    sleep 1; head "${HBASE_LOGOUT}"
  ;;

這裡發現通過start的時候 呼叫了foreground_start 來啟動

(foreground_start)
    trap cleanAfterRun SIGHUP SIGINT SIGTERM EXIT
    if [ "$HBASE_NO_REDIRECT_LOG" != "" ]; then
        # NO REDIRECT
        echo "`date` Starting $command on `hostname`"
        echo "`ulimit -a`"
        # in case the parent shell gets the kill make sure to trap signals.
        # Only one will get called. Either the trap or the flow will go through.
        nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \
            --config "${HBASE_CONF_DIR}" \
            $command "
[email protected]
" start & else echo "`date` Starting $command on `hostname`" >> ${HBASE_LOGLOG} echo "`ulimit -a`" >> "$HBASE_LOGLOG" 2>&1 # in case the parent shell gets the kill make sure to trap signals. # Only one will get called. Either the trap or the flow will go through. nice -n $HBASE_NICENESS "$HBASE_HOME"/bin/hbase \ --config "${HBASE_CONF_DIR}" \ $command "[email protected]" start >> ${HBASE_LOGOUT} 2>&1 & fi # Add to the command log file vital stats on our environment. hbase_pid=$! echo $hbase_pid > ${HBASE_PID} wait $hbase_pid ;;

可以看到最終呼叫了"$HBASE_HOME"/bin/hbase 指令碼來start,我們繼續檢視hbase指令碼

elif [ "$COMMAND" = "master" ] ; then
  CLASS='org.apache.hadoop.hbase.master.HMaster'
  if [ "$1" != "stop" ] && [ "$1" != "clear" ] ; then
    HBASE_OPTS="$HBASE_OPTS $HBASE_MASTER_OPTS"
  fi

匹配COMMAND為master時 CLASS 為 org.apache.hadoop.hbase.master.HMaster
最後通過java 命令啟動

if [ "${HBASE_NOEXEC}" != "" ]; then
  "$JAVA" -Dproc_$COMMAND -XX:OnOutOfMemoryError="kill -9 %p" $HEAP_SETTINGS $HBASE_OPTS $CLASS "[email protected]"
else
  exec "$JAVA" -Dproc_$COMMAND -XX:OnOutOfMemoryError="kill -9 %p" $HEAP_SETTINGS $HBASE_OPTS $CLASS "[email protected]"
fi

我們通過jps -v -m -l |grep HMaster檢視對應的啟動命令 (這裡是HBase 1.2.4)

8984 org.apache.hadoop.hbase.master.HMaster start -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -Xmx2G -Duser.timezone=Asia/Shanghai -Xss256k -XX:PermSize=128m -XX:MaxPermSize=128m -Xmx24g -Xms24g -Xmn4G -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:ParallelCMSThreads=8 -XX:+CMSClassUnloadingEnabled -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+HeapDumpOnOutOfMemoryError -XX:+UseCMSInitiatingOccupancyOnly -XX:MaxTenuringThreshold=4 -XX:SurvivorRatio=4 -verbose:gc -Xloggc:logs/gc_log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintCommandLineFlags -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1024M -XX:PermSize=256m -XX:MaxPermSize=256m -Dhbase.log.dir=/data/hbase_home/logs -Dhbase.log.file=hbase-wifi-master-hadoop1.log -Dhbase.home.dir=/data/hbase_home -Dhbase.id.str=hadoop  -Dhbase.root.logger=INFO,RFA -Djava.library.path=/data/hadoop-2.6.4/lib/native -Dhbase.security.logger=INFO,RFAS

下來我們開啟org.apache.hadoop.hbase.master.HMaster看看
主函式入口

 public static void main(String [] args) {
    LOG.info("STARTING service " + HMaster.class.getSimpleName());
    VersionInfo.logVersion();
    new HMasterCommandLine(HMaster.class).doMain(args);
  }

可以發現通過呼叫HMasterCommandLine的doMain方法來啟動
解析命令列引數並執行

/**
   * Parse and run the given command line. This may exit the JVM if
   * a nonzero exit code is returned from <code>run()</code>.
   */
  public void doMain(String args[]) {
    try {
      int ret = ToolRunner.run(HBaseConfiguration.create(), this, args);
      if (ret != 0) {
        System.exit(ret);
      }
    } catch (Exception e) {
      LOG.error("Failed to run", e);
      System.exit(-1);
    }
  }

進入run方法發現,他呼叫的是org.apache.hadoop.util中的ToolRunner,這裡的this將HMasterCommandLine物件實現的run方法傳遞進去,以達到呼叫的效果

public class ToolRunner {
    public ToolRunner() {
    }

    public static int run(Configuration conf, Tool tool, String[] args) throws Exception {
        if (conf == null) {
            conf = new Configuration();
        }

        GenericOptionsParser parser = new GenericOptionsParser(conf, args);
        tool.setConf(conf);
        String[] toolArgs = parser.getRemainingArgs();
        return tool.run(toolArgs);
    }
    

最後呼叫了 tool.run,我們發現 doMain方法內會呼叫ToolRunner的run方法,HMasterCommandLine繼承自ServerCommandLine類,ServerCommandLine類實現了Tool介面。所以最後會呼叫HMasterCommandLine的run方法。下面我們看看HMasterCommandLine的run方法

public int run(String args[]) throws Exception {
    Options opt = new Options();
    opt.addOption("localRegionServers", true,
      "RegionServers to start in master process when running standalone");
    opt.addOption("masters", true, "Masters to start in this process");
    opt.addOption("minRegionServers", true, "Minimum RegionServers needed to host user tables");
    opt.addOption("backup", false, "Do not try to become HMaster until the primary fails");

    CommandLine cmd;
    try {
      cmd = new GnuParser().parse(opt, args);
    } catch (ParseException e) {
      LOG.error("Could not parse: ", e);
      usage(null);
      return 1;
    }


    if (cmd.hasOption("minRegionServers")) {
      String val = cmd.getOptionValue("minRegionServers");
      getConf().setInt("hbase.regions.server.count.min",
                  Integer.parseInt(val));
      LOG.debug("minRegionServers set to " + val);
    }

    // minRegionServers used to be minServers.  Support it too.
    if (cmd.hasOption("minServers")) {
      String val = cmd.getOptionValue("minServers");
      getConf().setInt("hbase.regions.server.count.min", Integer.parseInt(val));
      LOG.debug("minServers set to " + val);
    }

    // check if we are the backup master - override the conf if so
    if (cmd.hasOption("backup")) {
      getConf().setBoolean(HConstants.MASTER_TYPE_BACKUP, true);
    }

    // How many regionservers to startup in this process (we run regionservers in same process as
    // master when we are in local/standalone mode. Useful testing)
    if (cmd.hasOption("localRegionServers")) {
      String val = cmd.getOptionValue("localRegionServers");
      getConf().setInt("hbase.regionservers", Integer.parseInt(val));
      LOG.debug("localRegionServers set to " + val);
    }
    // How many masters to startup inside this process; useful testing
    if (cmd.hasOption("masters")) {
      String val = cmd.getOptionValue("masters");
      getConf().setInt("hbase.masters", Integer.parseInt(val));
      LOG.debug("masters set to " + val);
    }

    @SuppressWarnings("unchecked")
    List<String> remainingArgs = cmd.getArgList();
    if (remainingArgs.size() != 1) {
      usage(null);
      return 1;
    }

    String command = remainingArgs.get(0);

    if ("start".equals(command)) {
      return startMaster();
    } else if ("stop".equals(command)) {
      return stopMaster();
    } else if ("clear".equals(command)) {
      return (ZNodeClearer.clear(getConf()) ? 0 : 1);
    } else {
      usage("Invalid command: " + command);
      return 1;
    }
  }

這裡的run幹了什麼?
一解析引數,二根據引數判斷是start還是stop 執行對應的方法,我們來看一些startMaster

private int startMaster() {
    Configuration conf = getConf();
    TraceUtil.initTracer(conf);

    try {
      // If 'local', defer to LocalHBaseCluster instance.  Starts master
      // and regionserver both in the one JVM.
      if (LocalHBaseCluster.isLocal(conf)) {
        DefaultMetricsSystem.setMiniClusterMode(true);
        final MiniZooKeeperCluster zooKeeperCluster = new MiniZooKeeperCluster(conf);
        File zkDataPath = new File(conf.get(HConstants.ZOOKEEPER_DATA_DIR));

        // find out the default client port
        int zkClientPort = 0;

        // If the zookeeper client port is specified in server quorum, use it.
        String zkserver = conf.get(HConstants.ZOOKEEPER_QUORUM);
        if (zkserver != null) {
          String[] zkservers = zkserver.split(",");

          if (zkservers.length > 1) {
            // In local mode deployment, we have the master + a region server and zookeeper server
            // started in the same process. Therefore, we only support one zookeeper server.
            String errorMsg = "Could not start ZK with " + zkservers.length +
                " ZK servers in local mode deployment. Aborting as clients (e.g. shell) will not "
                + "be able to find this ZK quorum.";
              System.err.println(errorMsg);
              throw new IOException(errorMsg);
          }

          String[] parts = zkservers[0].split(":");

          if (parts.length == 2) {
            // the second part is the client port
            zkClientPort = Integer.parseInt(parts [1]);
          }
        }
        // If the client port could not be find in server quorum conf, try another conf
        if (zkClientPort == 0) {
          zkClientPort = conf.getInt(HConstants.ZOOKEEPER_CLIENT_PORT, 0);
          // The client port has to be set by now; if not, throw exception.
          if (zkClientPort == 0) {
            throw new IOException("No config value for " + HConstants.ZOOKEEPER_CLIENT_PORT);
          }
        }
        zooKeeperCluster.setDefaultClientPort(zkClientPort);
        // set the ZK tick time if specified
        int zkTickTime = conf.getInt(HConstants.ZOOKEEPER_TICK_TIME, 0);
        if (zkTickTime > 0) {
          zooKeeperCluster.setTickTime(zkTickTime);
        }

        // login the zookeeper server principal (if using security)
        ZKUtil.loginServer(conf, HConstants.ZK_SERVER_KEYTAB_FILE,
          HConstants.ZK_SERVER_KERBEROS_PRINCIPAL, null);
        int localZKClusterSessionTimeout =
          conf.getInt(HConstants.ZK_SESSION_TIMEOUT + ".localHBaseCluster", 10*1000);
        conf.setInt(HConstants.ZK_SESSION_TIMEOUT, localZKClusterSessionTimeout);
        LOG.info("Starting a zookeeper cluster");
        int clientPort = zooKeeperCluster.startup(zkDataPath);
        if (clientPort != zkClientPort) {
          String errorMsg = "Could not start ZK at requested port of " +
            zkClientPort + ".  ZK was started at port: " + clientPort +
            ".  Aborting as clients (e.g. shell) will not be able to find " +
            "this ZK quorum.";
          System.err.println(errorMsg);
          throw new IOException(errorMsg);
        }
        conf.set(HConstants.ZOOKEEPER_CLIENT_PORT, Integer.toString(clientPort));

        // Need to have the zk cluster shutdown when master is shutdown.
        // Run a subclass that does the zk cluster shutdown on its way out.
        int mastersCount = conf.getInt("hbase.masters", 1);
        int regionServersCount = conf.getInt("hbase.regionservers", 1);
        // Set start timeout to 5 minutes for cmd line start operations
        conf.setIfUnset("hbase.master.start.timeout.localHBaseCluster", "300000");
        LOG.info("Starting up instance of localHBaseCluster; master=" + mastersCount +
          ", regionserversCount=" + regionServersCount);
        LocalHBaseCluster cluster = new LocalHBaseCluster(conf, mastersCount, regionServersCount,
          LocalHMaster.class, HRegionServer.class);
        ((LocalHMaster)cluster.getMaster(0)).setZKCluster(zooKeeperCluster);
        cluster.startup();
        waitOnMasterThreads(cluster);
      } else {
        logProcessInfo(getConf());
        HMaster master = HMaster.constructMaster(masterClass, conf);
        if (master.isStopped()) {
          LOG.info("Won't bring the Master up as a shutdown is requested");
          return 1;
        }
        master.start();
        master.join();
        if(master.isAborted())
          throw new RuntimeException("HMaster Aborted");
      }
    } catch (Throwable t) {
      LOG.error("Master exiting", t);
      return 1;
    }
    return 0;
  }

根據配置判斷啟動那種模式的master,可以看到有兩種模式,本地模式和分散式模式。如果是分散式模式,通過反射呼叫HMaster的構造方法,並呼叫其start和join方法。如果是本地模式,將會執行一個LocalHBaseCluster,包括:LocalHMaster、HRegionServer、MiniZooKeeperCluster (Starts master and regionserver both in the one JVM)
下面看看Hmaster構造

public static HMaster constructMaster(Class<? extends HMaster> masterClass,
      final Configuration conf)  {
    try {
      Constructor<? extends HMaster> c = masterClass.getConstructor(Configuration.class);
      return c.newInstance(conf);
    } catch(Exception e) {
      Throwable error = e;
      if (e instanceof InvocationTargetException &&
          ((InvocationTargetException)e).getTargetException() != null) {
        error = ((InvocationTargetException)e).getTargetException();
      }
      throw new RuntimeException("Failed construction of Master: " + masterClass.toString() + ". "
        , error);
    }
  }

通過反射構造Hmaster,java 反射機制 通過 getConstructor獲取有引數建構函式,然後newInstance執行有引數的建構函式
Hmaster 初始化啟動
a.初始化master 初始化
初始化replication資訊
master的壓縮和加密檢查
載入表描述符
預設不開啟服務資訊的傳送
開啟頁面監聽埠
b.啟動本地的HRegionServer
c.啟動 ActiveMasterManager

public HMaster(final Configuration conf)
      throws IOException, KeeperException {
    //呼叫父類的有參構造方法  
    super(conf);
    TraceUtil.initTracer(conf);
    try {
      this.rsFatals = new MemoryBoundedLogMessageBuffer(
          conf.getLong("hbase.master.buffer.for.rs.fatals", 1 * 1024 * 1024));
      LOG.info("hbase.rootdir=" + getRootDir() +
          ", hbase.cluster.distributed=" + this.conf.getBoolean(HConstants.CLUSTER_DISTRIBUTED, false));

      // Disable usage of meta replicas in the master
      this.conf.setBoolean(HConstants.USE_META_REPLICAS, false);

      decorateMasterConfiguration(this.conf);

      // Hack! Maps DFSClient => Master for logs.  HDFS made this
      // config param for task trackers, but we can piggyback off of it.
      if (this.conf.get("mapreduce.task.attempt.id") == null) {
        this.conf.set("mapreduce.task.attempt.id", "hb_m_" + this.serverName.toString());
      }

      // should we check the compression codec type at master side, default true, HBASE-6370
      this.masterCheckCompression = conf.getBoolean("hbase.master.check.compression", true);

      // should we check encryption settings at master side, default true
      this.masterCheckEncryption = conf.getBoolean("hbase.master.check.encryption", true);

      this.metricsMaster = new MetricsMaster(new MetricsMasterWrapperImpl(this));

      // preload table descriptor at startup
      this.preLoadTableDescriptors = conf.getBoolean("hbase.master.preload.tabledescriptors", true);

      this.maxBlancingTime = getMaxBalancingTime();
      this.maxRitPercent = conf.getDouble(HConstants.HBASE_MASTER_BALANCER_MAX_RIT_PERCENT,
          HConstants.DEFAULT_HBASE_MASTER_BALANCER_MAX_RIT_PERCENT);

      // Do we publish the status?

      boolean shouldPublish = conf.getBoolean(HConstants.STATUS_PUBLISHED,
          HConstants.STATUS_PUBLISHED_DEFAULT);
      Class<? extends ClusterStatusPublisher.Publisher> publisherClass =
          conf.getClass(ClusterStatusPublisher.STATUS_PUBLISHER_CLASS,
              ClusterStatusPublisher.DEFAULT_STATUS_PUBLISHER_CLASS,
              ClusterStatusPublisher.Publisher.class);

      if (shouldPublish) {
        if (publisherClass == null) {
          LOG.warn(HConstants.STATUS_PUBLISHED + " is true, but " +
              ClusterStatusPublisher.DEFAULT_STATUS_PUBLISHER_CLASS +
              " is not set - not publishing status");
        } else {
          clusterStatusPublisherChore = new ClusterStatusPublisher(this, conf, publisherClass);
          getChoreService().scheduleChore(clusterStatusPublisherChore);
        }
      }

      // Some unit tests don't need a cluster, so no zookeeper at all
      if (!conf.getBoolean("hbase.testing.nocluster", false)) {
        this.activeMasterManager = new ActiveMasterManager(zooKeeper, this.serverName, this);
      } else {
        this.activeMasterManager = null;
      }
    } catch (Throwable t) {
      // Make sure we log the exception. HMaster is often started via reflection and the
      // cause of failed startup is lost.
      LOG.error("Failed construction of Master", t);
      throw t;
    }
  }
  

這裡通過super呼叫Hregionserver的有參建構函式

public HRegionServer(Configuration conf) throws IOException {
    //設定執行緒名稱
    super("RegionServer");  // thread name
    TraceUtil.initTracer(conf);
    try {
      this.startcode = System.currentTimeMillis();
      this.conf = conf;
      this.fsOk = true;
      this.masterless = conf.getBoolean(MASTERLESS_CONFIG_NAME, false);
      this.eventLoopGroupConfig = setupNetty(this.conf);
      MemorySizeUtil.checkForClusterFreeHeapMemoryLimit(this.conf);
      HFile.checkHFileVersion(this.conf);
      checkCodecs(this.conf);
      this.userProvider = UserProvider.instantiate(conf);
      FSUtils.setupShortCircuitRead(this.conf);
      decorateRegionServerConfiguration(this.conf);

      // Disable usage of meta replicas in the regionserver
      this.conf.setBoolean(HConstants.USE_META_REPLICAS, false);
      // Config'ed params
      this.numRetries = this.conf.getInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER,
          HConstants.DEFAULT_HBASE_CLIENT_RETRIES_NUMBER);
      this.threadWakeFrequency = conf.getInt(HConstants.THREAD_WAKE_FREQUENCY, 10 * 1000);
      this.msgInterval = conf.getInt("hbase.regionserver.msginterval", 3 * 1000);

      this.sleeper = new Sleeper(this.msgInterval, this);

      boolean isNoncesEnabled = conf.getBoolean(HConstants.HBASE_RS_NONCES_ENABLED, true);
      this.nonceManager = isNoncesEnabled ? new ServerNonceManager(this.conf) : null;

      this.numRegionsToReport = conf.getInt("hbase.regionserver.numregionstoreport", 10);

      this.operationTimeout = conf.getInt(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT,
          HConstants.DEFAULT_HBASE_CLIENT_OPERATION_TIMEOUT);

      this.shortOperationTimeout = conf.getInt(HConstants.HBASE_RPC_SHORTOPERATION_TIMEOUT_KEY,
          HConstants.DEFAULT_HBASE_RPC_SHORTOPERATION_TIMEOUT);

      this.abortRequested = false;
      this.stopped = false;
      //啟動rpc服務
      rpcServices = createRpcServices();
      useThisHostnameInstead = getUseThisHostnameInstead(conf);
      String hostName =
          StringUtils.isBlank(useThisHostnameInstead) ? this.rpcServices.isa.getHostName()
              : this.useThisHostnameInstead;
      serverName = ServerName.valueOf(hostName, this.rpcServices.isa.getPort(), this.startcode);

      rpcControllerFactory = RpcControllerFactory.instantiate(this.conf);
      rpcRetryingCallerFactory = RpcRetryingCallerFactory.instantiate(this.conf);

      // login the zookeeper client principal (if using security)
      ZKUtil.loginClient(this.conf, HConstants.ZK_CLIENT_KEYTAB_FILE,
          HConstants.ZK_CLIENT_KERBEROS_PRINCIPAL, hostName);
      // login the server principal (if using secure Hadoop)
      login(userProvider, hostName);
      // init superusers and add the server principal (if using security)
      // or process owner as default super user.
      //檢查超級使用者 載入超級使用者列表等
      Superusers.initialize(conf);

      regionServerAccounting = new RegionServerAccounting(conf);
      cacheConfig = new CacheConfig(conf);
      mobCacheConfig = new MobCacheConfig(conf);
      uncaughtExceptionHandler = new UncaughtExceptionHandler() {
        @Override
        public void uncaughtException(Thread t, Throwable e) {
          abort("Uncaught exception in executorService thread " + t.getName(), e);
        }
      };
      //初始化檔案系統
      initializeFileSystem();
      spanReceiverHost = SpanReceiverHost.getInstance(getConfiguration());

      this.configurationManager = new ConfigurationManager();
      setupWindows(getConfiguration(), getConfigurationManager());

      // Some unit tests don't need a cluster, so no zookeeper at all
      if (!conf.getBoolean("hbase.testing.nocluster", false)) {
        // Open connection to zookeeper and set primary watcher
        zooKeeper = new ZKWatcher(conf, getProcessName() + ":" +
          rpcServices.isa.getPort(), this, canCreateBaseZNode());
        // If no master in cluster, skip trying to track one or look for a cluster status.
        if (!this.masterless) {
          this.csm = new ZkCoordinatedStateManager(this);

          masterAddressTracker = new MasterAddressTracker(getZooKeeper(), this);
          masterAddressTracker.start();

          clusterStatusTracker = new ClusterStatusTracker(zooKeeper, this);
          clusterStatusTracker.start();
        } else {
          masterAddressTracker = null;
          clusterStatusTracker = null;
        }
      } else {
        zooKeeper = null;
        masterAddressTracker = null;
        clusterStatusTracker = null;
      }
      this.rpcServices.start(zooKeeper);
      // This violates 'no starting stuff in Constructor' but Master depends on the below chore
      // and executor being created and takes a different startup route. Lots of overlap between HRS
      // and M (An M IS A HRS now). Need to refactor so less duplication between M and its super
      // Master expects Constructor to put up web servers. Ugh.
      // class HRS. TODO.
      this.choreService = new ChoreService(getName(), true);
      this.executorService = new ExecutorService(getName());
      putUpWebUI();
    } catch (Throwable t) {
      // Make sure we log the exception. HRegionServer is often started via reflection and the
      // cause of failed startup is lost.
      LOG.error("Failed construction RegionServer", t);
      throw t;
    }
  }

下面是Hmaster 的run函式,構造完成 ,開始啟動

// Main run loop. Calls through to the regionserver run loop AFTER becoming active Master; will
  // block in here until then.
  @Override
  public void run() {
    try {
      if (!conf.getBoolean("hbase.testing.nocluster", false)) {
        Threads.setDaemonThreadRunning(new Thread(() -> {
          try {
            int infoPort = putUpJettyServer();
            startActiveMasterManager(infoPort);
          } catch (Throwable t) {
            // Make sure we log the exception.
            String error = "Failed to become Active Master";
            LOG.error(error, t);
            // Abort should have been called already.
            if (!isAborted()) {
              abort(error, t);
            }
          }
        }));
      }
      // Fall in here even if we have been aborted. Need to run the shutdown services and
      // the super run call will do this for us.
      //一些啟動操作
      super.run();
    } finally {
      if (this.clusterSchemaService != null) {
        // If on way out, then we are no longer active master.
        this.clusterSchemaService.stopAsync();
        try {
          this.clusterSchemaService.awaitTerminated(
              getConfiguration().getInt(HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS,
              DEFAULT_HBASE_MASTER_WAIT_ON_SERVICE_IN_SECONDS), TimeUnit.SECONDS);
        } catch (TimeoutException te) {
          LOG.warn("Failed shutdown of clusterSchemaService", te);
        }
      }
      this.activeMaster = false;
    }
  }

startActiveMasterManager 中master註冊到zk,判斷是否有活躍的master,沒有切換為主master,
在 super.run()中 initializeZooKeeper();setupClusterConnection() 為master建立rpc的通訊客戶端

問題導讀:

  1. 什麼是HBase?
  2. 需要熟悉所需環境?
  3. 原始碼需要準備?

解決方案:

  1. 什麼是HBase

瞭解HBase的基本概念、架構和使用場景

瞭解HBase的核心知識點和高階特性

瞭解HBase 客戶端API的資料操作

瞭解HBase運維調優的相關內容

  1. 熟悉所需環境安裝

JDK

Eclipse

maven

VMWare虛擬機器

ubuntu/CentOS系統

  1. 原始碼準備

建立Eclipse工程

簡單瞭解原始碼組織結構

使用Maven編譯HBase原始碼

將專案匯入Eclipse工程

  1. 原始碼

分析原始碼的一些入口:

Zookeeper啟動過程

HMaster程式碼結構

HRegionServer程式碼結構

HMaster啟動過程

RegionServer啟動過程

HMaster與RegionServer通訊過程

Compact/Split過程

LSM資料模型

插入資料流程

查詢資料流程

掃描資料流程

刪除資料流程

HFile格式

HLog格式

WAL

Lock (RowLock)

Filter例項,作用範圍

Filter集合,對應類