【ElasticSearch】service啟動流程
阿新 • • 發佈：2021-01-08
技術標籤：大資料相關elasticsearch
目錄
Es啟動流程
main
啟動方法是org.elasticsearch.bootstrap.Elasticsearch.main，程式碼如下：
public static void main(final String[] args) throws Exception {
// we want the JVM to think there is a security manager installed so that if internal policy decisions that would be based on the 做了三件事：
- 載入安全配置
- 註冊錯誤監聽器
- 建立Elasticsearch例項。如果建立失敗則exit
接下來細看一下其中的main()。
static int main(final String[] args, final Elasticsearch elasticsearch, final Terminal terminal) throws Exception {
return elasticsearch.main(args, terminal);
}
Elasticsearch的靜態main函式呼叫了一個非靜態的main。這個main的實現位於Command中。
Command::main步驟：
- 新增Shutdown Hook。
Shutdown Hook是jvm在關閉服務時用於清理關閉資源的機制。當jvm關閉時，會先呼叫addShutdownHook中新增的所有Hook，系統執行完畢後，jvm才會關閉。因此，Shutdown Hook可以在關閉jvm時進行記憶體清理、物件銷燬等操作。
- beforeMain.run()執行main之前的操作，如日誌配置的載入。
- 呼叫mainWithoutErrorHandling。該方法位於Command類中。
public final int main(String[] args, Terminal terminal) throws Exception {
if (addShutdownHook()) {
shutdownHookThread = new Thread(() -> {
try {
this.close();
} catch (final IOException e) {
try (
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw)) {
e.printStackTrace(pw);
terminal.println(sw.toString());
} catch (final IOException impossible) {
// StringWriter#close declares a checked IOException from the Closeable interface but the Javadocs for StringWriter
// say that an exception here is impossible
throw new AssertionError(impossible);
}
}
});
Runtime.getRuntime().addShutdownHook(shutdownHookThread);
}
beforeMain.run();
try {
mainWithoutErrorHandling(args, terminal);
} catch (OptionException e) {
printHelp(terminal);
terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage());
return ExitCodes.USAGE;
} catch (UserException e) {
if (e.exitCode == ExitCodes.USAGE) {
printHelp(terminal);
}
terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage());
return e.exitCode;
}
return ExitCodes.OK;
}
Command.mainWithoutErrorHandling中執行啟動指令，程式碼如下：
void mainWithoutErrorHandling(String[] args, Terminal terminal) throws Exception {
final OptionSet options = parser.parse(args);
if (options.has(helpOption)) {
printHelp(terminal);
return;
}
if (options.has(silentOption)) {
terminal.setVerbosity(Terminal.Verbosity.SILENT);
} else if (options.has(verboseOption)) {
terminal.setVerbosity(Terminal.Verbosity.VERBOSE);
} else {
terminal.setVerbosity(Terminal.Verbosity.NORMAL);
}
execute(terminal, options);
}
核心程式碼是execute()。在Command類當中，該方法是個抽象方法：
protected abstract void execute(Terminal terminal, OptionSet options) throws Exception;
那麼具體執行的是哪個實現，就要看一下上文中涉及的兩個類的繼承關係。
execute
程式入口所在的Elasticsearch繼承了EnvironmentAwareCommand, EnvironmentAwareCommand繼承Command並實現execute方法。
因此在呼叫elasticSearch.main()時，實際執行的是EnvironmentAwareCommand的execute實現。實現程式碼如下：
protected void execute(Terminal terminal, OptionSet options) throws Exception {
final Map<String, String> settings = new HashMap<>();
for (final KeyValuePair kvp : settingOption.values(options)) {
if (kvp.value.isEmpty()) {
throw new UserException(ExitCodes.USAGE, "setting [" + kvp.key + "] must not be empty");
}
if (settings.containsKey(kvp.key)) {
final String message = String.format(
Locale.ROOT,
"setting [%s] already set, saw [%s] and [%s]",
kvp.key,
settings.get(kvp.key),
kvp.value);
throw new UserException(ExitCodes.USAGE, message);
}
settings.put(kvp.key, kvp.value);
}
putSystemPropertyIfSettingIsMissing(settings, "path.data", "es.path.data");
putSystemPropertyIfSettingIsMissing(settings, "path.home", "es.path.home");
putSystemPropertyIfSettingIsMissing(settings, "path.logs", "es.path.logs");
execute(terminal, options, createEnv(terminal, settings));
}
上述方法先讀取命令列配置，並檢查其中三個關鍵配置es.path.data、es.path.home、es.path.logs。接下來在進行execute。
這裡的execute在EnvironmentAwareCommand中同樣也是抽象函式：
protected abstract void execute(Terminal terminal, OptionSet options, Environment env) throws Exception;
這裡執行的是Elasticsearch::execute的實現。
protected void execute(Terminal terminal, OptionSet options, Environment env) throws UserException {
...
try {
init(daemonize, pidFile, quiet, env);
} catch (NodeValidationException e) {
throw new UserException(ExitCodes.CONFIG, e.getMessage());
}
}
上述程式碼省略了引數校驗的部分，我們直接關注init方法。
init
void init(final boolean daemonize, final Path pidFile, final boolean quiet, Environment initialEnv)
throws NodeValidationException, UserException {
try {
Bootstrap.init(!daemonize, pidFile, quiet, initialEnv);
} catch (BootstrapException | RuntimeException e) {
// format exceptions to the console in a special way
// to avoid 2MB stacktraces from guice, etc.
throw new StartupException(e);
}
}
Elasticsearch::init呼叫了Bootstrap.init方法，es啟動的執行過程是在方法中。程式碼很長，但是主要做了以下幾個步驟：
- 建立Bootstrap例項
- 載入安全配置
- 載入環境配置
- 配置logger
- 建立pid檔案
- 呼叫Bootstrap::setup
- 呼叫Bootstrap::start
static void init(
final boolean foreground,
final Path pidFile,
final boolean quiet,
final Environment initialEnv) throws BootstrapException, NodeValidationException, UserException {
// force the class initializer for BootstrapInfo to run before
// the security manager is installed
BootstrapInfo.init();
INSTANCE = new Bootstrap();
final SecureSettings keystore = loadSecureSettings(initialEnv);
final Environment environment = createEnvironment(foreground, pidFile, keystore, initialEnv.settings(), initialEnv.configFile());
try {
LogConfigurator.configure(environment);
} catch (IOException e) {
throw new BootstrapException(e);
}
if (environment.pidFile() != null) {
try {
PidFile.create(environment.pidFile(), true);
} catch (IOException e) {
throw new BootstrapException(e);
}
}
...
INSTANCE.setup(true, environment);
...
INSTANCE.start();
...
} catch (NodeValidationException | RuntimeException e) {
...
}
}
接下來細看一下setup和start方法。
setup方法用於初始化環境配置並建立Node例項。詳細步驟如下：
-
environment.settings獲取環境配置
-
spawnNativeControllers為每個模組（Plugin）建立native Controller，並通過ProcessBuilder建立子程序（Process）例項
-
initializeNatives初始化本地資源：
- 如果是root使用者，丟擲錯誤
- 啟用SystemCallFilter（需要JNA）
- 設定mlockAll
mlockAll配置：jvm在gc時會將堆中的頁寫回磁碟，當再次需要讀取該頁面時再從磁碟讀取。如果Elasticsearch的頁被jvm通過gc寫入磁碟，再次進行讀取時會產生大量磁碟抖動從而影響Es效能。因此，可以通過開啟memory lock將Es的堆頁面進行鎖定，不允許gc將Es的堆頁面進行交換，防止上述情況產生。
https://www.elastic.co/guide/en/elasticsearch/reference/6.3/_memory_lock_check.html
- 建立Windows關閉事件的監聽器，再監聽器當中呼叫Bootstrap::stop關閉服務
- 初始化Lucene隨機數種子
-
initializeProbes初始化探測器。該方法例項化了兩種探測器：ProcessProbe和OsProbe。分別對程序情況和系統情況進行監控。
-
新增Shutdown Hook
-
檢查Jar Hell
-
格式化輸出ifconfig日誌
-
初始化SecurityManager
-
建立Node節點。Node的建立還不是這一部分的重點，放到Node相關內容細看。
Jar Hell：jar包衝突
private void setup(boolean addShutdownHook, Environment environment) throws BootstrapException {
// 1. 獲取環境配置
Settings settings = environment.settings();
// 2. 建立native controllers並建立程序例項
try {
spawner.spawnNativeControllers(environment);
} catch (IOException e) {
throw new BootstrapException(e);
}
// 3. 初始化本地資源
initializeNatives(
environment.tmpFile(),
BootstrapSettings.MEMORY_LOCK_SETTING.get(settings),
BootstrapSettings.SYSTEM_CALL_FILTER_SETTING.get(settings),
BootstrapSettings.CTRLHANDLER_SETTING.get(settings));
// 4. 初始化程序和系統監控
// initialize probes before the security manager is installed
initializeProbes();
// 5. 新增shutdown hook
if (addShutdownHook) {
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
try {
IOUtils.close(node, spawner);
LoggerContext context = (LoggerContext) LogManager.getContext(false);
Configurator.shutdown(context);
} catch (IOException ex) {
throw new ElasticsearchException("failed to stop node", ex);
}
}
});
}
// 6. 檢查jar包衝突
try {
// look for jar hell
final Logger logger = ESLoggerFactory.getLogger(JarHell.class);
JarHell.checkJarHell(logger::debug);
} catch (IOException | URISyntaxException e) {
throw new BootstrapException(e);
}
// 7. 輸出ifconfig日誌
// Log ifconfig output before SecurityManager is installed
IfConfig.logIfNecessary();
// 8. 初始化SecurityManager
// install SM after natives, shutdown hooks, etc.
try {
Security.configure(environment, BootstrapSettings.SECURITY_FILTER_BAD_DEFAULTS_SETTING.get(settings));
} catch (IOException | NoSuchAlgorithmException e) {
throw new BootstrapException(e);
}
// 9. 建立Node例項
node = new Node(environment) {
@Override
protected void validateNodeBeforeAcceptingRequests(
final BootstrapContext context,
final BoundTransportAddress boundTransportAddress, List<BootstrapCheck> checks) throws NodeValidationException {
BootstrapChecks.check(context, boundTransportAddress, checks);
}
};
}
到這裡，節點的相關配置就設定完成了。接下來執行Bootstrap::start。
start方法的作用是啟用節點和KeepAliveThread。程式碼如下：
private void start() throws NodeValidationException {
node.start();
keepAliveThread.start();
}
node.start()的內容不在這裡詳細說。我們來看看keepAliveThread的定義。
private final CountDownLatch keepAliveLatch = new CountDownLatch(1);
private final Thread keepAliveThread;
/** creates a new instance */
Bootstrap() {
keepAliveThread = new Thread(new Runnable() {
@Override
public void run() {
try {
keepAliveLatch.await();
} catch (InterruptedException e) {
// bail out
}
}
}, "elasticsearch[keepAlive/" + Version.CURRENT + "]");
keepAliveThread.setDaemon(false);
// keep this thread alive (non daemon thread) until we shutdown
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
keepAliveLatch.countDown();
}
});
}
在Bootstrap的建構函式中初始化了keepAliveThread。其中KeepAliveLatch的詳細介紹與應用見：
https://blog.csdn.net/u012637358/article/details/90288585
在建構函式中，keepAliveLatch在Shutdown Hook當中才會呼叫countDown，因此keepAlive執行緒會一直處於await直到服務被關閉。
為什麼要有一個KeepAlive線城呢？在Es的程式碼中，main函式執行完畢後會返回ExitCode.OK並退出，查詢請求的執行將由NodeClient來執行。而主執行緒退出後，需要維持一個使用者執行緒來保持程序不退出。