Elasticsearch6.3.2啟動過程源碼閱讀記錄
Elasticsearch6.3.2啟動過程源碼閱讀記錄
網上有很多關於es的源碼分析,覺得自己技術深度還不夠,所以這些文章只是看源碼過程中的一個筆記,談不上分析。
整個啟動過程以類名.方法名,按順序依次描述如下:
Elasticsearch.main
啟動入口類,註冊JVM關閉鉤子用來清理資源。Command.mainwithoutErrorHandling
在es正式啟動之前,加載一些命令:比如./elasticsearch -help
命令starts elasticsearch Option Description ------ ----------- -E <KeyValuePair> Configure a setting -V, --version Prints elasticsearch version information and exits -d, --daemonize Starts Elasticsearch in the background -h, --help show help
EnvironmentAwareCommand.execute
加載配置參數putSystemPropertyIfSettingIsMissing(settings, "path.data", "es.path.data"); putSystemPropertyIfSettingIsMissing(settings, "path.home", "es.path.home"); putSystemPropertyIfSettingIsMissing(settings, "path.logs", "es.path.logs");
?
InternalSettingsPrepare.prePareEnvironment
Prepares the settings by gathering all elasticsearch system properties, optionally loading the configuration settings,and then replacing all property placeholders.and then replacing all property placeholders.
ElasticSearch.execute
執行初始化命令。另外在源碼中還有看到一些有趣的註釋,比如必須設置java.io.tmpdir
config/jvm.options
文件中指定。// a misconfigured java.io.tmpdir can cause hard-to-diagnose problems later, so reject it immediately try { env.validateTmpFile(); } catch (IOException e) { throw new UserException(ExitCodes.CONFIG, e.getMessage()); }
?
Bootstrap.init
正式開始啟動ElasticSearch。This method is invoked by {@link Elasticsearch#main(String[])} to startup elasticsearch。創建節點啟動時需要的環境變量參數
final Environment environment = createEnvironment(foreground, pidFile, keystore, initialEnv.settings(), initialEnv.configFile());
checkLucene()
檢查匹配的Lucene jar包。創建節點,在第7點中將詳細分析這個過程。
node = new Node(environment) { @Override protected void validateNodeBeforeAcceptingRequests( final BootstrapContext context, final BoundTransportAddress boundTransportAddress, List<BootstrapCheck> checks) throws NodeValidationException { BootstrapChecks.check(context, boundTransportAddress, checks); } };
?
Node.java 構造方法 Node(final Environment environment, Collection<Class<? extends Plugin>> classpathPlugins)
。在這個構建方法裏面,完成了創建一個節點所需的各種信息,這個方法非常重要,下面就例舉出幾個節點創建過程中幾個重要的流程:- 設置節點環境變量信息(A component that holds all data paths for a single node.)
nodeEnvironment = new NodeEnvironment(tmpSettings, environment);
?
構造插件服務(PluginService),
this.pluginsService = new PluginsService(tmpSettings, environment.configFile(), environment.modulesFile(), environment.pluginsFile(), classpathPlugins);
看這個構造方法的註釋:
/** * Constructs a new PluginService * @param settings The settings of the system * @param modulesDirectory The directory modules exist in, or null if modules should not be loaded from the filesystem * @param pluginsDirectory The directory plugins exist in, or null if plugins should not be loaded from the filesystem * @param classpathPlugins Plugins that exist in the classpath which should be loaded */ public PluginsService(Settings settings, Path configPath, Path modulesDirectory, Path pluginsDirectory, Collection<Class<? extends Plugin>> classpathPlugins) {
其實就是加載:
elasticsearch-6.3.2/modules
和elasticsearch-6.3.2/plugins
兩個目錄下的內容。
?
創建自定義的線程池,節點執行各種任務用的吧。
final ThreadPool threadPool = new ThreadPool(settings, executorBuilders.toArray(new ExecutorBuilder[0]));
?
創建NodeClient,Client that executes actions on the local node。
client = new NodeClient(settings, threadPool);
?
AnalysisModule (An internal registry for tokenizer, token filter, char filter and analyzer)各種分詞器。
AnalysisModule analysisModule = new AnalysisModule(this.environment, pluginsService.filterPlugins(AnalysisPlugin.class));
?
SettingsModule(A module that binds the provided settings to the {@link Settings} interface) 各種配置參數用到。比如 jvm.options 和 elasticsearch.yml裏面配置的各種參數。
final SettingsModule settingsModule = new SettingsModule(this.settings, additionalSettings, additionalSettingsFilter);
?
節點是集群的一份子,肯定需要集群相關的服務
final ClusterService clusterService = new ClusterService(settings, settingsModule.getClusterSettings(), threadPool, ClusterModule.getClusterStateCustomSuppliers(clusterPlugins));
?
集群信息相關服務(Interface for a class used to gather information about a cluster at regular intervals) 周期性同步集群狀態。
final ClusterInfoService clusterInfoService = newClusterInfoService(settings, clusterService, threadPool, client, listener::onNewInfo);
?
創建Module
ModulesBuilder modules = new ModulesBuilder(); // plugin modules must be added here, before others or we can get crazy injection errors... for (Module pluginModule : pluginsService.createGuiceModules()) { modules.add(pluginModule); }
比如:SearchModule(Sets up things that can be done at search time like queries, aggregations, and suggesters)
SearchModule searchModule = new SearchModule(settings, false, pluginsService.filterPlugins(SearchPlugin.class));
還有 ActionModule(Builds and binds the generic action map, all {@link TransportAction}s, and {@link ActionFilters}.)
ActionModule actionModule = new ActionModule(false, settings, clusterModule.getIndexNameExpressionResolver(), settingsModule.getIndexScopedSettings(), settingsModule.getClusterSettings(), settingsModule.getSettingsFilter(), threadPool, pluginsService.filterPlugins(ActionPlugin.class), client, circuitBreakerService, usageService); modules.add(actionModule);
還有 DiscoveryModule(A module for loading classes for node discovery)
final DiscoveryModule discoveryModule = new DiscoveryModule(this.settings, threadPool, transportService, namedWriteableRegistry, networkService, clusterService.getMasterService(), clusterService.getClusterApplierService(), clusterService.getClusterSettings(), pluginsService.filterPlugins(DiscoveryPlugin.class), clusterModule.getAllocationService());
看一下,一共都有哪些module:
?
最終關聯了一大批的Module
modules.add(b -> { b.bind(Node.class).toInstance(this); b.bind(NodeService.class).toInstance(nodeService); b.bind(NamedXContentRegistry.class).toInstance(xContentRegistry); b.bind(PluginsService.class).toInstance(pluginsService); b.bind(Client.class).toInstance(client); b.bind(NodeClient.class).toInstance(client); b.bind(Environment.class).toInstance(this.environment); b.bind(ThreadPool.class).toInstance(threadPool); b.bind(NodeEnvironment.class).toInstance(nodeEnvironment); b.bind(ResourceWatcherService.class).toInstance(resourceWatcherService); b.bind(CircuitBreakerService.class).toInstance(circuitBreakerService); b.bind(BigArrays.class).toInstance(bigArrays); b.bind(ScriptService.class).toInstance(scriptModule.getScriptService()); b.bind(AnalysisRegistry.class).toInstance(analysisModule.getAnalysisRegistry()); b.bind(IngestService.class).toInstance(ingestService); b.bind(UsageService.class).toInstance(usageService); b.bind(NamedWriteableRegistry.class).toInstance(namedWriteableRegistry); b.bind(MetaDataUpgrader.class).toInstance(metaDataUpgrader); b.bind(MetaStateService.class).toInstance(metaStateService); b.bind(IndicesService.class).toInstance(indicesService); b.bind(SearchService.class).toInstance(searchService); b.bind(SearchTransportService.class).toInstance(searchTransportService); b.bind(SearchPhaseController.class).toInstance(new SearchPhaseController(settings, searchService::createReduceContext)); b.bind(Transport.class).toInstance(transport); b.bind(TransportService.class).toInstance(transportService); b.bind(NetworkService.class).toInstance(networkService); b.bind(UpdateHelper.class).toInstance(new UpdateHelper(settings, scriptModule.getScriptService())); b.bind(MetaDataIndexUpgradeService.class).toInstance(metaDataIndexUpgradeService); b.bind(ClusterInfoService.class).toInstance(clusterInfoService); b.bind(GatewayMetaState.class).toInstance(gatewayMetaState); b.bind(Discovery.class).toInstance(discoveryModule.getDiscovery());
總之,Node.java的構造方法裏面實現了創建一個ElasticSearch節點所必須的各種信息,想要了解ElasticSearch節點的內部結構,應該就得多看看這個方法裏面的代碼吧。
ModulesBuilder.createInjector
使用了Guice 依賴註入。injector = modules.createInjector();
Node.start
,前面創建了節點,現在開始啟動節點。(Start the node. If the node is already started, this method is no-op)先拿到對象實例,再啟動
injector.getInstance(MappingUpdatedAction.class).setClient(client); injector.getInstance(IndicesService.class).start(); injector.getInstance(IndicesClusterStateService.class).start(); injector.getInstance(SnapshotsService.class).start(); injector.getInstance(SnapshotShardsService.class).start(); injector.getInstance(RoutingService.class).start(); injector.getInstance(SearchService.class).start(); nodeService.getMonitorService().start(); //... Discovery discovery = injector.getInstance(Discovery.class); clusterService.getMasterService().setClusterStatePublisher(discovery::publish); discovery.start(); // start before cluster service so that it can set initial state on ClusterApplierService clusterService.start();
裏面的每個方法,都值得花時間去深入研究下。哈哈。。。
總結
總的來看,Elasticsearch啟動過程三大步,第一步:加載各種配置信息,這些配置信息既有自定義的配置信息,也有機器的環境變量信息,它們告訴es,我想要創建一個什麽樣的節點。第二步:創建節點,節點具有各種各樣的功能,比如說執行搜索查詢請求、選主、與其他節點同步集群狀態信息……這些功能需要各種服務/插件/模塊Module來實現。第三步:啟動節點,其實就是各種模塊、插件、服務的啟動。
最後放一張整理上上面的9個方法的調用關系圖:
原文:https://www.cnblogs.com/hapjin/p/10124269.html
Elasticsearch6.3.2啟動過程源碼閱讀記錄