Giraph原始碼分析(二)—啟動Master/Worker服務
作者 | 白松
注:本文為原創,引用轉載需與數瀾聯絡。
1、org.apache.giraph.bsp.CentralizedService 介面
功能:Basic service interface shared by both CentralizedServiceMaster and CentralizedServiceWorker.
2、org.apache.giraph.bsp.CentralizedServiceMaster介面
功能:At most, there will be one active master at a time, but many threads can be trying to be the active master.
3、org.apache.giraph.bsp.CentralizedServiceWorker介面
功能:All workers should have access to this centralized service to execute the following methods.
4、org.apache.giraph.bsp.BspService抽象類
**功能:**Zookeeper-based implementation of CentralizedService.
5、org.apache.giraph.master.BspServiceMaster類
功能:ZooKeeper-based implementation of CentralizedServiceMaster.
6、org.apache.giraph.worker.BspServiceWorker類
功能:ZooKeeper-based implementation of CentralizedServiceWorker.
BspServiceWorker類有WorkerClient和WorkerServer例項,分別作為IPC通訊的客戶端和伺服器端,通過Netty來發送資料。WorkerClient例項實際為NettyWorkerClient物件,WorkerServert例項實際為NettyWorkerServer物件。
NettyWorkerClient implements WorkerClient介面,NettyWorkerServer implements WorkerServer介面。
NettyWorkerServer類的構造方法中建立一個NettyServer物件,用於底層的IPC的通訊,還有一個ServerData物件,作為資料實體。ServerData中包含該Worker的partitionStore、edgeStore、incomingMessageStore、currentMessageStore、聚集值等。
NettyWorkerClient類的構造方法中建立一個NettyClient物件,用於底層的IPC的通訊,作為客戶端。
7、org.apache.giraph.worker.InputSplitsCallable 抽象類,繼承 Callable介面。
功能:用於載入頂點或邊 輸入splits,每個執行緒都有一個WorkerClientRequestProcessor例項(實為 NettyWorkerClientRequestProcessor物件),負責向遠端的worker傳送資料。
NettyWorkerClientRequestProcessor物件用於傳送的WorkerClient物件就是BspServiceWorker裡面的WorkerClient物件。
VertexInputSplitsCallable類中的readInputSplit()方法用來從split中讀取頂點的資訊,然後呼叫NettyWorkerClientRequestProcessor物件的sendVertexRequest()方法把頂點發送到它所屬的Partition上。
8、org.apache.giraph.graph.ComputeCallable 類,繼承Callable介面。
在該物件中完成“計算-通訊-同步”的過程。每個執行緒都有一個WorkerClientRequestProcessor例項(實為 NettyWorkerClientRequestProcessor物件),負責向遠端的