flume原始碼分析--Log4j日誌直接傳送到Flume過程分析(三)
一、分析Log4j向flume傳送日誌的過程
按照前述除錯過程,理清Log4j向flume傳送日誌所經過的過程,如下所示:
首先在初始化日誌類時,就需要讀取配置檔案資訊,並對其進行解讀定位,關鍵是用到了org.apache.flume.clients.log4jappender.Log4jAppender類。在類初始化過程中,重點對該類進行呼叫,使得log4j在輸出時可以通過netty,輸出到相應的目標當中。
private static final Logger logger = LoggerFactory.getLogger(MyTest.class);
初始化過程中其呼叫關係為log4j->log4jAppender->RpcClientFactory ->NettyAvroRpcClient,如下圖所示
可以看出,log4j將日誌直接輸送到flume當中,同傳送到檔案、console等appender中一樣,flume在其原始碼過程中,特意增加Log4jAppender類來解決相關的傳送問題。
logger.info("Now the time is:{}",String.valueOf(new Date().getTime()));
log4j在輸出的時候呼叫了log4jAppender通過netty將相關資料傳遞到flume中,有關log4j的如何呼叫Appender網上相關分析很多,大家可以自行搜尋。
二、log4jAppender分析
有關log4j的Appender自定義開發可以參考
1、繼承log4j公共的基類:AppenderSkeleton
public class Log4jAppender extends AppenderSkeleton {
private String hostname;
private int port;
……
2、重寫列印日誌核心方法:abstract protected void append(LoggingEvent event);
@Override
public synchronized void append(LoggingEvent event) throws FlumeException{
//If rpcClient is null, it means either this appender object was never
//setup by setting hostname and port and then calling activateOptions
//or this appender object was closed by calling close(), so we throw an
//exception to show the appender is no longer accessible.
if (rpcClient == null) {
String errorMsg = "Cannot Append to Appender! Appender either closed or" +
" not setup correctly!";
LogLog.error(errorMsg);
if (unsafeMode) {
return;
}
throw new FlumeException(errorMsg);
}
……
3、初始化載入資源:public void activateOptions(),
@Override
public void activateOptions() throws FlumeException {
Properties props = new Properties();
props.setProperty(RpcClientConfigurationConstants.CONFIG_HOSTS, "h1");
props.setProperty(RpcClientConfigurationConstants.CONFIG_HOSTS_PREFIX + "h1",
hostname + ":" + port);
props.setProperty(RpcClientConfigurationConstants.CONFIG_CONNECT_TIMEOUT,
……
這裡最關鍵的是在引數初始化過程中初始化了netty連線,這也是flume-NG一個比較大的改動,使用了JBOSS的netty來完成傳輸任務。
其他如釋放資源:public void close();是否需要按格式輸出文字:public boolean requiresLayout() 等等在類中都有體現,可以進一步進行仔細的分析。