1. 程式人生 > >[轉] flume使用(六):後臺啟動及日誌檢視

[轉] flume使用(六):後臺啟動及日誌檢視

【From】 https://blog.csdn.net/maoyuanming0806/article/details/80807087

 

處理的問題
flume 普通方式啟動會有自己自動停掉的問題,這可能是linux的程序機制把他停掉的原因。
普通執行方式:

./flume-ng agent -c ../conf -f ../conf/g01-taildir-avro-sink.conf -n agent1 -Dflume.root.logger=INFO,console

 

flume後臺執行

nohup加在原命令頭,&加在原命令尾部

nohup ./flume-ng agent -c ../conf -f ../conf/g01-taildir-avro-sink.conf -n agent1 -Dflume.root.logger=INFO,console &

有人說直接加 & 就可以了,對,但是再加上nohup可以把原本在console輸出的執行日誌輸出在nohup.out中

 

flume執行日誌檢視

這裡介紹兩種方式

nohup後臺啟動檢視控制檯日誌


以nohup 和 & 組合的執行方式,會將原本在console的輸出定向到flume的bin目錄下的nohup.out檔案中。

大多flume博文都有介紹flume的啟動命令,但是命令都是將執行日誌列印在控制檯,這是入門級的demo執行方式,實際上flume原本就配置了執行日誌輸出位置,如下小節說明

flume執行日誌根據log4j.properties配置輸出
在flume的配置檔案目錄及 conf 目錄中有log4j.properties配置檔案,其實就已經配置了flume啟動日誌輸出位置。

【log4j.prpoerties】

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 
2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # # Define some default values that can be overridden by system properties. # # For testing, it may also be convenient to specify # -Dflume.root.logger=DEBUG,console when launching flume. #flume.root.logger=DEBUG,console flume.root.logger=INFO,LOGFILE flume.log.dir=./logs flume.log.file=flume.log log4j.logger.org.apache.flume.lifecycle = INFO log4j.logger.org.jboss = WARN log4j.logger.org.mortbay = INFO log4j.logger.org.apache.avro.ipc.NettyTransceiver = WARN log4j.logger.org.apache.hadoop = INFO log4j.logger.org.apache.hadoop.hive = ERROR # Define the root logger to the system property "flume.root.logger". log4j.rootLogger=${flume.root.logger} # Stock log4j rolling file appender # Default log rotation configuration log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender log4j.appender.LOGFILE.MaxFileSize=100MB log4j.appender.LOGFILE.MaxBackupIndex=10 log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file} log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout log4j.appender.LOGFILE.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n # Warning: If you enable the following appender it will fill up your disk if you don't have a cleanup job! # This uses the updated rolling file appender from log4j-extras that supports a reliable time-based rolling policy. # See http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html # Add "DAILY" to flume.root.logger above if you want to use this log4j.appender.DAILY=org.apache.log4j.rolling.RollingFileAppender log4j.appender.DAILY.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy log4j.appender.DAILY.rollingPolicy.ActiveFileName=${flume.log.dir}/${flume.log.file} log4j.appender.DAILY.rollingPolicy.FileNamePattern=${flume.log.dir}/${flume.log.file}.%d{yyyy-MM-dd} log4j.appender.DAILY.layout=org.apache.log4j.PatternLayout log4j.appender.DAILY.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n # console # Add "console" to flume.root.logger above if you want to use this log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.target=System.err log4j.appender.console.layout=org.apache.log4j.PatternLayout log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n

 然而大多博文的命令都是如此:……. -Dflume.root.logger=INFO,console

./flume-ng agent -c ../conf -f ../conf/g01-taildir-avro-sink.conf -n agent1 -Dflume.root.logger=INFO,console

 

其中  -Dflume.root.logger=INFO,console 

就表示將執行日誌輸出到控制檯。

 

那麼把該條刪除就可以按照log4j的配置輸出到對應日誌檔案中

 

注意,如果沒有輸出到對應日誌檔案,那麼看下flume的lib包中是否含有log4j的依賴jar包

 

log4j-1.2.16.jar
slf4j-api-1.6.4.jar
slf4j-log4j12-1.6.1.j

 

---------------------
作者:maoyuanming0806
來源:CSDN
原文:https://blog.csdn.net/maoyuanming0806/article/details/80807087
版權宣告:本文為博主原創文章,轉載請附上博文連結!