1. 程式人生 > >log4j滾動輸出壓縮格式的檔案

log4j滾動輸出壓縮格式的檔案

因為需求需要,將原來預設切分生成的檔案輸出為壓縮包的格式,不要原始檔的格式,這樣能節省儲存空間。話不多說,下面為程式碼

(注:下面程式碼只是實現了在maxBackupIndex<0的情況下同時按照日期和檔案大小輸出壓縮包gz的格式,以後可以做成靈活擴充套件,比如在配置檔案裡面配置,然後程式碼裡去實現相應壓縮包的生成,本文只是提供一種參考):

package com.ihangmei.datapro.commons.logger;


import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.Writer;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;
import java.util.TimeZone;
import java.util.zip.GZIPOutputStream;

import org.apache.log4j.FileAppender;
import org.apache.log4j.Layout;
import org.apache.log4j.helpers.CountingQuietWriter;
import org.apache.log4j.helpers.LogLog;
import org.apache.log4j.helpers.OptionConverter;
import org.apache.log4j.spi.LoggingEvent;

/**
 * 目前滾動在maxBackupIndex可以生成無限個的條件下是以gz包的形式分割檔案的
 * 後期可以擴充套件成定製化的相容各種格式的檔案,需要自己實現寫入各類壓縮包的程式碼
 * @author 
 *
 */

public class MyDailyRollingFileAppender extends FileAppender {

    // The code assumes that the following constants are in a increasing
    // sequence.
    static final int TOP_OF_TROUBLE = -1;

    static final int TOP_OF_MINUTE = 0;

    static final int TOP_OF_HOUR = 1;

    static final int HALF_DAY = 2;

    static final int TOP_OF_DAY = 3;

    static final int TOP_OF_WEEK = 4;

    static final int TOP_OF_MONTH = 5;
    
    
    static final int BUFFER_GZIP = 1024 * 1024 * 2;//2M
    
    static final String SUFFIX_GZIP = ".gz";

    /**
     * The default maximum file size is 10MB.
     */
    protected long maxFileSize = 10 * 1024 * 1024;

    /**
     * There is one backup file by default.
     */
    protected int maxBackupIndex = 1;

    /**
     * meaning daily rollover.
     */
    private String datePattern = "'.'yyyy-MM-dd";

    /**
     * "scheduledFilename" at the beginning of the next hour.
     */
    private String scheduledFilename;

    /**
     * The next time we estimate a rollover should occur.
     */
    private long nextCheck = System.currentTimeMillis() - 1;

    Date now = new Date();

    SimpleDateFormat sdf;

    RollingCalendar rc = new RollingCalendar();

    int checkPeriod = TOP_OF_TROUBLE;

    // The gmtTimeZone is used only in computeCheckPeriod() method.
    static final TimeZone gmtTimeZone = TimeZone.getTimeZone("GMT");

    /**
     * The default constructor does nothing.
     */
    public MyDailyRollingFileAppender() {
    }

    /**
     * ouput destination for this appender.
     */
    public MyDailyRollingFileAppender(Layout layout, String filename, String datePattern) throws IOException {
        super(layout, filename, true);
        this.datePattern = datePattern;
        activateOptions();
    }

    /**
     * being rolled over to backup files.
     *
     * @since 1.1
     */
    public long getMaximumFileSize() {
        return maxFileSize;
    }

    /**
     * being rolled over to backup files.
     * <p>
     * <p>
     * JavaBeans {@link java.beans.Introspector Introspector}.
     *
     * @see #setMaxFileSize(String)
     */
    public void setMaximumFileSize(long maxFileSize) {
        this.maxFileSize = maxFileSize;
    }

    /**
     * being rolled over to backup files.
     * <p>
     * <p>
     * the value "10KB" will be interpreted as 10240.
     */
    public void setMaxFileSize(String value) {
        maxFileSize = OptionConverter.toFileSize(value, maxFileSize + 1);
    }

    /**
     * Returns the value of the <b>MaxBackupIndex</b> option.
     */
    public int getMaxBackupIndex() {
        return maxBackupIndex;
    }

    /**
     * Set the maximum number of backup files to keep around.
     * <p>
     * <p>
     */
    public void setMaxBackupIndex(int maxBackups) {
        this.maxBackupIndex = maxBackups;
    }

    /**
     */
    public void setDatePattern(String pattern) {
        datePattern = pattern;
    }

    /**
     * Returns the value of the <b>DatePattern</b> option.
     */
    public String getDatePattern() {
        return datePattern;
    }

    @Override
    public void activateOptions() {
        super.activateOptions();
        if (datePattern != null && fileName != null) {
            now.setTime(System.currentTimeMillis());
            sdf = new SimpleDateFormat(datePattern);
            int type = computeCheckPeriod();
            printPeriodicity(type);
            rc.setType(type);
            File file = new File(fileName);
            scheduledFilename = fileName + sdf.format(new Date(file.lastModified())) + SUFFIX_GZIP;

        } else {
            LogLog.error("Either File or DatePattern options are not set for appender [" + name + "].");
        }
    }

    void printPeriodicity(int type) {
        switch (type) {
            case TOP_OF_MINUTE:
                LogLog.debug("Appender [" + name + "] to be rolled every minute.");
                break;
            case TOP_OF_HOUR:
                LogLog.debug("Appender [" + name + "] to be rolled on top of every hour.");
                break;
            case HALF_DAY:
                LogLog.debug("Appender [" + name + "] to be rolled at midday and midnight.");
                break;
            case TOP_OF_DAY:
                LogLog.debug("Appender [" + name + "] to be rolled at midnight.");
                break;
            case TOP_OF_WEEK:
                LogLog.debug("Appender [" + name + "] to be rolled at start of week.");
                break;
            case TOP_OF_MONTH:
                LogLog.debug("Appender [" + name + "] to be rolled at start of every month.");
                break;
            default:
                LogLog.warn("Unknown periodicity for appender [" + name + "].");
        }
    }

    // This method computes the roll over period by looping over the
    // periods, starting with the shortest, and stopping when the r0 is
    // different from from r1, where r0 is the epoch formatted according
    // the datePattern (supplied by the user) and r1 is the
    // epoch+nextMillis(i) formatted according to datePattern. All date
    // formatting is done in GMT and not local format because the test
    // logic is based on comparisons relative to 1970-01-01 00:00:00
    // GMT (the epoch).

    int computeCheckPeriod() {
        RollingCalendar rollingCalendar = new RollingCalendar(gmtTimeZone, Locale.ENGLISH);
        // set sate to 1970-01-01 00:00:00 GMT
        Date epoch = new Date(0);
        if (datePattern != null) {
            for (int i = TOP_OF_MINUTE; i <= TOP_OF_MONTH; i++) {
                SimpleDateFormat simpleDateFormat = new SimpleDateFormat(datePattern);
                simpleDateFormat.setTimeZone(gmtTimeZone); // do all date
                // formatting in GMT
                String r0 = simpleDateFormat.format(epoch);
                rollingCalendar.setType(i);
                Date next = new Date(rollingCalendar.getNextCheckMillis(epoch));
                String r1 = simpleDateFormat.format(next);
                // System.out.println("Type = "+i+", r0 = "+r0+", r1 = "+r1);
                if (r0 != null && r1 != null && !r0.equals(r1)) {
                    return i;
                }
            }
        }
        return TOP_OF_TROUBLE; // Deliberately head for trouble...
    }

    /**
     * Implements the usual roll over behaviour.
     * <p>
     * <p>
     * If <code>MaxBackupIndex</code> is positive, then files {
     * <p>
     * <p>
     * <p>
     * If <code>MaxBackupIndex</code> is equal to zero, then the
     * <code>File</code> is truncated with no backup files created.
     */
    public // synchronization not necessary since doAppend is alreasy synched
    void sizeRollOver() {
        File target;
        File file;

        LogLog.debug("rolling over count=" + ((CountingQuietWriter) qw).getCount());
        LogLog.debug("maxBackupIndex=" + maxBackupIndex);

        String datedFilename = fileName + sdf.format(now);

        if (maxBackupIndex > 0) {
            // Delete the oldest file, to keep Windows happy.
            file = new File(datedFilename + '.' + maxBackupIndex);
            if (file.exists())
                file.delete();

            // Map {(maxBackupIndex - 1), ..., 2, 1} to {maxBackupIndex, ..., 3,
            // 2}
            for (int i = maxBackupIndex - 1; i >= 1; i--) {
                file = new File(datedFilename + "." + i);
                if (file.exists()) {
                    target = new File(datedFilename + '.' + (i + 1));
                    LogLog.debug("Renaming file " + file + " to " + target);
                    file.renameTo(target);
                }
            }

            // Rename fileName to datedFilename.1
            target = new File(datedFilename + "." + 1);

            this.closeFile(); // keep windows happy.

            file = new File(fileName);
            LogLog.debug("Renaming file " + file + " to " + target);
            file.renameTo(target);
        } else if (maxBackupIndex < 0) { // infinite number of files
            // find the max backup index
            for (int i = 1; i < Integer.MAX_VALUE; i++) {
                target = new File(datedFilename + "." + i + SUFFIX_GZIP);
                if (!target.exists()) { // Rename fileName to datedFilename.i
                    this.closeFile();
                    writeGzip(target);
//                    file = new File(fileName);
//                    file.renameTo(target);
//                    LogLog.debug("Renaming file " + file + " to " + target);
                    break;
                }
            }
        }

        try {
            // This will also close the file. This is OK since multiple
            // close operations are safe.
            this.setFile(fileName, false, bufferedIO, bufferSize);
        } catch (IOException e) {
            LogLog.error("setFile(" + fileName + ", false) call failed.", e);
        }
        scheduledFilename = datedFilename + SUFFIX_GZIP;
    }

    @Override
    public synchronized void setFile(String fileName, boolean append, boolean bufferedIO, int bufferSize) throws IOException {
        super.setFile(fileName, append, this.bufferedIO, this.bufferSize);
        if (append) {
            File f = new File(fileName);
            ((CountingQuietWriter) qw).setCount(f.length());
        }
    }

    @Override
    protected void setQWForFiles(Writer writer) {
        this.qw = new CountingQuietWriter(writer, errorHandler);
    }

    /**
     * Rollover the current file to a new file.
     */
    void timeRollOver() throws IOException {
 
        /* Compute filename, but only if datePattern is specified */
        if (datePattern == null) {
            errorHandler.error("Missing DatePattern option in rollOver().");
            return;
        }

        String datedFilename = fileName + sdf.format(now) + SUFFIX_GZIP;
        // It is too early to roll over because we are still within the
        // bounds of the current interval. Rollover will occur once the
        // next interval is reached.
        if (scheduledFilename.equals(datedFilename)) {
            return;
        }

        // close current file, and rename it to datedFilename
        this.closeFile();

        File target = new File(scheduledFilename);
        if (target.exists()) {
            target.delete();
        }

        writeGzip(target);
        
//        File file = new File(fileName);
//        boolean result = file.renameTo(target);
//        if (result) {
//            LogLog.debug(fileName + " -> " + scheduledFilename);
//        } else {
//            LogLog.error("Failed to rename [" + fileName + "] to [" + scheduledFilename + "].");
//        }

        try {
            // This will also close the file. This is OK since multiple
            // close operations are safe.
            super.setFile(fileName, false, this.bufferedIO, this.bufferSize);
        } catch (IOException e) {
            errorHandler.error("setFile(" + fileName + ", false) call failed.");
        }
        scheduledFilename = datedFilename;
    }

    /**
     * <p>
     * rollover.
     */
    @Override
    protected void subAppend(LoggingEvent event) {
        long n = System.currentTimeMillis();

        if (n >= nextCheck) {
            now.setTime(n);
            nextCheck = rc.getNextCheckMillis(now);
            try {
                timeRollOver();
            } catch (IOException ioe) {
                LogLog.error("rollOver() failed.", ioe);
            }
        } else if ((fileName != null) && ((CountingQuietWriter) qw).getCount() >= maxFileSize) {
            sizeRollOver();
        }
        super.subAppend(event);

    }
    
    void writeGzip(File target){
    	
    	GZIPOutputStream out = null;
        FileInputStream fis = null;
        try {
        	fis = new FileInputStream(fileName);
        	out = new GZIPOutputStream(new FileOutputStream(target));
        	
        	int count=-1;
            byte data[] = new byte[BUFFER_GZIP];
            while ((count = fis.read(data, 0, BUFFER_GZIP)) != -1) {
            	out.write(data, 0, count);
            }
		}catch (Exception e) {
			e.printStackTrace();
		} finally {
			try {
				if(fis != null){
					fis.close();
				}
				if(out != null){
					out.close();
				}
			} catch (Exception e) {
				e.printStackTrace();
			}
		}
    }
}


相關推薦

log4j滾動輸出壓縮格式檔案

因為需求需要,將原來預設切分生成的檔案輸出為壓縮包的格式,不要原始檔的格式,這樣能節省儲存空間。話不多說,下面為程式碼 (注:下面程式碼只是實現了在maxBackupIndex<0的情況下同時按照日期和檔案大小輸出壓縮包gz的格式,以後可以做成靈活擴充套件,比如在配

【已解決】Matlab 畫圖輸出 EPS 格式檔案中文出現亂碼

今天遇到一個問題,當我用Matlab輸出eps格式的圖片時,中文會出現亂碼。 好啦,鼓搗一個晚上總算是搞定了。。。以下是解決方案,接好不謝! 解決方法: 網上說先輸出PDF檔案,在轉

java學習:log4j輸出xml格式的日誌檔案(log4j2篇)

上一篇講解了log4j輸出xml格式的日誌檔案,本篇講述log4j2.x版本。log4j2.x版本相比log4j1變化很大,使用起來也更麻煩,主要是jar包的依賴的問題。先上程式碼,然後再列舉遇到的問題。 1、Log類程式碼 import org.apache.logg

使用kettle將csv格式檔案輸入,sql表輸出

1.    在kettle中新建一個轉換,再儲存,再點選新建一個DB連線, 2.    Csv檔案內容 3.    按住shift連線兩個圖示,

LearnPython - Zip格式檔案的解壓縮

1 import zipfile 2 import os 3 4 5 def unzip(zip_name, target_dir): 6 files = zipfile.ZipFile(zip_name) 7 for zip_file in files.namelis

autoCAD2018圖紙怎麼批處理列印輸出PDF格式檔案

因為dwg 格式需要 AutoCAD 等特定軟體才能開啟,其中涉及版本、相容性,PDF還可以保證文件完整性、受限制。比如提交審閱或交給甲方檢視時多了一層保障,也可以設定加密,限制檢視、修改、列印,如果是不需要編輯,只需要檢視、列印,pdf 擁有體積小、所見即所得(列印也是如此)、檢視方便(向

PCL系列1——讀入和輸出pcd格式點雲檔案

1.從檔案讀取點雲 寫法1: //1.loadPCDFile讀取點雲 pcl::PointCloud<pcl::PointXYZ>::Ptr cloud1(new pcl::PointCloud<pcl::PointXYZ>); if (pcl::i

問題定位分享(12)Spark儲存文字型別檔案(text、csv、json等)到hdfs時為什麼是壓縮格式

問題重現 rdd.repartition(1).write.csv(outPath) 寫檔案之後發現檔案是壓縮過的   write時首先會獲取hadoopConf,然後從中獲取是否壓縮以及壓縮格式 org.apache.spark.sql.execution.datasource

Hive的壓縮檔案儲存格式

1、壓縮 hive主要包括如下幾種壓縮:Snappy、LZ4/LZO、Gzip和Bzip2。 壓縮格式 壓縮比 檔案格式 檔案是否支援分割 Snappy 50% .

qtCSV格式檔案輸出

CSV格式的檔案 其實就是我們常用的表格文件 傳入資料時,每個資料用英文","隔開,隔開代表兩個單元格,如果不加入“\n”,就會在第一行一直寫入。 QString fileNameCSV = QFileDialog::getSaveFileName(this

l配置log4j完成日誌輸出與配置檔案log4j2.xml詳解

一、配置檔案節點解析    (1)根節點Configuration有兩個屬性:status和monitorinterval,有兩個子節點:Appenders和Loggers(表明可以定義多個Appender和Logger). status用來指定log4j本身的列印日誌的級別.

asp.net輸出重寫壓縮頁面檔案例項程式碼

例子複製程式碼 程式碼如下:using System;using System.Data;using System.Configuration;using System.Web;using System.Web.Security;using System.Web.UI;usi

Python雜俎 —— 自動壓縮指定格式檔案&自動刪除

 |||一、問題場景: 之前寫過指令碼,在遠端主機裡連線Oracle,每天自動查詢資料,匯出csv檔案,按日期命名排序。但是每天我需要自己手動壓縮然後再複製出來。結果就是一個元旦假期回來,後臺堆滿了每天的csv檔案-=。=#   所以想在原本的指令碼上新增一個每日自動壓縮當天的資

JAVA Tomcat Log4j 日誌輸出檔案

工程引用     Log4j.jar    要輸出日誌的類中引用     import org.apache.log4j.Logger;     import org.apache.log4j.PropertyConfigurator; 建立log4j.prope

java web輸出壓縮檔案到網頁

後臺程式碼: public void process(HttpServletRequest req, HttpServletResponse res) {// excel所在目錄String path = req.getRealPath("") + File.separa

壓縮zip檔案和解壓zip格式檔案

<span style="white-space:pre"> </span>/** * *單檔案壓縮 * */ @Test public void zip() throws IOException{ String path="D://compresssss

Log4j不同級別輸出到不同檔案的幾種方式

log4j已經是古董了,但是現在專案還在用,需要用到不同級別輸出到不同檔案,所以把幾種實現方式記錄下來,備忘! 下面的幾種配置都是使用properties的情況,但是xml的原理也一樣. 使用LevelRangeFilter 使用LevelMatchFil

Log4j info和error輸出到不同檔案

1、log4j提供了為不同的 Appender 設定日誌輸出級別的功能,方法是配置Appender的Threshold(log4j.appender.D.Threshold = DEBUG)。例如: ### set log levels ### log4j.rootLog

Log4j配置輸出log檔案的相對路徑

1 配置log4j log4j.properties檔案內容如下: log4j.logger.info=info log4j.appender.info=org.apache.log4j.DailyRollingFileAppender log4j.appender.i

windows cmd視窗,輸出UTF-8格式檔案,顯示亂碼

本文來自網路,參考文件見文件末尾 想在windows cmd視窗中檢視utf-8中文,需要先執行以下步驟 chcp 65001 將CMD視窗切換成UTF-8內碼表 在命令列標題欄上點選右鍵,選擇"屬性"->"字型",將字型修改為True Typ