2. 程式人生 > >清晰weblog日誌，過濾不合法的記錄

清晰weblog日誌，過濾不合法的記錄

阿新 • • 發佈：2019-01-09

日誌：

94.237.142.21 - - [18/Sep/2013:06:49:18 +0000] "GET /wp-content/uploads/2013/07/rstudio-git3.png HTTP/1.1" 304 0 "-" "Mozilla/4.0 (compatible;)"
183.49.46.228 - - [18/Sep/2013:06:49:23 +0000] "-" 400 0 "-" "-"
163.177.71.12 - - [18/Sep/2013:06:49:33 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
163.177.71.12 - - [18/Sep/2013:06:49:36 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
101.226.68.137 - - [18/Sep/2013:06:49:42 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
101.226.68.137 - - [18/Sep/2013:06:49:45 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
60.208.6.156 - - [18/Sep/2013:06:49:48 +0000] "GET /wp-content/uploads/2013/07/rcassandra.png HTTP/1.0" 200 185524 "http://cos.name/category/software/packages/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36"
222.68.172.190 - - [18/Sep/2013:06:49:57 +0000] "GET /images/my.jpg HTTP/1.1" 200 19939 "http://www.angularjs.cn/A00n" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36"
222.68.172.190 - - [18/Sep/2013:06:50:08 +0000] "-" 400 0 "-" "-"
183.195.232.138 - - [18/Sep/2013:06:50:16 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
183.195.232.138 - - [18/Sep/2013:06:50:16 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
66.249.66.84 - - [18/Sep/2013:06:50:28 +0000] "GET /page/6/ HTTP/1.1" 200 27777 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
221.130.41.168 - - [18/Sep/2013:06:50:37 +0000] "GET /feed/ HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36"
157.55.35.40 - - [18/Sep/2013:06:51:13 +0000] "GET /robots.txt HTTP/1.1" 200 150 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
50.116.27.194 - - [18/Sep/2013:06:51:35 +0000] "POST /wp-cron.php?doing_wp_cron=1379487095.2510800361633300781250 HTTP/1.0" 200 0 "-" "WordPress/3.6; http://blog.fens.me"
58.215.204.118 - - [18/Sep/2013:06:51:35 +0000] "GET /nodejs-socketio-chat/ HTTP/1.1" 200 10818 "http://www.google.com/url?sa=t&rct=j&q=nodejs%20%E5%BC%82%E6%AD%A5%E5%B9%BF%E6%92%AD&source=web&cd=1&cad=rja&ved=0CCgQFjAA&url=%68%74%74%70%3a%2f%2f%62%6c%6f%67%2e%66%65%6e%73%2e%6d%65%2f%6e%6f%64%65%6a%73%2d%73%6f%63%6b%65%74%69%6f%2d%63%68%61%74%2f&ei=rko5UrylAefOiAe7_IGQBw&usg=AFQjCNG6YWoZsJ_bSj8kTnMHcH51hYQkAA&bvm=bv.52288139,d.aGc" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:51:36 +0000] "GET /wp-includes/js/jquery/jquery-migrate.min.js?ver=1.2.1 HTTP/1.1" 304 0 "http://blog.fens.me/nodejs-socketio-chat/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:51:35 +0000] "GET /wp-includes/js/jquery/jquery.js?ver=1.10.2 HTTP/1.1" 304 0 "http://blog.fens.me/nodejs-socketio-chat/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:51:36 +0000] "GET /wp-includes/js/comment-reply.min.js?ver=3.6 HTTP/1.1" 304 0 "http://blog.fens.me/nodejs-socketio-chat/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:51:36 +0000] "GET /wp-content/uploads/2013/08/chat.png HTTP/1.1" 200 48968 "http://blog.fens.me/nodejs-socketio-chat/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:51:36 +0000] "GET /wp-content/uploads/2013/08/chat2.png HTTP/1.1" 200 59852 "http://blog.fens.me/nodejs-socketio-chat/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:51:37 +0000] "GET /wp-content/uploads/2013/08/socketio.png HTTP/1.1" 200 80493 "http://blog.fens.me/nodejs-socketio-chat/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.248.178.212 - - [18/Sep/2013:06:51:37 +0000] "GET /nodejs-grunt-intro/ HTTP/1.1" 200 51770 "http://blog.fens.me/series-nodejs/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.248.178.212 - - [18/Sep/2013:06:51:40 +0000] "GET /wp-includes/js/jquery/jquery-migrate.min.js?ver=1.2.1 HTTP/1.1" 200 7200 "http://blog.fens.me/nodejs-grunt-intro/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.248.178.212 - - [18/Sep/2013:06:51:40 +0000] "GET /wp-includes/js/comment-reply.min.js?ver=3.6 HTTP/1.1" 200 786 "http://blog.fens.me/nodejs-grunt-intro/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.248.178.212 - - [18/Sep/2013:06:51:40 +0000] "GET /wp-includes/js/jquery/jquery.js?ver=1.10.2 HTTP/1.1" 200 45307 "http://blog.fens.me/nodejs-grunt-intro/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.248.178.212 - - [18/Sep/2013:06:51:40 +0000] "GET /wp-includes/js/jquery/jquery.js?ver=1.10.2 HTTP/1.1" 200 93128 "http://blog.fens.me/nodejs-grunt-intro/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.248.178.212 - - [18/Sep/2013:06:51:40 +0000] "GET /wp-includes/js/comment-reply.min.js?ver=3.6 HTTP/1.1" 200 786 "http://blog.fens.me/nodejs-grunt-intro/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.215.204.118 - - [18/Sep/2013:06:51:41 +0000] "-" 400 0 "-" "-"
58.215.204.118 - - [18/Sep/2013:06:51:41 +0000] "-" 400 0 "-" "-"
58.215.204.118 - - [18/Sep/2013:06:51:41 +0000] "-" 400 0 "-" "-"
157.55.35.40 - - [18/Sep/2013:06:51:43 +0000] "GET /rhadoop-java-basic/ HTTP/1.1" 200 26780 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
58.248.178.212 - - [18/Sep/2013:06:51:43 +0000] "GET /wp-includes/js/jquery/jquery-migrate.min.js?ver=1.2.1 HTTP/1.1" 200 7200 "http://blog.fens.me/nodejs-grunt-intro/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.248.178.212 - - [18/Sep/2013:06:51:45 +0000] "GET /wp-content/uploads/2013/08/grunt.png HTTP/1.1" 200 199040 "http://blog.fens.me/nodejs-grunt-intro/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MDDR; InfoPath.2; .NET4.0C)"
58.215.204.118 - - [18/Sep/2013:06:52:26 +0000] "GET /nodejs-async/ HTTP/1.1" 200 12647 "http://blog.fens.me/nodejs-socketio-chat/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:52:27 +0000] "GET /wp-includes/js/jquery/jquery.js?ver=1.10.2 HTTP/1.1" 304 0 "http://blog.fens.me/nodejs-async/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:52:27 +0000] "GET /wp-includes/js/jquery/jquery-migrate.min.js?ver=1.2.1 HTTP/1.1" 304 0 "http://blog.fens.me/nodejs-async/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
58.215.204.118 - - [18/Sep/2013:06:52:27 +0000] "GET /wp-includes/js/comment-reply.min.js?ver=3.6 HTTP/1.1" 304 0 "http://blog.fens.me/nodejs-async/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
163.177.71.12 - - [18/Sep/2013:06:52:29 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
58.215.204.118 - - [18/Sep/2013:06:52:29 +0000] "GET /nodejs-async/?cf_action=sync_comments&post_id=2357 HTTP/1.1" 200 48 "http://blog.fens.me/nodejs-async/" "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"
163.177.71.12 - - [18/Sep/2013:06:52:32 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
58.215.204.118 - - [18/Sep/2013:06:52:32 +0000] "-" 400 0 "-" "-"
58.215.204.118 - - [18/Sep/2013:06:52:33 +0000] "-" 400 0 "-" "-"
58.215.204.118 - - [18/Sep/2013:06:52:33 +0000] "-" 400 0 "-" "-"
101.226.68.137 - - [18/Sep/2013:06:52:36 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
101.226.68.137 - - [18/Sep/2013:06:52:39 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
183.195.232.138 - - [18/Sep/2013:06:53:12 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
183.195.232.138 - - [18/Sep/2013:06:53:12 +0000] "HEAD / HTTP/1.1" 200 20 "-" "DNSPod-Monitor/1.0"
主要時寫一個過濾邏輯


涉及的類：WebLogBean.java,WebLogParser.java,WeblogPreProcess.java

webLogBean.java

package cn.itcast.bigdata.mr.weblogwash;

public class WebLogBean {
	
    private String remote_addr;// 記錄客戶端的ip地址
    private String remote_user;// 記錄客戶端使用者名稱稱,忽略屬性"-"
    private String time_local;// 記錄訪問時間與時區
    private String request;// 記錄請求的url與http協議
    private String status;// 記錄請求狀態；成功是200
    private String body_bytes_sent;// 記錄傳送給客戶端檔案主體內容大小
    private String http_referer;// 用來記錄從那個頁面連結訪問過來的
    private String http_user_agent;// 記錄客戶瀏覽器的相關資訊

    private boolean valid = true;// 判斷資料是否合法

    
    
	public String getRemote_addr() {
		return remote_addr;
	}

	public void setRemote_addr(String remote_addr) {
		this.remote_addr = remote_addr;
	}

	public String getRemote_user() {
		return remote_user;
	}

	public void setRemote_user(String remote_user) {
		this.remote_user = remote_user;
	}

	public String getTime_local() {
		return time_local;
	}

	public void setTime_local(String time_local) {
		this.time_local = time_local;
	}

	public String getRequest() {
		return request;
	}

	public void setRequest(String request) {
		this.request = request;
	}

	public String getStatus() {
		return status;
	}

	public void setStatus(String status) {
		this.status = status;
	}

	public String getBody_bytes_sent() {
		return body_bytes_sent;
	}

	public void setBody_bytes_sent(String body_bytes_sent) {
		this.body_bytes_sent = body_bytes_sent;
	}

	public String getHttp_referer() {
		return http_referer;
	}

	public void setHttp_referer(String http_referer) {
		this.http_referer = http_referer;
	}

	public String getHttp_user_agent() {
		return http_user_agent;
	}

	public void setHttp_user_agent(String http_user_agent) {
		this.http_user_agent = http_user_agent;
	}

	public boolean isValid() {
		return valid;
	}

	public void setValid(boolean valid) {
		this.valid = valid;
	}
    
    
	@Override
	public String toString() {
        StringBuilder sb = new StringBuilder();
        sb.append(this.valid);
        sb.append("\001").append(this.remote_addr);
        sb.append("\001").append(this.remote_user);
        sb.append("\001").append(this.time_local);
        sb.append("\001").append(this.request);
        sb.append("\001").append(this.status);
        sb.append("\001").append(this.body_bytes_sent);
        sb.append("\001").append(this.http_referer);
        sb.append("\001").append(this.http_user_agent);
        return sb.toString();
}
}

WebLogParser.java,
package cn.itcast.bigdata.mr.weblogwash;

import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;

public class WebLogParser {

	static SimpleDateFormat sd1 = new SimpleDateFormat("dd/MMM/yyyy:HH:mm:ss", Locale.US);

	static SimpleDateFormat sd2 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

	public static WebLogBean parser(String line) {
		WebLogBean webLogBean = new WebLogBean();
		String[] arr = line.split(" ");
		if (arr.length > 11) {
			webLogBean.setRemote_addr(arr[0]);
			webLogBean.setRemote_user(arr[1]);
			webLogBean.setTime_local(parseTime(arr[3].substring(1)));
			webLogBean.setRequest(arr[6]);
			webLogBean.setStatus(arr[8]);
			webLogBean.setBody_bytes_sent(arr[9]);
			webLogBean.setHttp_referer(arr[10]);

			if (arr.length > 12) {
				webLogBean.setHttp_user_agent(arr[11] + " " + arr[12]);
			} else {
				webLogBean.setHttp_user_agent(arr[11]);
			}
			if (Integer.parseInt(webLogBean.getStatus()) >= 400) {// 大於400，HTTP錯誤
				webLogBean.setValid(false);
			}
		} else {
			webLogBean.setValid(false);
		}
		return webLogBean;
	}

	public static String parseTime(String dt) {

		String timeString = "";
		try {
			Date parse = sd1.parse(dt);
			timeString = sd2.format(parse);

		} catch (ParseException e) {
			e.printStackTrace();
		}

		return timeString;
	}

	public static void main(String[] args) {
		WebLogParser wp = new WebLogParser();
		String parseTime = wp.parseTime("18/Sep/2013:06:49:48");
		System.out.println(parseTime);
	}

}

WeblogPreProcess.java
package cn.itcast.bigdata.mr.weblogwash;

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class WeblogPreProcess {

	static class WeblogPreProcessMapper extends Mapper<LongWritable, Text, Text, NullWritable> {
		Text k = new Text();
		NullWritable v = NullWritable.get();

		@Override
		protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {

			String line = value.toString();
			WebLogBean webLogBean = WebLogParser.parser(line);
			//可以插入一個靜態資源過濾（.....）
			/*WebLogParser.filterStaticResource(webLogBean);*/
			if (!webLogBean.isValid())
				return;
			k.set(webLogBean.toString());
			context.write(k, v);

		}

	}

	public static void main(String[] args) throws Exception {

		Configuration conf = new Configuration();
		Job job = Job.getInstance(conf);

		job.setJarByClass(WeblogPreProcess.class);

		job.setMapperClass(WeblogPreProcessMapper.class);

		job.setOutputKeyClass(Text.class);
		job.setOutputValueClass(NullWritable.class);

		FileInputFormat.setInputPaths(job, new Path("C:/weblog/input"));
		FileOutputFormat.setOutputPath(job, new Path("C:weblog/output"));

		job.waitForCompletion(true);

	}
}

建立目錄:C:\weblog\input

 放入日誌

輸出結果

true1.162.203.134-2013-09-18 13:47:35/images/my.jpg20019939"http://www.angularjs.cn/A0d9""Mozilla/5.0 (Windows
true1.202.186.37-2013-09-18 15:39:11/wp-content/uploads/2013/08/windjs.png20034613"http://cnodejs.org/topic/521a30d4bee8d3cb1272ac0f""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/nodejs-async-windjs/20010139"http://cnodejs.org/topic/521a30d4bee8d3cb1272ac0f""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-content/themes/silesia/functions/css/shortcodes.css2002899"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-content/themes/silesia/functions/js/shortcode.js200333"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-content/themes/silesia/js/jquery.cycle.all.min.js2007784"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-content/themes/silesia/js/load.js200715"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-content/themes/silesia/style.css2007554"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-includes/js/comment-reply.min.js?ver=3.6200786"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-includes/js/jquery/jquery-migrate.min.js?ver=1.2.12007200"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:18/wp-includes/js/jquery/jquery.js?ver=1.10.220032851"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/js/baidu.js200249"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/js/google.js200475"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/themes/silesia/images/bullets/5.gif20062"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/themes/silesia/images/crubms-div.png2001255"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/themes/silesia/images/home-ico.png2001103"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/themes/silesia/images/ico-meta.gif20073"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/themes/silesia/images/natty-logo.png2001438"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/themes/silesia/images/slide-bg.png200934"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/themes/silesia/images/sprites/post-type.png2002009"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/uploads/2013/08/stage1.png20010208"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/uploads/2013/08/stage2.png20016419"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:19/wp-content/uploads/2013/08/stage3.png20015227"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:20/wp-content/uploads/2013/05/favicon.ico2001150"-""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:20/wp-content/uploads/2013/05/favicon.ico2001150"-""Mozilla/5.0 (Macintosh;
true1.202.186.37-2013-09-18 15:39:23/nodejs-async-windjs/?cf_action=sync_comments&post_id=197420048"http://blog.fens.me/nodejs-async-windjs/""Mozilla/5.0 (Macintosh;
true1.202.186.37-20

相關推薦

清晰weblog日誌，過濾不合法的記錄

日誌： 94.237.142.21 - - [18/Sep/2013:06:49:18 +0000] "GET /wp-content/uploads/2013/07/rstudio-git3.png HTTP/1.1" 304 0 "-" "Mozilla/4.0 (co

Nginx Access Log 設定訪問日誌，過濾日誌中成功的請求不輸出到日誌中

Nginx處理請求後把關於客戶端請求的資訊寫到訪問日誌。預設，訪問日誌位於 logs/access.log，寫到日誌的資訊是預定義的、組合的格式。要覆蓋預設的配置，使用log_format指令來配置一個記錄資訊的格式，同樣使用access_log 指令到設定日誌和格式和位置。

Haproxy開啟日誌，但不重複往messages裡寫

Haproxy.cfg修改內容 log           127.0.0.1 local3 #加入日誌格式，#log format    &nb

無異常日誌，就不能排查問題了？？？

## 小聲逼逼 眾所周知，日誌是排查問題的重要手段。關於日誌設計，以及怎麼根據從【使用者報障】環節開始到秒級定位問題這個我們下一期說（絕非套路），這一期，主要講一下，`在沒有異常日誌的情況下`，如何定位問題。沒有日誌當真能排查問題，不會是標題黨吧！ ![](https://img2020.cnblogs.

LNMP架構（nginx訪問日誌，Nginx日誌切割，靜態文件不記錄訪問日誌）

tab ane pac 12月 計劃 asdasd sans min ref 一、nginx訪問日誌[root@lnmp ~]# vim /usr/local/nginx/conf/nginx.conf （修改Nginx的配置文件）搜索/log_form

Nginx配置：訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

Nginx配置 訪問日誌 一、訪問日誌 1、查看Nginx日誌格式 [root@zhulinux-02 ~]# grep -A2 log_format /usr/local/nginx/conf/nginx.conf log_format combined_realip ‘$remote_ad

nginx訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

nginx訪問日誌 日誌切割 靜態文件不記錄日誌 靜態文件不記錄過期時間 nginx訪問日誌 日誌格式vim /usr/local/nginx/conf/nginx.conf //搜索log_format 編輯虛擬主機配置文件，配置日誌格式，（註意日誌後面的名稱與主配置文件保持一致）驗證訪問其

91.Nginx配置：訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

Nginx配置：訪問日誌日誌切割靜態一、訪問日誌 1、查看Nginx日誌格式 [root@sdwaqw ~]# grep -A2 log_format /usr/local/nginx/conf/nginx.conflog_format combined_realip ‘$remote_addr $http_

12.10-12.12 Nginx的訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

nginx訪問日誌12.10 Nginx訪問日誌12.11 Nginx日誌切割12.12 靜態文件不記錄日誌和過期時間12.10 Nginx訪問日誌1 打開配置文件，搜索/log_format，查看日誌文件格式。或者直接grep過濾出來[root@AliKvn vhost]# grep -A2 log_for

nginx訪問日誌，日誌切割，靜態文件不記錄日誌

-c 增加 app live 主機名 perm exp one html nginx訪問日誌 日誌格式 [root@aminglinux-02 nginx]# vim conf/nginx.conf log_format combined_realip ‘$remote_

Nginx的訪問日誌，Nginx日誌切割，Nginx不記錄靜態檔案

Nginx的訪問日誌 Nginx的日誌格式是在Nginx的主配置檔案中（/usr/local/nginx/conf/nginx.conf） [[email protected] vhost]# vim /usr/local/nginx/conf/n

過濾掉Abp框架不需要記錄的日誌

該文章是系列文章 基於.NetCore和ABP框架如何讓Windows服務執行Quartz定時作業 的其中一篇。 問題 ABP.WindowsService/Demo.MyJob/4.0.0該專案不僅包含了Job服務，還有一個Abp專案自動生成的web站點服務。之前改造了job的日誌記錄，之後嘗試改造web站

[fixed] 解決 slf4j + log4j eclipse 可以打印日誌，而在雲服務器上不能打印

.com 單獨 com img 雲服務器 -1 設置 日誌 logs 今天發現服務上沒有打印任何日誌，而log4j已經設置為了INFO 很奇怪，在eclipse中是可以打印的，也能輸出到單獨的日誌中 後來發現原來是沖突了 把log4j註釋掉即可 保留slf4j即可 [

原！tomcat啟動超時（打印了幾行日誌，後面沒了。也不報錯，處於啟動狀態，直到超時）

str tro 啟動項 dao 配置 spa 配置文件 size 結果 項目框架：spring+struts2+mybatis 今天優化代碼，改了一堆mybatis dao和xml文件，啟動項目時，就出現如標題描述的狀況：打印了幾行日誌，後面就不打印了，也不報錯，處於啟動狀

ubuntu修改網卡名稱，找不到eth0,無法ssh鏈接問題記錄

虛擬機 inux ubun mod ubunt long 問題解決 clas tar 一、修改網卡名稱： 如出現網卡名稱不是eth0,可以將/etc/default/grub文件編輯，然後添加一下內容： vi /etc/default/grub 然後通過update-

qq46843大騙子qq，我給他做了一個網站，然後他不掏錢，大騙子真實記錄！

讀取 道理 ava cep 總結 同步代碼塊 編程 類型 線程 qq46843大騙子qq他叫花總，大騙子！我給他做了一個網站，然後他不掏錢，大騙子真實記錄！qq46843大騙子qq，我給他做了一個網站，然後他不掏錢，大騙子真實記錄！有聊天截圖，真的不要相信，空間都是一些吹牛

thinkphp5.0 編輯界面驗證不合法後重定向到編輯界面時，給編輯界面重新分配數據

規則 names 得到 還記得 輸入 分配 優先級 rule 獲取 首先，看一看 錯誤場景： 比如說我們有這樣一個編輯界面： 而我們有這樣一些驗證方法（註：采用的是thinkphp5.0的驗證規則） namespace app\admin\validate; use th

編寫一個簡單登錄驗證需要記錄日誌，Servlet中的Cookie

cookie java servlet j2ee javaweb 登錄驗證並記錄日誌之前介紹了如何使用Server、mysql、tomcat等知識點編寫了一個簡單的登錄驗證。但是現在有了一個新的需求，我想要在登錄成功的時候往數據庫記錄一條日誌，登錄失敗的時候也要記錄一下。這個日誌要記錄用戶

fiddler filters 使用（fiddler只顯示指定請求，fiddler不顯示指定請求，即filter請求過濾）（轉）

alt 正則 完全 字符 真的 upload 比較 left 模塊化 fiddler filters 使用（fiddler只顯示指定請求，fiddler不顯示指定請求，即filter請求過濾） Fiddler 有一個filters可以很好的幫助我們只顯示我們關系的請求或者隱

lamp-日誌設置（訪問日誌切割、不記錄靜態文件日誌、緩存文件過期時間）

lamp 訪問日誌 訪問日誌切割 設置靜態文件不記錄日誌 設置緩存靜態文件過期時間 訪問日誌不記錄指定類型的文件 在網站大多數元素為靜態文件，比如圖片、css、js等，這些靜態文件都可以不用記錄，我們只需要記錄哪個ip訪問了那個網頁就可以； 編輯配置文件 vim /usr/local/ap