ELK+syslog+nginx訪問日志收集+分詞處理
一、nginx訪問日志配置:
公司專注于為企業(yè)提供成都網(wǎng)站建設(shè)、網(wǎng)站設(shè)計、微信公眾號開發(fā)、商城網(wǎng)站建設(shè)，小程序開發(fā)，軟件按需求定制制作等一站式互聯(lián)網(wǎng)企業(yè)服務(wù)。憑借多年豐富的經(jīng)驗，我們會仔細了解各客戶的需求而做出多方面的分析、設(shè)計、整合，為客戶設(shè)計出具風(fēng)格及創(chuàng)意性的商業(yè)解決方案，成都創(chuàng)新互聯(lián)更提供一系列網(wǎng)站制作和網(wǎng)站推廣的服務(wù)。
1、日志格式配置：
log_format json '{"@timestamp":"$time_iso8601",'
'"host":"$server_addr",'
'"clientip":"$remote_addr",'
'"size":$body_bytes_sent,'
'"xff":"$http_x_forwarded_for",'
'"upstreamhost":"$upstream_addr",'
'"status":"$status",'
'"responsetime":$request_time,'
'"upstreamtime":"$upstream_response_time",'
'"http_host":"$host",'
'"url":"$uri"}';
2、訪問日志配置：
access_log syslog:server=xx.xx.xx.xx:5140 json;
二、logstash配置
input {
syslog{
port => "5140"
}
}
filter {
json {
source =>"message"
remove_field => ["message"]
}
}
output {
elasticsearch { hosts => ['xx.xx.xx.xx:9200','xx.xx.xx.xx:9200']
index => 'nginx_rfd-%{+YYYY.MM.dd.HH}'
template => "/usr/local/logstash/nginx.json"
template_name => "nginx_*"
template_overwrite => true
}
}
注意index名稱要包含到template_name里
三、自定義mappings
cat /usr/local/logstash/nginx.json
{
"template": "nginx_*",
"order":1,
"settings": { "index.refresh_interval" : "60s" },
"mappings": {
"_default_": {
"_all" : { "enabled" : false },
"properties": {
"@timestamp" : { "type" : "date" },
"@version" : { "type" : "integer", "index" : "not_analyzed" },
"url": {
"type": "string",
"index": "not_analyzed"
},
"host": {
"type": "ip",
"index": "not_analyzed"
},
"clientip": {
"type": "ip",
"index": "not_analyzed"
},
"size": {
"type": "integer"
},
"xff": {
"type": "string",
"index": "not_analyzed"
},
"upstreamhost": {
"type": "string",
"index": "not_analyzed"
},
"http_host": {
"type": "string",
"index": "not_analyzed"
},
"status": {
"type": "integer"
},
"responseTime": {
"type": "string",
"index": "not_analyzed"
},
"upstreamtime": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
四、生成統(tǒng)計圖形
五、參考文檔
https://elasticsearch.cn/article/154
http://blog.csdn.net/choelea/article/details/53320140
http://www.cnblogs.com/hanyifeng/p/5860731.html
網(wǎng)站題目：ELK+syslog+nginx訪問日志收集+分詞處理
文章地址：http://m.jiaotiyi.com/article/gsedoo.html