Nginx日志分析监控系统(二)-Logstash安装配置

Logstash主要用来收集Nginx数据，在这之前为了方便，我们需要将Nginx日志格式改为json。以下是我用的格式，你可以根据自己需要修改

Nginx日式格式修改

log_format main   '{"@timestamp":"$time_iso8601",'
                        '"@source":"$server_addr",'
                        '"hostname":"$hostname",'
                        '"ip":"$http_x_forwarded_for",'
                        '"client":"$remote_addr",'
                        '"request_method":"$request_method",'
                        '"scheme":"$scheme",'
                        '"domain":"$server_name",'
                        '"referer":"$http_referer",'
                        '"request":"$request_uri",'
                        '"args":"$args",'
                        '"size":$body_bytes_sent,'
                        '"status": $status,'
                        '"responsetime":$request_time,'
                        '"upstreamtime":"$upstream_response_time",'
                        '"upstreamaddr":"$upstream_addr",'
                        '"http_user_agent":"$http_user_agent",'
                        '"https":"$https"'
                        '}';

修改完之后记得

-s reload```

我nginx访问日志命名格式是 domain_access.log,存放目录是/usr/local/nginx/logs

# 安装logstash


## 依赖解决
Logstash依赖JDK,请自行百度解决。还有我们等会需要依赖GeoIP

yum install GeoIP* -y

## 下载源码包
```wget https://artifacts.elastic.co/downloads/logstash/logstash-5.5.2.tar.gz

解压

zxvf logstash-5.5.2.tar.gz -C```

## 配置

cd /usr/local/logstash-5.5.0/
mkdir conf
tee conf/nginx_access.conf << EOF
input {
file {

    #这里根据自己日志命名使用正则匹配所有域名访问日志
    path => [ "/usr/local/nginx/logs/*_access.log" ]
    ignore_older => 0
codec => json
}

}

filter {
mutate {
convert => [ “status”,”integer” ]
convert => [ “size”,”integer” ]
convert => [ “upstreatime”,”float” ]
remove_field => “message”
}
geoip {
source => “ip”
}

}
output {
elasticsearch {
hosts => “127.0.0.1:9200”
index => “logstash-nginx-access-%{+YYYY.MM.dd}”
}

stdout {codec => rubydebug}

}

## 检查配置

./bin/logstash -t -f conf/nginx_access.conf
`

如果没有问题则证明logstash是ok的，但是我们现在还不能启动，需要先安装配置elasticsearch