使用ELK收集、存储、监控Nginx的日志信息

曾经终败给现在 2023-02-13 12:24 1阅读 0赞

# 1.场景 #

使用ELK生态栈对Nginx的日志信息进行解析、存储，并根据解析后的数据进行监控，如果日志信息有异常，会通过自定义配置的告警服务进行告警。

# 2.架构 #

nginx ==> filebeat ==> kafka==> logstash ==> ES ==> Kibana

filebeat：只是收集客户端（能干的很少）  
kafka：缓冲，削峰（测试阶段没有选择使用）  
logstash：解析日志字段（grok表达式）

版本选择：7.4.2

# 3. nginx日志格式 #

/data/app/log/access.log

192.168.10.132 - - [08/Jul/2019:12:53:45 +0800] "GET /saudgsg/bujguj HTTP/1.0" 200 1201 "-" "ApacheBench/2.3" "-"
    192.168.10.1 - - [08/Jul/2019:12:54:36 +0800] "GET /indexfsd HTTP/1.1" 200 1201 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 SE 2.X MetaSr 1.0" "-"
    192.168.10.1 - - [08/Jul/2019:12:54:36 +0800] "GET /favicon.ico HTTP/1.1" 200 1320 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36(KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 SE 2.X MetaSr 1.0" "-"
    192.168.10.1 - - [08/Jul/2019:12:54:36 +0800] "POST /bs/base/searchIndexImage.htm?v=1&device=10 HTTP/1.1" 502 575 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 SE 2.X MetaSr 1.0" "-"
    192.168.10.1 - - [08/Jul/2019:12:54:36 +0800] "POST /bs/base/getArticleList.htm?v=1&device=10 HTTP/1.1" 502 575 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 SE 2.X MetaSr 1.0" "-"
    192.168.10.1 - - [08/Jul/2019:12:54:36 +0800] "POST /bs/thirdparthy/getShareUrl.htm?t=1562561715347 HTTP/1.1" 502 575 "-" "Mozilla/5.0 (WindowsNT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 SE 2.X MetaSr 1.0" "-"

# 4.安装组件 #

## 4.1 filebeat-7.4.2 ##

filebeat.yml

filebeat.inputs:
    
    - type: log
    
      # Change to true to enable this input configuration.
      enabled: true
    
      # Paths that should be crawled and fetched. Glob based paths.
      paths:
    
        #- /data/app/log/elk.log
        - /data/app/log/access.log
    
    
    filebeat.config.modules:
      # Glob pattern for configuration loading
      path: ${path.config}/modules.d/*.yml
    
      # Set to true to enable config reloading
      reload.enabled: true
    
      # Period on which files under path should be checked for changes
      reload.period: 10s
    
    
    setup.template.settings:
      index.number_of_shards: 1
      #index.codec: best_compression
      #_source.enabled: false
    
    
    output.logstash:
      # The Logstash hosts
      hosts: ["0.0.0.0:5044"]

## 4.2 logstash-7.4.2 ##

/data/app/logstash-7.4.2/bin/config/beats\_logstash\_es.conf    （主配置文件）

input{
        beats {
                    port => 5044
                    ssl  => false
                    codec => line
            }
    }
    
    filter{
        grok{
            patterns_dir => "/data/app/logstash-7.4.2/patterns"
            match => { "message" => "%{NGINXACCESS}" }
            remove_field => "message"
        }
    }
    
    output{
         elasticsearch {
            hosts => ["localhost:9200"]
            index =>  "beats"
            template => "/data/app/logstash-7.4.2/templete/logstash.json"
    
            manage_template => false #关闭logstash默认索引模板
            template_overwrite => true
            template_name => "crawl" #映射模板的名字
    
            user => "xxx"
            password => "xxx"
        }
    }

/data/app/logstash-7.4.2/patterns/nginx  （正则解析）

NGINXACCESS %{IP:clientip} - (%{USERNAME:user}|-) \[%{HTTPDATE:timestamp}\] \"%{WORD:request_verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:status:int} %{NUMBER:body_sent:int} \"-\" \"%{GREEDYDATA:agent}\" \"-\"

/data/app/logstash-7.4.2/templete/logstash.json  (索引模板)

{
      "index_patterns": ["*"],
      "order" : 0,
      "version": 1,
      "settings": {
        "number_of_shards": 1,
        "number_of_replicas":0
      },
        "mappings": {
          "date_detection": true,
          "numeric_detection": true,
                "dynamic_templates": [
                    {
                        "string_fields": {
                            "match": "*",
                            "match_mapping_type": "string",
                            "mapping": {
                                "type": "text",
                                "norms": false,
                                "analyzer": "ik_max_word",
                                "fields": {
                                    "keyword": {
                                        "type": "keyword"
                                    }
                                }
                            }
                        }
                    }
                ]
        }
    }

# 5. 启动顺序 #

1）ES

2）logstash

3）filebeat