Centos7.2 安装 ELK环境

谁践踏了优雅 2021-09-20 04:12 421阅读 0赞

[2019独角兽企业重金招聘Python工程师标准>>> ][2019_Python_] ![hot3.png][]

1，rpm下载：https://www.elastic.co/downloads

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.0.rpm

wget https://artifacts.elastic.co/downloads/kibana/kibana-5.3.0-x86\_64.rpm

wget https://artifacts.elastic.co/downloads/logstash/logstash-5.3.0.rpm

2，安装elasticsearch

rpm -ivh elasticsearch-5.3.0.rpm

vim /etc/elasticsearch/elasticsearch.yml

cluster.name: elasticsearch  
    node.name: node0  
    path.data: /var/lib/elasticsearch  
    path.logs: /var/log/elasticsearch  
    \#path.work: /var/elasticsearch/work  
    \#transport.tcp.port: 9300  
    http.port: 9200  
    http.host: 192.168.180.94

systemctl start elasticsearch.service

systemctl enable elasticsearch.service

使用：

/etc/init.d/elasticsearch status 查看状态

whereis elasticsearch  查看所在目录

3，安装Kibana

rpm -ivh kibana-5.3.0-x86\_64.rpm

vim /etc/kibana/kibana.yml

server.port: 5601  
    server.host: "192.168.180.94"  
    elasticsearch.url: "http://192.168.180.94:9200"

4，安装logstash

rpm -ivh logstash-5.3.0.rpm

vim /etc/logstash/conf.d/first-pipeline.conf

input \{  
    beats \{  
        \#default  
        codec => "plain"  
        client\_inactivity\_timeout => 60  
        host => "0.0.0.0"

id => "input\_beats"  
        port => "5043"  
    \}  
\}

filter \{  
    if "nginx\_access\_log" in \[tags\] \{  
        grok \{  
            \#match => \{ "message" => "%\{IPORHOST:client\_ip\} - %\{NOTSPACE:remote\_user\} \\\[%\{HTTPDATE:log\_date\}\\\] \\"(?:%\{WORD:method\} %\{NOTSPACE:request\}(?: HTTP/%\{NUMBER:version\})?|%\{DATA:rawrequest\})\\" %\{NUMBER:response\} (?:%\{NUMBER:sent\_bytes\}|-) %\{QS:referer\} %\{QS:user\_agent\} \\"%\{IPV4:x\_forwarded\_for\}\\""\}  
            match => \{ "message" => "%\{IPORHOST:client\_ip\} - %\{NOTSPACE:remote\_user\} \\\[%\{HTTPDATE:log\_date\}\\\] \\"(?:%\{WORD:method\} %\{NOTSPACE:request\}(?: HTTP/%\{NUMBER:version\})?|%\{DATA:rawrequest\})\\" %\{NUMBER:response\} (?:%\{NUMBER:sent\_bytes\}|-) %\{QS:referer\} %\{QS:user\_agent\}"\}  
        \}  
        geoip \{  
            source => "client\_ip"  
        \}  
    \}  
   
    if "nginx\_error\_log" in \[tags\] \{  
        grok \{  
            match => \{ "message" => "%\{DATESTAMP:log\_date\} \\\[%\{LOGLEVEL:log\_level\}\\\] (?:%\{GREEDYDATA:err\_str\})"\}  
        \}  
    \}

date \{  
        match => \[ "log\_date", "dd/MMM/yyyy:HH:mm:ss Z", "yy/MM/dd HH:mm:ss"\]  
    \}  
\}  
output \{  
    stdout \{ codec => rubydebug \}

if "nginx\_access\_log" in \[tags\] \{  
        elasticsearch \{  
            hosts => \[ "192.168.180.94:9200" \]  
            index => "logstash-nginx-access-%\{+YYYY.MM.dd\}"  
        \}  
    \}

if "nginx\_error\_log" in \[tags\] \{  
        elasticsearch \{  
            hosts => \[ "192.168.180.94:9200" \]  
            index => "logstash-nginx-error-%\{+YYYY.MM.dd\}"  
        \}  
    \}  
\}

logstash 用service启动导致不能自动创建索引，所以用nohup的方式启动：

nohup /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/first-pipeline.conf &

重新加载文件的操作：

systemctl stop filebeat.service

mv /var/lib/filebeat/registry /var/lib/filebeat/registry.bak

kill -9 $logstash\_pid

curl -XDELETE 'http://192.168.180.94:9200/logstash-\*'

systemctl start filebeat.service

nohup /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/first-pipeline.conf &

5，部署方案：

方案一、（Nginx to syslog + logstash(udp)）-> es集群+kibana

方案二、(Nginx to file + featbeat(file))-> logstash + es集群+kibana