Logstash + ElasticSearch笔记

青旅半醒 2023-05-30 08:48 38阅读 0赞

#### LogStash ####

1.具有实时渠道能力的数据引擎，包含输入、过滤、输出模块，一般在过滤模块中做日志格式的解析工作（日志收集，过滤。存储供搜索）

LogStash的配置文件： 检测一个文件的变化，并采集变化（增加）的数据  
1、在cd /apps/logStash…下  
新建：vi filetoes.conf

input {  
    	file{ 
            path => "/root/apps/logstash-5.6.16/testFile.txt"
        }
     }
    output { 
      elasticsearch {  hosts => ["hdp-2:9200"] }
      stdout {  codec => rubydebug }
    }

1.  \[root@hdp-2 logstash-5.6.16\]\# vi testFile.txt

[root@hdp-2 logstash-5.6.16]# vi testFile.txt
    hello you

3 \[root@hdp-2 logstash-5.6.16\]\# bin/logstash -f filetoes.conf

4、在kibanna查看  
GET \_search  
\{  
“took”: 514,  
“timed\_out”: false,  
“\_shards”: \{  
“total”: 20,  
“successful”: 20,  
“skipped”: 0,  
“failed”: 0  
\},  
“hits”: \{  
“total”: 3,  
“max\_score”: 1,  
“hits”: \[  
\{  
“\_index”: “logstash-2019.11.08”,  
“\_type”: “logs”,  
“\_id”: “B3TsSW4B5eFe6rqLqTa4”,  
“\_score”: 1,  
“\_source”: \{  
“@version”: “1”,  
“host”: “hdp-2”,  
“path”: “/root/apps/logstash-5.6.16/testFile.txt”,  
“@timestamp”: “2019-11-08T07:31:16.359Z”,  
“message”: “hello me”  
\}  
\},  
\{  
“\_index”: “blog”,  
“\_type”: “article”,  
“\_id”: “1”,  
“\_score”: 1,  
“\_source”: \{  
“title”: “test title”  
\}  
\},  
\{  
“\_index”: “search2”,  
“\_type”: “article2”,  
“\_id”: “1”,  
“\_score”: 1,  
“\_source”: \{  
“title”: “Test1”,  
“tags”: “test”,  
“summary”: “test”,  
“content”: “test1 content”,  
“pubtime”: “0”  
\}  
\}  
\]  
\}  
\}