SpringBoot整合Logstash,实现日志统计

红太狼 2024-02-19 21:38 41阅读 0赞

## SpringBoot整合Logstash ##

### 一. ELK简介 ###

ELK是Elastic公司的三个组件,他们三个配合实现日志收集.

*  ElasticSearch:用于存储日志信息;
 *  Logstash:用于收集、处理和转发日志信息;
 *  Kibana:提供可视化的Web界面.

### 二. ELK环境安装 ###

有关ELK安装请参考我之前的教程.

*  Windows下Logstash安装:  
    [https://blog.csdn.net/syc000666/article/details/98527232][https_blog.csdn.net_syc000666_article_details_98527232]
 *  Windows下ElasticSearch安装:  
    [https://blog.csdn.net/syc000666/article/details/94664999][https_blog.csdn.net_syc000666_article_details_94664999]
 *  Windows下Kibana安装:  
    [https://blog.csdn.net/syc000666/article/details/94665625][https_blog.csdn.net_syc000666_article_details_94665625]

### 三. 配置Logstash ###

#### 1. 创建Logstash.conf配置文件 ####

进入到logstash安装目录下,我的目录为E:\\ELK\\logstash-5.6.8\\bin目录.  
![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70][]

**在该目录下创建一个Logstash.conf配置文件,文件内容如下:**

input {
      tcp {
        mode => "server"
        host => "0.0.0.0"
        port => 4560
        codec => json_lines
      }
    }
    output {
      elasticsearch {
        hosts => "localhost:9200"
        index => "springboot-logstash-%{+YYYY.MM.dd}"
      }
    }

#### 2. 启动logstash ####

![20190806221529163.png][]

logstash -f logstash.conf

效果如下:  
![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 1][]

### 四. SpringBoot整合Logstash ###

#### 1. 添加依赖包 ####

<dependencies>
        
        <dependency>
            <groupId>net.logstash.logback</groupId>
            <artifactId>logstash-logback-encoder</artifactId>
            <version>5.3</version>
        </dependency>
    
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
    
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.16.22</version>
        </dependency>
    </dependencies>

#### 2. 创建logback-spring.xml文件 ####

新建一个logback-spring.xml文件,配置logback日志信息,注意这里配置的destination属性,输出的要和logstash配置的对应上,不然收集不上,内容如下:

<?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE configuration>
    <configuration>
        <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
        <include resource="org/springframework/boot/logging/logback/base.xml"/>
    
        <springProperty scope="context" name="appName" source="spring.application.name"/>
    
        
        <property name="LOG_FILE" value="${BUILD_FOLDER:-build}/${appName}"/>
    
        
        <property name="CONSOLE_LOG_PATTERN" value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
    
        
        <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
            <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
                <level>INFO</level>
            </filter>
            
            <encoder>
                <pattern>${CONSOLE_LOG_PATTERN}</pattern>
                <charset>utf8</charset>
            </encoder>
        </appender>
    
        
        <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
            <destination>127.0.0.1:4560</destination>
            
            <encoder charset="UTF-8" class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
                <providers>
                    <timestamp>
                        <timeZone>UTC</timeZone>
                    </timestamp>
                    <pattern>
                        <pattern>
                            {
                            "logLevel": "%level",
                            "serviceName": "${springAppName:-}",
                            "pid": "${PID:-}",
                            "thread": "%thread",
                            "class": "%logger{40}",
                            "rest": "%message"
                            }
                        </pattern>
                    </pattern>
                </providers>
            </encoder>
            
        </appender>
    
        <root level="INFO">
            <appender-ref ref="LOGSTASH"/>
            <appender-ref ref="CONSOLE"/>
        </root>
    
    </configuration>

#### 3. 创建Controller测试类 ####

package com.syc.boot;
    
    import lombok.extern.slf4j.Slf4j;
    import org.springframework.web.bind.annotation.GetMapping;
    import org.springframework.web.bind.annotation.RestController;
    
    @Slf4j
    @RestController
    public class LogController {
    
        @GetMapping("/test")
        public String showLog(){
            log.warn("日志测试,啦啦啦啦,lalala");
    
            return "ok";
        }
    
    }

#### 4. 创建入口类 ####

package com.syc.boot;
    
    import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.SpringBootApplication;
    
    @SpringBootApplication
    public class LogApplication {
    
        public static void main(String[] args){
            SpringApplication.run(LogApplication.class,args);
        }
    }

#### 5. 完整项目结构 ####

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 2][]

#### 6. 测试 ####

#### 6.1 在浏览器执行test请求进行测试 ####

![20190806221611574.png][]

#### 6.2 kibana中查看索引 ####

打开kibana管理页面,添加刚刚创建的索引,如图所示:  
![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 3][]  
接下来在浏览器多次访问刚刚在项目中输出日志的方法,查询控制台,如下所示.  
![20190806221637585.png][]  
然后在进入kibana查看,不光是日志内容,还有自定义的属性也显示出来了.  
![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 4][]

至此,我们在SpringBoot中整合ELK实现了日志收集.

[https_blog.csdn.net_syc000666_article_details_98527232]: https://blog.csdn.net/syc000666/article/details/98527232
[https_blog.csdn.net_syc000666_article_details_94664999]: https://blog.csdn.net/syc000666/article/details/94664999
[https_blog.csdn.net_syc000666_article_details_94665625]: https://blog.csdn.net/syc000666/article/details/94665625
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/807064e4ff26420588e3a0d468a694e1.png
[20190806221529163.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/4af6a7bb2e5f4f94adff013ca04f95dc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 1]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/9ed739828e2a4c2a8719dc0b3d367f23.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 2]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/32f658deb40f4399b4a11455fc5cc7d7.png
[20190806221611574.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/ba3c4e8544da4ba88fea3beb3dc98705.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 3]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/1b4106e10408424c81ec14f17ce25d84.png
[20190806221637585.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/9e6727a0256e476b9fac0f87e4c1cca6.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3N5YzAwMDY2Ng_size_16_color_FFFFFF_t_70 4]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/f01c1fe5a10b4e7581925f873f01d8c1.png