Spring Boot Actuator监控数据可视化

旧城等待， 2023-02-23 08:26 698阅读 0赞

> ###### TIPS ######
> 
> 本文基于Spring Boot 2.1.5，理论支持Spring Boot 2.x所有版本

众所周知，Spring Boot有个子项目**SpringBoot Actuator**，它为应用提供了强大的监控能力。从Spring Boot 2.0开始，Actuator将底层改为**Micrometer**，提供了更强、更灵活的监控能力。Micrometer是一个监控门面，可以类比成监控界的 Slf4j 。

借助Micrometer，应用能够对接各种监控系统，例如：

AppOptics、Atlas、Datadog、Dynatrace、Elastic、Ganglia、Graphite、Humio、Influx、JMX、KairosDB、New Relic、Prometheus、SignalFx、Simple (in-memory)、StatsD、Wavefront

下面演示如何对接 Prometheus ，并使用 Grafana 实现数据的可视化。

##### 编码 #####

###### 编写代码 ######

*  加pom依赖

<dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
      <groupId>io.micrometer</groupId>
      <artifactId>micrometer-registry-prometheus</artifactId>
    </dependency>

这里，我们为应用引入了 **micrometer-registry-prometheus** ，事实上，你想对接上文列表中的哪款监控系统，就写啥。例如想对接 Influx ，则需添加依赖 **micrometer-registry-influx** 。

*  写配置

server:
      port: 8080
    spring:
      application:
        name: prometheus-test
    management:
      endpoints:
        web:
          exposure:
            include: 'prometheus'
      metrics:
        tags:
          application: ${ spring.application.name}

如配置所示，指定应用名为 **prometheus-test** ，并将 Actuator 的 **/actuator/prometheus** 端点暴露出来； **management.metrics.tags.application=prometheus-test** 作用是为指标设置一个名为**application=“prometheus-test”** 的Tag，Tag是Prometheus提供的一种能力，从而实现更加灵活的筛选。

##### 测试 #####

*  启动应用
 *  访问 **http://localhost:8080/actuator/prometheus** 可获得类似如下的结果：

# HELP jvm_memory_used_bytes The amount of used memory
    # TYPE jvm_memory_used_bytes gauge
    jvm_memory_used_bytes{application="prometheus-test",area="heap",id="PS Old Gen",} 2.1193976E7
    jvm_memory_used_bytes{application="prometheus-test",area="nonheap",id="Metaspace",} 3.8791688E7
    jvm_memory_used_bytes{application="prometheus-test",area="heap",id="PS Survivor Space",} 0.0
    jvm_memory_used_bytes{application="prometheus-test",area="nonheap",id="Compressed Class Space",} 5303976.0
    jvm_memory_used_bytes{application="prometheus-test",area="heap",id="PS Eden Space",} 8.2574816E7
    jvm_memory_used_bytes{application="prometheus-test",area="nonheap",id="Code Cache",} 8693824.0
    # HELP tomcat_global_received_bytes_total  
    # TYPE tomcat_global_received_bytes_total counter
    tomcat_global_received_bytes_total{application="prometheus-test",name="http-nio-8080",} 0.0
    # HELP jvm_threads_daemon_threads The current number of live daemon threads
    # TYPE jvm_threads_daemon_threads gauge
    jvm_threads_daemon_threads{application="prometheus-test",} 20.0
    # HELP tomcat_sessions_alive_max_seconds  
    # TYPE tomcat_sessions_alive_max_seconds gauge
    tomcat_sessions_alive_max_seconds{application="prometheus-test",} 0.0
    # HELP jvm_buffer_memory_used_bytes An estimate of the memory that the Java virtual machine is using for this buffer pool
    # TYPE jvm_buffer_memory_used_bytes gauge
    jvm_buffer_memory_used_bytes{application="prometheus-test",id="mapped",} 0.0
    jvm_buffer_memory_used_bytes{application="prometheus-test",id="direct",} 90112.0
    # HELP jvm_threads_states_threads The current number of threads having NEW state
    # TYPE jvm_threads_states_threads gauge
    jvm_threads_states_threads{application="prometheus-test",state="runnable",} 9.0
    jvm_threads_states_threads{application="prometheus-test",state="new",} 0.0
    jvm_threads_states_threads{application="prometheus-test",state="terminated",} 0.0
    jvm_threads_states_threads{application="prometheus-test",state="blocked",} 0.0
    jvm_threads_states_threads{application="prometheus-test",state="waiting",} 12.0
    jvm_threads_states_threads{application="prometheus-test",state="timed-waiting",} 3.0
    # HELP process_cpu_usage The "recent cpu usage" for the Java Virtual Machine process
    # TYPE process_cpu_usage gauge
    process_cpu_usage{application="prometheus-test",} 0.0030590633504868434
    # HELP logback_events_total Number of error level events that made it to the logs
    # TYPE logback_events_total counter
    logback_events_total{application="prometheus-test",level="info",} 7.0
    logback_events_total{application="prometheus-test",level="warn",} 0.0
    logback_events_total{application="prometheus-test",level="trace",} 0.0
    logback_events_total{application="prometheus-test",level="debug",} 0.0
    logback_events_total{application="prometheus-test",level="error",} 0.0
    # HELP tomcat_global_sent_bytes_total  
    # TYPE tomcat_global_sent_bytes_total counter
    tomcat_global_sent_bytes_total{application="prometheus-test",name="http-nio-8080",} 195356.0
    # HELP process_files_max_files The maximum file descriptor count
    # TYPE process_files_max_files gauge
    process_files_max_files{application="prometheus-test",} 10240.0
    # HELP tomcat_threads_busy_threads  
    # TYPE tomcat_threads_busy_threads gauge
    tomcat_threads_busy_threads{application="prometheus-test",name="http-nio-8080",} 1.0
    # HELP process_files_open_files The open file descriptor count
    # TYPE process_files_open_files gauge
    process_files_open_files{application="prometheus-test",} 101.0
    # HELP tomcat_sessions_active_current_sessions  
    # TYPE tomcat_sessions_active_current_sessions gauge
    tomcat_sessions_active_current_sessions{application="prometheus-test",} 0.0
    # HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution
    # TYPE jvm_classes_unloaded_classes_total counter
    jvm_classes_unloaded_classes_total{application="prometheus-test",} 2.0
    # HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use
    # TYPE jvm_memory_committed_bytes gauge
    jvm_memory_committed_bytes{application="prometheus-test",area="heap",id="PS Old Gen",} 1.5466496E8
    jvm_memory_committed_bytes{application="prometheus-test",area="nonheap",id="Metaspace",} 4.1418752E7
    jvm_memory_committed_bytes{application="prometheus-test",area="heap",id="PS Survivor Space",} 1.6252928E7
    jvm_memory_committed_bytes{application="prometheus-test",area="nonheap",id="Compressed Class Space",} 5767168.0
    jvm_memory_committed_bytes{application="prometheus-test",area="heap",id="PS Eden Space",} 1.73539328E8
    jvm_memory_committed_bytes{application="prometheus-test",area="nonheap",id="Code Cache",} 8716288.0
    # HELP http_server_requests_seconds  
    # TYPE http_server_requests_seconds summary
    http_server_requests_seconds_count{application="prometheus-test",exception="None",method="GET",outcome="SUCCESS",status="200",uri="/actuator/prometheus",} 16.0
    http_server_requests_seconds_sum{application="prometheus-test",exception="None",method="GET",outcome="SUCCESS",status="200",uri="/actuator/prometheus",} 0.326299973
    http_server_requests_seconds_count{application="prometheus-test",exception="None",method="GET",outcome="SUCCESS",status="200",uri="/**/favicon.ico",} 3.0
    http_server_requests_seconds_sum{application="prometheus-test",exception="None",method="GET",outcome="SUCCESS",status="200",uri="/**/favicon.ico",} 0.028434427
    # HELP http_server_requests_seconds_max  
    # TYPE http_server_requests_seconds_max gauge
    http_server_requests_seconds_max{application="prometheus-test",exception="None",method="GET",outcome="SUCCESS",status="200",uri="/actuator/prometheus",} 0.120627904
    http_server_requests_seconds_max{application="prometheus-test",exception="None",method="GET",outcome="SUCCESS",status="200",uri="/**/favicon.ico",} 0.015596512
    # HELP tomcat_global_error_total  
    # TYPE tomcat_global_error_total counter
    tomcat_global_error_total{application="prometheus-test",name="http-nio-8080",} 0.0
    # HELP jvm_gc_max_data_size_bytes Max size of old generation memory pool
    # TYPE jvm_gc_max_data_size_bytes gauge
    jvm_gc_max_data_size_bytes{application="prometheus-test",} 2.863661056E9
    # HELP jvm_gc_live_data_size_bytes Size of old generation memory pool after a full GC
    # TYPE jvm_gc_live_data_size_bytes gauge
    jvm_gc_live_data_size_bytes{application="prometheus-test",} 2.1193976E7
    # HELP system_load_average_1m The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time
    # TYPE system_load_average_1m gauge
    system_load_average_1m{application="prometheus-test",} 3.9423828125
    # HELP process_uptime_seconds The uptime of the Java virtual machine
    # TYPE process_uptime_seconds gauge
    process_uptime_seconds{application="prometheus-test",} 173.424
    # HELP tomcat_sessions_expired_sessions_total  
    # TYPE tomcat_sessions_expired_sessions_total counter
    tomcat_sessions_expired_sessions_total{application="prometheus-test",} 0.0
    # HELP jvm_gc_pause_seconds Time spent in GC pause
    # TYPE jvm_gc_pause_seconds summary
    jvm_gc_pause_seconds_count{action="end of minor GC",application="prometheus-test",cause="Metadata GC Threshold",} 1.0
    jvm_gc_pause_seconds_sum{action="end of minor GC",application="prometheus-test",cause="Metadata GC Threshold",} 0.012
    jvm_gc_pause_seconds_count{action="end of major GC",application="prometheus-test",cause="Metadata GC Threshold",} 1.0
    jvm_gc_pause_seconds_sum{action="end of major GC",application="prometheus-test",cause="Metadata GC Threshold",} 0.045
    # HELP jvm_gc_pause_seconds_max Time spent in GC pause
    # TYPE jvm_gc_pause_seconds_max gauge
    jvm_gc_pause_seconds_max{action="end of minor GC",application="prometheus-test",cause="Metadata GC Threshold",} 0.012
    jvm_gc_pause_seconds_max{action="end of major GC",application="prometheus-test",cause="Metadata GC Threshold",} 0.045
    # HELP jvm_gc_memory_promoted_bytes_total Count of positive increases in the size of the old generation memory pool before GC to after GC
    # TYPE jvm_gc_memory_promoted_bytes_total counter
    jvm_gc_memory_promoted_bytes_total{application="prometheus-test",} 1.06878E7
    # HELP jvm_gc_memory_allocated_bytes_total Incremented for an increase in the size of the young generation memory pool after one GC to before the next
    # TYPE jvm_gc_memory_allocated_bytes_total counter
    jvm_gc_memory_allocated_bytes_total{application="prometheus-test",} 7.8713648E7
    # HELP tomcat_global_request_seconds  
    # TYPE tomcat_global_request_seconds summary
    tomcat_global_request_seconds_count{application="prometheus-test",name="http-nio-8080",} 19.0
    tomcat_global_request_seconds_sum{application="prometheus-test",name="http-nio-8080",} 0.432
    # HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads
    # TYPE jvm_threads_live_threads gauge
    jvm_threads_live_threads{application="prometheus-test",} 24.0
    # HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool
    # TYPE jvm_buffer_count_buffers gauge
    jvm_buffer_count_buffers{application="prometheus-test",id="mapped",} 0.0
    jvm_buffer_count_buffers{application="prometheus-test",id="direct",} 11.0
    # HELP process_start_time_seconds Start time of the process since unix epoch.
    # TYPE process_start_time_seconds gauge
    process_start_time_seconds{application="prometheus-test",} 1.556017153576E9
    # HELP tomcat_threads_config_max_threads  
    # TYPE tomcat_threads_config_max_threads gauge
    tomcat_threads_config_max_threads{application="prometheus-test",name="http-nio-8080",} 200.0
    # HELP system_cpu_count The number of processors available to the Java virtual machine
    # TYPE system_cpu_count gauge
    system_cpu_count{application="prometheus-test",} 8.0
    # HELP tomcat_threads_current_threads  
    # TYPE tomcat_threads_current_threads gauge
    tomcat_threads_current_threads{application="prometheus-test",name="http-nio-8080",} 10.0
    # HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management
    # TYPE jvm_memory_max_bytes gauge
    jvm_memory_max_bytes{application="prometheus-test",area="heap",id="PS Old Gen",} 2.863661056E9
    jvm_memory_max_bytes{application="prometheus-test",area="nonheap",id="Metaspace",} -1.0
    jvm_memory_max_bytes{application="prometheus-test",area="heap",id="PS Survivor Space",} 1.6252928E7
    jvm_memory_max_bytes{application="prometheus-test",area="nonheap",id="Compressed Class Space",} 1.073741824E9
    jvm_memory_max_bytes{application="prometheus-test",area="heap",id="PS Eden Space",} 1.395654656E9
    jvm_memory_max_bytes{application="prometheus-test",area="nonheap",id="Code Cache",} 2.5165824E8
    # HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset
    # TYPE jvm_threads_peak_threads gauge
    jvm_threads_peak_threads{application="prometheus-test",} 29.0
    # HELP jvm_classes_loaded_classes The number of classes that are currently loaded in the Java virtual machine
    # TYPE jvm_classes_loaded_classes gauge
    jvm_classes_loaded_classes{application="prometheus-test",} 7936.0
    # HELP tomcat_sessions_active_max_sessions  
    # TYPE tomcat_sessions_active_max_sessions gauge
    tomcat_sessions_active_max_sessions{application="prometheus-test",} 0.0
    # HELP tomcat_global_request_max_seconds  
    # TYPE tomcat_global_request_max_seconds gauge
    tomcat_global_request_max_seconds{application="prometheus-test",name="http-nio-8080",} 0.161
    # HELP tomcat_sessions_created_sessions_total  
    # TYPE tomcat_sessions_created_sessions_total counter
    tomcat_sessions_created_sessions_total{application="prometheus-test",} 0.0
    # HELP tomcat_sessions_rejected_sessions_total  
    # TYPE tomcat_sessions_rejected_sessions_total counter
    tomcat_sessions_rejected_sessions_total{application="prometheus-test",} 0.0
    # HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool
    # TYPE jvm_buffer_total_capacity_bytes gauge
    jvm_buffer_total_capacity_bytes{application="prometheus-test",id="mapped",} 0.0
    jvm_buffer_total_capacity_bytes{application="prometheus-test",id="direct",} 90112.0
    # HELP system_cpu_usage The "recent cpu usage" for the whole system
    # TYPE system_cpu_usage gauge
    system_cpu_usage{application="prometheus-test",} 0.2523364485981308

该端点返回的数据结构是Prometheus需要使用的。对照注释，相信大家能看懂每一项的含义。例如：

# HELP jvm_memory_used_bytes The amount of used memory
    # TYPE jvm_memory_used_bytes gauge
    jvm_memory_used_bytes{application="prometheus-test",area="heap",id="PS Old Gen",} 2.1193976E7

表示：**prometheus-test** 应用堆内存中的 **PS Old Gen** 区域占用的空间是 **2.1193976E7** 字节。

##### 对接Prometheus和Grafana #####

Prometheus+Grafana的安装和使用可以参考  
[《监控体系搭建（开箱即用）：node\_expoter + promethous + grafana》][node_expoter _ promethous _ grafana]

*  Prometheus  
    修改prometheus.yml配置文件，添加以下配置

scrape_configs:
    # 任意写，建议英文，不要包含特殊字符
    - job_name: 'spring'
      # 多久采集一次数据
      scrape_interval: 15s
      # 采集时的超时时间
      scrape_timeout: 10s
      # 采集的路径是啥
      metrics_path: '/actuator/prometheus'
      # 采集服务的地址，设置成上面SpringBoot应用所在服务器的具体地址。
      static_configs:
      - targets: ['127.0.0.1:8080']

这样配置后，Prometheus服务器就会自动每隔15秒请求 http://127.0.0.1:8080/actuator/prometheus ，然后启动Prometheus。

*  Grafana

按照上文[参考文章][node_expoter _ promethous _ grafana]启动Grafana，并添加Prometheus数据源。

点击导航栏上的 + 按钮，并点击Dashboard，将会看到类似如下的界面：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70]

点击 Add Query ，即可看到类似如下的界面：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 1]  
在白框标记的位置输入指标，指标的取值详见Spring Boot应用的 **/actuator/prometheus** 端点，例如**jvm\_memory\_used\_bytes** 、**jvm\_threads\_states\_threads** 、**jvm\_threads\_live\_threads** 等，Grafana会给你较好的提示，并且支持较为复杂的计算，例如聚合、求和、平均等。如果想要绘制多个线条，可点击Add Query 按钮，如上图所示，笔者为图表绘制了两条线，分别代表daemon以及peak线程。

点击下图的按钮，并填入Title，即可设置图表标题。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 2]  
按下图步骤操作，即可为Dashboard添加新的图表。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 3]

###### Dashboard市场 ######

至此，已可用Grafana实现较为丰富的图表展示——可将关心的监控指标放置到Dashboard上，并且非常灵活！然而，这个配置的操作虽然不难，但还是挺费时间的……

那么是否有配置好的又强大、又通用、拿来即用的Dashboard呢？答案是肯定的！前往 [Grafana Lab - Dashboards][] ，输入关键词即可搜索指定Dashboard。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 4]

如上图所示，可以找到若干款以 Prometheus 作为数据源，点击进入Dashboard详情页，记住Dashboard ID。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 5]

按照上文[参考文章][node_expoter _ promethous _ grafana]内import dashboard的操作，即可直接使用。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 6]

[node_expoter _ promethous _ grafana]: https://blog.csdn.net/lt326030434/article/details/107045794
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200706154643384.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200706154653584.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20200706154753267.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20200706154803563.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0,size_16,color_FFFFFF,t_70
[Grafana Lab - Dashboards]: https://grafana.com/grafana/dashboards
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20200706155100153.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20200706155233460.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20200706155415652.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2x0MzI2MDMwNDM0,size_16,color_FFFFFF,t_70