记录一次K8S的容器压测监控

忘是亡心i 2022-12-15 14:09 129阅读 0赞

> 记录一次K8S的容器压测

##### 基本情况 #####

<table> 
 <thead> 
  <tr> 
   <th>服务名称</th> 
   <th>POD数量</th> 
   <th>POD规格</th> 
   <th>压测并发数</th> 
   <th>压测时间</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>用户中心登录服务</td> 
   <td>1个POD</td> 
   <td>1C2G</td> 
   <td>30个用户登录</td> 
   <td>2h</td> 
  </tr> 
 </tbody> 
</table>

##### GC情况 #####

<table> 
 <thead> 
  <tr> 
   <th>YGC次数</th> 
   <th>YGC/毫秒</th> 
   <th>FGC次数</th> 
   <th>FGCT/毫秒</th> 
   <th>GCT/毫秒</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>6849</td> 
   <td>209.355</td> 
   <td>12</td> 
   <td>3.190</td> 
   <td>212.545</td> 
  </tr> 
 </tbody> 
</table>

--------------------

总结：FGC次数在2小时中只进行了12次，耗时3.190毫秒；YGC总共6849次，耗时209.355毫秒；

##### 监控POD内存使用情况 #####

PromQL：

topk(5,round(sum by(name, id, job, node) (container_memory_working_set_bytes{ image!="",name=~".*54p5h.*"}) / sum by(name, id, job, node) (container_spec_memory_limit_bytes{ image!="",name=~".*54p5h.*"}) * 100 != +Inf) )

查询地址：http://10.152.17.11/graph  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxNTU1OTUx_size_16_color_FFFFFF_t_70_pic_center]

--------------------

总结：压测的过程中内存一直持续上升，最后稳定在90%左右的利用率上；

--------------------

##### 监控POD的CPU使用情况 #####

PromQL：

topk(15,sum by(name, namespace, job) (rate(container_cpu_usage_seconds_total{ image!="",name=~".*54p5h.*"}[10m])) / (sum by(name, namespace, job) (container_spec_cpu_quota{ image!="",name=~".*54p5h.*"} / 100000)))

![查询地址：http://10.152.17.11/graph\[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-hzKrzo0a-1602921155798)(http://10.152.160.36/server/index.php?s=/api/attachment/visitFile/sign/74d1e0c9d977940b0d5074405e5d30e9&showdoc=.jpg)\])][http_10.152.17.11_graph_img-hzKrzo0a-1602921155798_http_10.152.160.36_server_index.php_s_api_attachment_visitFile_sign_74d1e0c9d977940b0d5074405e5d30e9_showdoc_.jpg]

--------------------

总结：容器在30个并发的压测下CUP达到瓶颈已经使用100%;此时监控容器生命周期中度过的受限的cpu周期总数已达到37510次，此压测下CPU达到瓶颈；

--------------------

sum(container_cpu_cfs_throttled_periods_total{ image!="",name=~".*54p5h.*",container_label_license=""}) by (name)

*  container生命周期中度过的受限的cpu周期总数

PromQL：  
查询地址：http://10.152.17.11/graph  
![\[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-wsxj0yKb-1602921155799)(http://10.152.160.36/server/index.php?s=/api/attachment/visitFile/sign/749f03ca5a77c7501e45526230474862&showdoc=.jpg)\]][img-wsxj0yKb-1602921155799_http_10.152.160.36_server_index.php_s_api_attachment_visitFile_sign_749f03ca5a77c7501e45526230474862_showdoc_.jpg]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxNTU1OTUx_size_16_color_FFFFFF_t_70_pic_center]: /images/20221123/d716de05fd684c45b9a9214c019235b3.png
[http_10.152.17.11_graph_img-hzKrzo0a-1602921155798_http_10.152.160.36_server_index.php_s_api_attachment_visitFile_sign_74d1e0c9d977940b0d5074405e5d30e9_showdoc_.jpg]: /images/20221123/9b9498ae704445ce9e1a636b6d8c4dfc.png
[img-wsxj0yKb-1602921155799_http_10.152.160.36_server_index.php_s_api_attachment_visitFile_sign_749f03ca5a77c7501e45526230474862_showdoc_.jpg]: /images/20221123/fadcf0efeab349158a9e4a6d133547d2.png