检测损失函数比较

向右看齐 2022-02-01 14:15 289阅读 0赞

检测算法中回归部分常用到L1, L2（yolo）,Smooth L1(Faster rcnn, ssd)等损失函数，如下将对不同损失函数进行简单对比。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2h6aGoyMDA3_size_16_color_FFFFFF_t_70][]

**损失函数形式**：

![S=\\sum ^\{n\}\_\{i=1\}\\left | y\_\{i\} - f(x\_\{i\}) \\right |=\\left \\| y ^\{\\string^\} -y \\right \\|\_\{1\}][S_sum _n_i_1_left _ y_i_ - f_x_i_ _right _left _ y _string_ -y _right _1]                   L1 Loss

![S=\\sum ^\{n\}\_\{i=1\}(y\_\{i\} - f\{x\_\{i\}\})^\{2\}=\\left \\| y^\{\\string^\} -y \\right \\|\_\{2\}][S_sum _n_i_1_y_i_ - f_x_i_2_left _ y_string_ -y _right _2]                    L2 Loss

![\\left|d\\right|\_\{smooth\}=\\left\\\{\\begin\{matrix\} 0.5\*d^\{2\} \\qquad ,if \\left|d\\right| \\leq 1\\\\\\left|d\\right| -0.5 \\qquad , otherwise\\end\{matrix\}\\right.][left_d_right_smooth_left_begin_matrix_ 0.5_d_2_ _qquad _if _left_d_right_ _leq 1_left_d_right_ -0.5 _qquad _ otherwise_end_matrix_right.]          Smooth L1 Loss

**特点**：

<table style="width:500px;"> 
 <tbody> 
  <tr> 
   <td>L2 Loss</td> 
   <td>L1 Loss</td> 
  </tr> 
  <tr> 
   <td>不太鲁棒</td> 
   <td>鲁棒</td> 
  </tr> 
  <tr> 
   <td>稳定解</td> 
   <td>不稳定解</td> 
  </tr> 
  <tr> 
   <td>总是唯一解&nbsp; &nbsp; &nbsp;&nbsp;</td> 
   <td>可能存在多个解</td> 
  </tr> 
 </tbody> 
</table>

ps: 鲁棒性是指受离群点的影响，由于计算方式不同，导致L2对离群样本更敏感，故鲁棒性差。

稳定性指对有很多解的数据，当某个数据发生微小移动时，L1的回归线的会和之前的线形成夹角，而L2的回归线仅会发生微小移动。

参考文献：

1.  [Differences between L1 and L2 as Loss Function and Regularization][]，[中文翻译地址][Link 1]
2.  [About loss functions][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2h6aGoyMDA3_size_16_color_FFFFFF_t_70]: /images/20220201/6d48e41722c149a9942f1458f587acce.png
[S_sum _n_i_1_left _ y_i_ - f_x_i_ _right _left _ y _string_ -y _right _1]: https://private.codecogs.com/gif.latex?S%3D%5Csum%20%5E%7Bn%7D_%7Bi%3D1%7D%5Cleft%20%7C%20y_%7Bi%7D%20-%20f%28x_%7Bi%7D%29%20%5Cright%20%7C%3D%5Cleft%20%5C%7C%20y%20%5E%7B%5Cstring%5E%7D%20-y%20%5Cright%20%5C%7C_%7B1%7D
[S_sum _n_i_1_y_i_ - f_x_i_2_left _ y_string_ -y _right _2]: https://private.codecogs.com/gif.latex?S%3D%5Csum%20%5E%7Bn%7D_%7Bi%3D1%7D%28y_%7Bi%7D%20-%20f%7Bx_%7Bi%7D%7D%29%5E%7B2%7D%3D%5Cleft%20%5C%7C%20y%5E%7B%5Cstring%5E%7D%20-y%20%5Cright%20%5C%7C_%7B2%7D
[left_d_right_smooth_left_begin_matrix_ 0.5_d_2_ _qquad _if _left_d_right_ _leq 1_left_d_right_ -0.5 _qquad _ otherwise_end_matrix_right.]: https://private.codecogs.com/gif.latex?%5Cleft%7Cd%5Cright%7C_%7Bsmooth%7D%3D%5Cleft%5C%7B%5Cbegin%7Bmatrix%7D%200.5*d%5E%7B2%7D%20%5Cqquad%20%2Cif%20%5Cleft%7Cd%5Cright%7C%20%5Cleq%201%5C%5C%5Cleft%7Cd%5Cright%7C%20-0.5%20%5Cqquad%20%2C%20otherwise%5Cend%7Bmatrix%7D%5Cright.
[Differences between L1 and L2 as Loss Function and Regularization]: http://www.chioka.in/differences-between-l1-and-l2-as-loss-function-and-regularization/
[Link 1]: https://www.cnblogs.com/jclian91/p/9824310.html
[About loss functions]: http://christopher5106.github.io/deep/learning/2016/09/16/about-loss-functions-multinomial-logistic-logarithm-cross-entropy-square-errors-euclidian-absolute-frobenius-hinge.html