平方损失函数

曾经终败给现在 2024-04-17 21:34 96阅读 0赞

### 一、简介 ###

**损失函数**是一个非负实数函数，**用来量化模型预测和真实标签之间的差异。**

其中，**平方损失函数**经常用在**预测标签y为实数值**的任务中，一般**不适用于分类问题**。

平方损失函数公式，![y][]为真实值，![\\bar\{y\}][bar_y]为预测值：

![J(\\theta)=\\frac\{1\}\{2\}(y-\\bar\{y\})^2][J_theta_frac_1_2_y-_bar_y_2] ![(1)][1]

### 二、为什么回归问题中损失函数可以用平方形式？（平方损失函数的由来） ###

**基础准备：**

*  正态分布 ![X\\sim N(\\mu ,\\sigma ^\{2\})][X_sim N_mu _sigma _2] ，连续型随机变量X的概率密度为：  
    ![f(x)=\\frac\{1\}\{\\sqrt\{2\\pi \\sigma \}\} e^\{-\\frac\{(x-\\mu )^\{2\}\}\{2 \\sigma^\{2\}\}\}][f_x_frac_1_sqrt_2_pi _sigma _ e_-_frac_x-_mu _2_2 _sigma_2] ![(2)][2]
 *  设总体X的概率密度为![f(x,\\theta )][f_x_theta]，其中![\\theta][theta]为未知参数，![(x\_\{1\},x\_\{2\},...,x\_\{n\})][x_1_x_2_..._x_n]是一次试验中所获得的样本观察值，则似然函数为  
    ![L(\\theta ,x\_\{1\},x\_\{2\},\\cdot\\cdot\\cdot ,x\_\{n\})=\\prod\_\{n\}^\{j=1\}f(x\_\{j\},\\theta )][L_theta _x_1_x_2_cdot_cdot_cdot _x_n_prod_n_j_1_f_x_j_theta] ![(3)][3]

**证明：**

设![y][]为真实值，![\\bar\{y\}][bar_y]为预测值，x为输入，![\\epsilon][epsilon]为误差

则：

![\\left\\\{\\begin\{matrix\} y^\{(i)\}-\\bar\{y^\{(i)\}\}=\\epsilon ^\{(i)\}\\\\ y^\{(i)\} = \\theta^Tx^\{(i)\} \\end\{matrix\}\\right.][left_begin_matrix_ y_i_-_bar_y_i_epsilon _i_ y_i_ _ _theta_Tx_i_ _end_matrix_right.]

整理可得：  
![y^\{(i)\} = \\theta^Tx^\{(i)\} + \\epsilon^\{(i)\}][y_i_ _ _theta_Tx_i_ _ _epsilon_i]

假设![\\epsilon^\{(i)\} \\thicksim \\mathcal\{N\}(0,\\sigma^2)][epsilon_i_ _thicksim _mathcal_N_0_sigma_2]，分布是均值为0，方差为![\\sigma^2][sigma_2]的正态分布，那么根据公式(2)可得，的概率密度为：

![f(\\epsilon^\{(i)\})=\\frac\{1\}\{\\sqrt\{2\\pi\}\\sigma\}\\mathrm\{exp\}\\biggl(-\\frac\{(\\epsilon^\{(i)\})^2\}\{2\\sigma^2\}\\biggl)][f_epsilon_i_frac_1_sqrt_2_pi_sigma_mathrm_exp_biggl_-_frac_epsilon_i_2_2_sigma_2_biggl]

整理后等价于：

![f(y^\{(i)\}|x^\{(i)\};\\theta)=\\frac\{1\}\{\\sqrt\{2\\pi\}\\sigma\}\\mathrm\{exp\}\\biggl(-\\frac\{(y^\{(i)\}-\\theta^Tx^\{(i)\})^2\}\{2\\sigma^2\}\\bigg)][f_y_i_x_i_theta_frac_1_sqrt_2_pi_sigma_mathrm_exp_biggl_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_bigg]

根据似然函数的定义得:

![\\begin\{align\*\} L(\\theta) &= \\prod\_\{i=1\}^nf(y^\{(i)\}|x^\{(i)\};\\theta) \\\\ &= \\prod\_\{i=1\}^n\\frac\{1\}\{\\sqrt\{2\\pi\}\\sigma\}\\mathrm\{exp\}\\biggl(-\\frac\{(y^\{(i)\}-\\theta^Tx^\{(i)\})^2\}\{2\\sigma^2\}\\biggr) \\end\{align\*\}][begin_align_ L_theta_ _ _prod_i_1_nf_y_i_x_i_theta_ _ _ _prod_i_1_n_frac_1_sqrt_2_pi_sigma_mathrm_exp_biggl_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_biggr_ _end_align]

两边同取log，得：

![\\begin\{align\*\} \\mathrm\{log\}L(\\theta) &=\\mathrm\{log\}\\prod\_\{i=1\}^n\\frac\{1\}\{\\sqrt\{2\\pi\}\\sigma\}\\mathrm\{exp\}\\bigg(-\\frac\{(y^\{(i)\}-\\theta^Tx^\{(i)\})^2\}\{2\\sigma^2\}\\bigg) \\\\&= \\sum\_\{i=1\}^n\\mathrm\{log\}\\frac\{1\}\{\\sqrt\{2\\pi\}\\sigma\}\\mathrm\{exp\}\\bigg(-\\frac\{(y^\{(i)\}-\\theta^Tx^\{(i)\})^2\}\{2\\sigma^2\}\\bigg) \\\\ &=-n\\mathrm\{log\}\{\\sqrt\{2\\pi\}\\sigma\}-\\frac\{1\}\{\\sigma^2\}\\cdot\\frac\{1\}\{2\}\\sum\_\{i=1\}^n(y^\{(i)\}-\\theta^Tx^\{(i)\})^2\\end\{align\*\}][begin_align_ _mathrm_log_L_theta_ _mathrm_log_prod_i_1_n_frac_1_sqrt_2_pi_sigma_mathrm_exp_bigg_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_bigg_ _ _sum_i_1_n_mathrm_log_frac_1_sqrt_2_pi_sigma_mathrm_exp_bigg_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_bigg_ _ _-n_mathrm_log_sqrt_2_pi_sigma_-_frac_1_sigma_2_cdot_frac_1_2_sum_i_1_n_y_i_-_theta_Tx_i_2_end_align]

为了让似然函数尽可能打，需要让 ![\\frac\{1\}\{2\}\\sum\_\{i=1\}^n(y^\{(i)\}-\\theta^Tx^\{(i)\})^2][frac_1_2_sum_i_1_n_y_i_-_theta_Tx_i_2]尽可能小，（其余部分都为定值）；

也就等价于![\\frac\{1\}\{2\}\\cdot(y-\\bar\{y\})^2][frac_1_2_cdot_y-_bar_y_2]最小化。

我们把这个函数称为损失函数![J(\\theta )][J_theta]，在训练模型时，通过训练来找到使![J(\\theta )][J_theta]最小的![\\theta][theta]值。

[y]: https://private.codecogs.com/gif.latex?y
[bar_y]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9wcml2YXRlLmNvZGVjb2dzLmNvbS9naWYubGF0ZXg_JTVDYmFyJTdCeSU3RA?x-oss-process=image/format,png
[J_theta_frac_1_2_y-_bar_y_2]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9wcml2YXRlLmNvZGVjb2dzLmNvbS9naWYubGF0ZXg_SiUyOCU1Q3RoZXRhJTI5JTNEJTVDZnJhYyU3QjElN0QlN0IyJTdEJTI4eS0lNUNiYXIlN0J5JTdEJTI5JTVFMg?x-oss-process=image/format,png
[1]: https://private.codecogs.com/gif.latex?%281%29
[X_sim N_mu _sigma _2]: https://private.codecogs.com/gif.latex?%5Cinline%20X%5Csim%20N%28%5Cmu%20%2C%5Csigma%20%5E%7B2%7D%29
[f_x_frac_1_sqrt_2_pi _sigma _ e_-_frac_x-_mu _2_2 _sigma_2]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20f%28x%29%3D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%20%5Csigma%20%7D%7D%20e%5E%7B-%5Cfrac%7B%28x-%5Cmu%20%29%5E%7B2%7D%7D%7B2%20%5Csigma%5E%7B2%7D%7D%7D
[2]: https://private.codecogs.com/gif.latex?%282%29
[f_x_theta]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20f%28x%2C%5Ctheta%20%29
[theta]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Ctheta
[x_1_x_2_..._x_n]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%28x_%7B1%7D%2Cx_%7B2%7D%2C...%2Cx_%7Bn%7D%29
[L_theta _x_1_x_2_cdot_cdot_cdot _x_n_prod_n_j_1_f_x_j_theta]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20L%28%5Ctheta%20%2Cx_%7B1%7D%2Cx_%7B2%7D%2C%5Ccdot%5Ccdot%5Ccdot%20%2Cx_%7Bn%7D%29%3D%5Cprod_%7Bn%7D%5E%7Bj%3D1%7Df%28x_%7Bj%7D%2C%5Ctheta%20%29
[3]: https://private.codecogs.com/gif.latex?%283%29
[epsilon]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Cepsilon
[left_begin_matrix_ y_i_-_bar_y_i_epsilon _i_ y_i_ _ _theta_Tx_i_ _end_matrix_right.]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Cleft%5C%7B%5Cbegin%7Bmatrix%7D%20y%5E%7B%28i%29%7D-%5Cbar%7By%5E%7B%28i%29%7D%7D%3D%5Cepsilon%20%5E%7B%28i%29%7D%5C%5C%20y%5E%7B%28i%29%7D%20%3D%20%5Ctheta%5ETx%5E%7B%28i%29%7D%20%5Cend%7Bmatrix%7D%5Cright.
[y_i_ _ _theta_Tx_i_ _ _epsilon_i]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20y%5E%7B%28i%29%7D%20%3D%20%5Ctheta%5ETx%5E%7B%28i%29%7D%20&plus;%20%5Cepsilon%5E%7B%28i%29%7D
[epsilon_i_ _thicksim _mathcal_N_0_sigma_2]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Cepsilon%5E%7B%28i%29%7D%20%5Cthicksim%20%5Cmathcal%7BN%7D%280%2C%5Csigma%5E2%29
[sigma_2]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Csigma%5E2
[f_epsilon_i_frac_1_sqrt_2_pi_sigma_mathrm_exp_biggl_-_frac_epsilon_i_2_2_sigma_2_biggl]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20f%28%5Cepsilon%5E%7B%28i%29%7D%29%3D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%7D%5Csigma%7D%5Cmathrm%7Bexp%7D%5Cbiggl%28-%5Cfrac%7B%28%5Cepsilon%5E%7B%28i%29%7D%29%5E2%7D%7B2%5Csigma%5E2%7D%5Cbiggl%29
[f_y_i_x_i_theta_frac_1_sqrt_2_pi_sigma_mathrm_exp_biggl_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_bigg]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20f%28y%5E%7B%28i%29%7D%7Cx%5E%7B%28i%29%7D%3B%5Ctheta%29%3D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%7D%5Csigma%7D%5Cmathrm%7Bexp%7D%5Cbiggl%28-%5Cfrac%7B%28y%5E%7B%28i%29%7D-%5Ctheta%5ETx%5E%7B%28i%29%7D%29%5E2%7D%7B2%5Csigma%5E2%7D%5Cbigg%29
[begin_align_ L_theta_ _ _prod_i_1_nf_y_i_x_i_theta_ _ _ _prod_i_1_n_frac_1_sqrt_2_pi_sigma_mathrm_exp_biggl_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_biggr_ _end_align]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Cbegin%7Balign*%7D%20L%28%5Ctheta%29%20%26%3D%20%5Cprod_%7Bi%3D1%7D%5Enf%28y%5E%7B%28i%29%7D%7Cx%5E%7B%28i%29%7D%3B%5Ctheta%29%20%5C%5C%20%26%3D%20%5Cprod_%7Bi%3D1%7D%5En%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%7D%5Csigma%7D%5Cmathrm%7Bexp%7D%5Cbiggl%28-%5Cfrac%7B%28y%5E%7B%28i%29%7D-%5Ctheta%5ETx%5E%7B%28i%29%7D%29%5E2%7D%7B2%5Csigma%5E2%7D%5Cbiggr%29%20%5Cend%7Balign*%7D
[begin_align_ _mathrm_log_L_theta_ _mathrm_log_prod_i_1_n_frac_1_sqrt_2_pi_sigma_mathrm_exp_bigg_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_bigg_ _ _sum_i_1_n_mathrm_log_frac_1_sqrt_2_pi_sigma_mathrm_exp_bigg_-_frac_y_i_-_theta_Tx_i_2_2_sigma_2_bigg_ _ _-n_mathrm_log_sqrt_2_pi_sigma_-_frac_1_sigma_2_cdot_frac_1_2_sum_i_1_n_y_i_-_theta_Tx_i_2_end_align]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Cbegin%7Balign*%7D%20%5Cmathrm%7Blog%7DL%28%5Ctheta%29%20%26%3D%5Cmathrm%7Blog%7D%5Cprod_%7Bi%3D1%7D%5En%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%7D%5Csigma%7D%5Cmathrm%7Bexp%7D%5Cbigg%28-%5Cfrac%7B%28y%5E%7B%28i%29%7D-%5Ctheta%5ETx%5E%7B%28i%29%7D%29%5E2%7D%7B2%5Csigma%5E2%7D%5Cbigg%29%20%5C%5C%26%3D%20%5Csum_%7Bi%3D1%7D%5En%5Cmathrm%7Blog%7D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%7D%5Csigma%7D%5Cmathrm%7Bexp%7D%5Cbigg%28-%5Cfrac%7B%28y%5E%7B%28i%29%7D-%5Ctheta%5ETx%5E%7B%28i%29%7D%29%5E2%7D%7B2%5Csigma%5E2%7D%5Cbigg%29%20%5C%5C%20%26%3D-n%5Cmathrm%7Blog%7D%7B%5Csqrt%7B2%5Cpi%7D%5Csigma%7D-%5Cfrac%7B1%7D%7B%5Csigma%5E2%7D%5Ccdot%5Cfrac%7B1%7D%7B2%7D%5Csum_%7Bi%3D1%7D%5En%28y%5E%7B%28i%29%7D-%5Ctheta%5ETx%5E%7B%28i%29%7D%29%5E2%5Cend%7Balign*%7D
[frac_1_2_sum_i_1_n_y_i_-_theta_Tx_i_2]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Cfrac%7B1%7D%7B2%7D%5Csum_%7Bi%3D1%7D%5En%28y%5E%7B%28i%29%7D-%5Ctheta%5ETx%5E%7B%28i%29%7D%29%5E2
[frac_1_2_cdot_y-_bar_y_2]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20%5Cfrac%7B1%7D%7B2%7D%5Ccdot%28y-%5Cbar%7By%7D%29%5E2
[J_theta]: https://private.codecogs.com/gif.latex?%5Cdpi%7B120%7D%20J%28%5Ctheta%20%29