一元线性回归方程的参数估计

  这篇文章详细推导了一元线性回归方程的参数解,供新手朋友参考。
  假定一元线性回归方程的具体形式为
y = a + b x (1) y=a+bx \tag{1} y=a+bx(1)
现在,为确定参数 a , b a,b a,b进行了 n n n次观测,观测结果为:
i 1 2 3 ⋯ n x x 1 x 2 x 3 ⋯ x n y y 1 y 2 y 3 ⋯ y n \begin{array}{c|ccccc} i & \text{1} & \text{2} & \text{3} & \cdots & \text{n} \\ \hline x & x_1 & x_2 & x_3 & \cdots & x_n\\ y & y_1 & y_2 & y_3 & \cdots & y_n \\ \end{array} ixy1x1y12x2y23x3y3nxnyn
  参数估计即从这 n n n组数据中解出 a , b a,b a,b。由于观测不可避免的带有误差(观测仪器、人为或环境因素引起),故 n n n组方程
{ y 1 = a + b x 1 y 2 = a + b x 2 ⋮ y n = a + b x n (2) \left\{ \begin{array}{c} y_1=a+bx_1 \\ y_2=a+bx_2 \\ \vdots \\ y_n=a+bx_n \\ \end{array} \right. \tag{2} y1=a+bx1y2=a+bx2yn=a+bxn(2)
不相容(为矛盾方程组)。为消除矛盾并确定 a , b a,b a,b的最佳估值,可采用最小二乘法来求解,目标函数为
Q = ∑ i = 1 n ( y i − a − b x i ) 2 = m i n (3) Q=\sum_{i=1}^n \left ( y_i-a-bx_i \right ) ^2 = min \tag{3} Q=i=1n(yiabxi)2=min(3)
由于 Q Q Q是关于 a , b a,b a,b的凸函数(《南瓜书》),根据凸函数极值特性,可知在 ∂ Q ∂ a = 0 \frac{ \partial Q}{\partial a}=0 aQ=0 ∂ Q ∂ b = 0 \frac{ \partial Q}{\partial b}=0 bQ=0对应的 a , b a,b a,b处取得极小值(最小值)。
   Q Q Q关于 a , b a,b a,b的偏导数如下
∂ Q ∂ a = ∑ i = 1 n 2 ( y i − a − b x i ) ⋅ ( − 1 ) = 2 ∑ i = 1 n ( a + b x i − y i ) (4) \frac{\partial Q}{\partial a}=\sum_{i=1}^n 2 \left (y_i-a-bx_i \right )\cdot(-1) =2 \sum_{i=1}^n \left (a+bx_i-y_i \right ) \tag{4} aQ=i=1n2(yiabxi)(1)=2i=1n(a+bxiyi)(4)
∂ Q ∂ b = ∑ i = 1 n 2 ( y i − a − b x i ) ⋅ ( − x i ) = 2 ∑ i = 1 n x i ( a + b x i − y i ) (5) \frac{\partial Q}{\partial b}=\sum_{i=1}^n 2 \left (y_i-a-bx_i \right )\cdot(-x_i) =2 \sum_{i=1}^n x_i \left (a+bx_i-y_i \right ) \tag{5} bQ=i=1n2(yiabxi)(xi)=2i=1nxi(a+bxiyi)(5)
当令 ( 4 ) = 0 (4)=0 (4)=0可得:
∑ i = 1 n ( a + b x i − y i ) = 0       n a + b ∑ i = 1 n x i − ∑ i = 1 n y i = 0       a = y ˉ − b x ˉ (6) \sum_{i=1}^n \left( a+bx_i-y_i \right)=0 \implies na+b\sum_{i=1}^nx_i- \sum_{i=1}^n y_i=0 \implies a=\bar{y}-b\bar{x} \tag{6} i=1n(a+bxiyi)=0na+bi=1nxii=1nyi=0a=yˉbxˉ(6)
( 5 ) = 0 (5)=0 (5)=0并代入式 ( 6 ) (6) (6)可得:
∑ i = 1 n x i ( a + b x i − y i ) = 0       a ∑ i = 1 n x i + b ∑ i = 1 n x i 2 − ∑ i = 1 n x i y i = 0       b = ∑ i = 1 n ( x i y i − y ˉ x i ) ∑ i = 1 n ( x i 2 − x ˉ x i ) (7) \sum_{i=1}^nx_i \left (a+bx_i-y_i \right )=0 \implies a\sum_{i=1}^n x_i +b\sum_{i=1}^n x_i^2 – \sum_{i=1}^n x_iy_i =0 \implies b=\frac{\sum_{i=1}^n \left(x_iy_i- \bar{y}x_i \right)}{\sum_{i=1}^n \left(x_i^2-\bar{x}x_i \right)} \tag{7} i=1nxi(a+bxiyi)=0ai=1nxi+bi=1nxi2i=1nxiyi=0b=i=1n(xi2xˉxi)i=1n(xiyiyˉxi)(7)
再顾及
∑ i = 1 n ( x i − x ˉ ) ( y i − y ˉ ) = ∑ i = 1 n ( x i y i − y ˉ x i ) a n d ∑ i = 1 n ( x i − x ˉ ) 2 = ∑ i = 1 n ( x i 2 − x ˉ x i ) \sum_{i=1}^n \left( x_i-\bar{x} \right) \left( y_i-\bar{y} \right)=\sum_{i=1}^n \left(x_iy_i- \bar{y}x_i \right) and \sum_{i=1}^n \left( x_i-\bar{x} \right)^2 =\sum_{i=1}^n \left( x_i^2-\bar{x}x_i \right) i=1n(xixˉ)(yiyˉ)=i=1n(xiyiyˉxi)andi=1n(xixˉ)2=i=1n(xi2xˉxi)
则一元线性回归方程的参数解为:
b = ∑ i = 1 n ( x i − x ˉ ) ( y i − y ˉ ) ∑ i = 1 n ( x i − x ˉ ) 2 (8) b=\frac{\sum_{i=1}^n \left( x_i-\bar{x} \right) \left( y_i-\bar{y} \right)}{\sum_{i=1}^n \left( x_i-\bar{x} \right)^2} \tag{8} b=i=1n(xixˉ)2i=1n(xixˉ)(yiyˉ)(8)
a = y ˉ − b x ˉ (9) a=\bar{y}-b\bar{x} \tag{9} a=yˉbxˉ(9)
  以上。

    原文作者:C_xxy
    原文地址: https://blog.csdn.net/C_xxy/article/details/119284389
    本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系博主进行删除。
点赞