線性迴歸學習筆記
阿新 • • 發佈：2020-12-23
線性迴歸學習筆記
推導過程
f
(
x
i
)
=
w
x
i
+
b
f(x_i)=wx_i+b
f(xi)=wxi+b
目標函式為
m
i
n
∑
i
=
1
m
(
f
(
x
i
)
−
y
i
)
2
=
m
i
n
∑
i
=
1
m
(
y
i
−
w
x
i
−
b
)
2
min \sum_{i=1}^{m}(f(x_i)-y_i)^2=min \sum_{i=1}^{m}(y_i-wx_i-b)^2
mini=1∑m(f(xi)−yi)2=mini=1∑m(yi−wxi−b)2
分別對
w
w
w和
b
b
b求偏導，
∂ E ( w , b ) ∂ b = 2 ( ∑ i = 1 m ( w x i − y i ) + m b ) \frac {\partial E_{(w,b)}}{\partial b}=2\bigl( \sum_{i=1}^{m}(wx_i-y_i) + mb \bigr)
令兩式為0，得
∑ i = 1 m ( x i y i − w x i 2 ) = b ∑ i = 1 m x i \sum_{i=1}^{m}(x_iy_i-wx_i^2)=b\sum_{i=1}^{m}x_i i=1∑m(xiyi−wxi2)=bi=1∑mxi
∑ i = 1 m ( y i − w x i ) = m b \sum_{i=1}^{m}(y_i-wx_i) = mb i=1∑m(yi−wxi)=mb
消去 b b b，得
m ∑ i = 1 m ( x i y i − w x i 2 ) ∑ i = 1 m x i = ∑ i = 1 m ( y i − w x i ) m\frac {\sum_{i=1}^{m}(x_iy_i-wx_i^2)}{\sum_{i=1}^{m}x_i}=\sum_{i=1}^{m}(y_i-wx_i)
m ∑ i = 1 m ( x i y i − w x i 2 ) = ∑ i = 1 m x i ∑ i = 1 m y i − w ∑ i = 1 m x i ∑ i = 1 m x i m\sum_{i=1}^{m}(x_iy_i-wx_i^2)=\sum_{i=1}^{m}x_i\sum_{i=1}^{m}y_i-w\sum_{i=1}^{m}x_i\sum_{i=1}^{m}x_i mi=1∑m(xiyi−wxi2)=i=1∑mxii=1∑myi−wi=1∑mxii=1∑mxi
w ( ∑ i = 1 m x i ∑ i = 1 m x i − m ∑ i = 1 m x i 2 ) = ∑ i = 1 m x i ∑ i = 1 m y i − m ∑ i = 1 m x i y i w(\sum_{i=1}^{m}x_i\sum_{i=1}^{m}x_i- m\sum_{i=1}^{m}x_i^2)= \sum_{i=1}^{m}x_i\sum_{i=1}^{m}y_i-m\sum_{i=1}^{m}x_iy_i w(i=1∑mxii=1∑mxi−mi=1∑mxi2)=i=1∑mxii=1∑myi−mi=1∑mxiyi
w = x ‾ ∑ i = 1 m y i − ∑ i = 1 m x i y i 1 / m ∑ i = 1 m x i ∑ i = 1 m x i − ∑ i = 1 m x i 2 w= \frac{\overline x\sum_{i=1}^{m}y_i-\sum_{i=1}^{m}x_iy_i}{1/m\sum_{i=1}^{m}x_i\sum_{i=1}^{m}x_i- \sum_{i=1}^{m}x_i^2} w=1/m∑i=1mxi∑i=1mxi−∑i=1mxi2x∑i=1myi−∑i=1mxiyi
最終得
w = ∑ i = 1 m y i ( x i − x ‾ ) ∑ i = 1 m x i 2 − 1 m ( ∑ i = 1 m x i ) 2 w= \frac{\sum_{i=1}^{m}y_i(x_i-\overline x)}{\sum_{i=1}^{m}x_i^2-\frac{1}{m}(\sum_{i=1}^{m}x_i)^2} w=∑i=1mxi2−m1(∑i=1mxi)2∑i=1myi(xi−x)
其中 x ‾ = ∑ i = 1 m x i m \overline x=\frac{\sum_{i=1}^{m}x_i}{m} x=m∑i=1mxi
那麼
b
b
b的結果為
b
=
∑
i
=
1
m
(
y
i
−
w
x
i
)
m
b=\frac{\sum_{i=1}^{m}(y_i-wx_i)}{m}
b=m∑i=1m(yi−wxi)
Python實現
import numpy as np
def linear_regression(x, y, px):
sx = np.sum(x)
sy = np.sum(y)
sxy = np.dot(x, y.reshape(slid, 1))
sx2 = np.dot(x, x.reshape(slid, 1))
w = (sxy-sx*sy/slid)/(sx2-sx*sx/slid)
b = (sy-w*sx)/slid
py = w * px + b
return py