線形重回帰分析の回帰係数の導出(説明変数が2個の場合)

■説明変数が2つの場合

データNo.	データ $X_{1}$ （説明変数1）	データ $X_{2}$ （説明変数2）	データ $Y$ （目的変数）
1	$x_{11}$	$x_{12}$	$y_{1}$
2	$x_{21}$	$x_{22}$	$y_{2}$
3	$x_{31}$	$x_{32}$	$y_{3}$
$⋮$	$⋮$	$⋮$	$⋮$
$n$	$x_{n 1}$	$x_{n 2}$	$y_{n}$

線形重回帰式

$\hat{y} = a_{0} + a_{1} x_{1} + a_{2} x_{2}$ ･･････(1)

の定数項 $a_{0}$ ，および，偏回帰係数 $a_{1}$ ， $a_{2}$ を求める．

残差平方和 $R_{S S}$ は

$R_{S S} = \sum_{i = 1}^{n} ε_{i}^{2}$

$= \sum_{i = 1}^{n} {(y_{i} - \hat{y_{i}})}^{2}$

$= \sum_{i = 1}^{n} {\{y_{i} - (a_{0} + a_{1} x_{i 1} + a_{2} x_{i 2})\}}^{2}$ ･･････(2)

となる．

回帰式では， $R_{S S}$ が最小(極小)となるとなるように $a_{0}$ ， $a_{1}$ ， $a_{2}$ が求められている．よって， $R_{S S}$ が極小となるときに成り立つ式

\frac{\partial R_{S S}}{\partial a_{0}} = 0

･･････(3)

\frac{\partial R_{S S}}{\partial a_{1}} = 0

･･････(4)

\frac{\partial R_{S S}}{\partial a_{2}} = 0

･･････(5)

を満たす $a_{0}$ ， $a_{1}$ ， $a_{2}$ を求めればよい．

備考：残差平方和の特徴として，極小は存在するが，極大は存在しない．よって，(3)，(4)，(5)を満たすのは極小のときのみである．

(3)と(2)より(偏微分の計算はここを参照)

$\frac{\partial}{\partial a_{0}} \sum_{i = 1}^{n} {\{y_{i} - (a_{0} + a_{1} x_{i 1} + a_{2} x_{i 2})\}}^{2} = 0$

$\sum_{i = 1}^{n} \frac{\partial}{\partial a_{0}} {\{y_{i} - (a_{0} + a_{1} x_{i 1} + a_{2} x_{i 2})\}}^{2} = 0$

$\sum_{i = 1}^{n} 2 \{y_{i} - (a_{0} + a_{1} x_{i 1} + a_{2} x_{i 2})\} (- 1) = 0$

$\sum_{i = 1}^{n} \{y_{i} - (a_{0} + a_{1} x_{i 1} + a_{2} x_{i 2})\} = 0$ ･･････(6)

が得られる．

(6)の両辺を $n$ で割る．

$\frac{1}{n} \sum_{i = 1}^{n} \{y_{i} - (a_{0} + a_{1} x_{i 1} + a_{2} x_{i 2})\} = 0$

$\frac{1}{n} \sum_{i = 1}^{n} y_{i} - \frac{1}{n} \sum_{i = 1}^{n} a_{0} - a_{1} \frac{1}{n} \sum_{i = 1}^{n} x_{i 1} - a_{2} \frac{1}{n} \sum_{i = 1}^{n} x_{i 2} = 0$

ここで

\bar{y} = \frac{1}{n} \sum_{i = 1}^{n} y_{i}

･･････(7)

\bar{x_{1}} = \frac{1}{n} \sum_{i = 1}^{n} x_{i 1}

･･････(8)

\bar{x_{2}} = \frac{1}{n} \sum_{i = 1}^{n} x_{i 2}

･･････(9)

とおくと

$\bar{y} - a_{0} - a_{1} \bar{x_{1}} - a_{2} \bar{x_{2}} = 0$

$a_{0} = \bar{y} - a_{1} \bar{x_{1}} - a_{2} \bar{x_{2}}$ ･･････(10)

(10)を(2)に代入する．

$R_{S S} = \sum_{i = 1}^{n} {\{y_{i} - (\bar{y} - a_{1} \bar{x_{1}} - a_{2} \bar{x_{2}} + a_{1} x_{i 1} + a_{2} x_{i 2})\}}^{2}$

$= \sum_{i = 1}^{n} {\{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\}}^{2}$ ･･････(11)

(4)と(11)より

$\frac{\partial}{\partial a_{1}} \sum_{i = 1}^{n} {\{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} \frac{\partial}{\partial a_{1}} {\{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} 2 \{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\} \{- (x_{i 1} - \bar{x_{1}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) \{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (y_{i} - \bar{y})$ $- a_{1} \sum_{i = 1}^{n} {(x_{i 1} - \bar{x_{1}})}^{2}$ $- a_{2} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}}) = 0$ ･･････(12)

(12)の両辺を $n$ で割る．

$\frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (y_{i} - \bar{y})$ $- a_{1} \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 1} - \bar{x_{1}})}^{2}$ $- a_{2} \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}}) = 0$

ここで

σ_{x_{1} y} = \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (y_{i} - \bar{y})

･･･(13)

σ_{x_{1}}^{2} = \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 1} - \bar{x_{1}})}^{2}

･･･(14)

σ_{x_{1} x_{2}} = \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}})

･･･(15)

おくと

$σ_{x_{1} y} - a_{1} σ_{x_{1}}^{2} - a_{2} σ_{x_{1} x_{2}} = 0$

$σ_{x_{1} y} = a_{1} σ_{x_{1}}^{2} + a_{2} σ_{x_{1} x_{2}}$ ･･････(16)

(5)と(11)より

$\frac{\partial}{\partial a_{2}} \sum_{i = 1}^{n} {\{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} \frac{\partial}{\partial a_{2}} {\{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} 2 \{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\} \{- (x_{i 2} - \bar{x_{2}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) \{y_{i} - \bar{y} - a_{1} (x_{i 1} - \bar{x_{1}}) - a_{2} (x_{i 2} - \bar{x_{2}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) (y_{i} - \bar{y})$ $- a_{1} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}})$ $- a_{2} \sum_{i = 1}^{n} {(x_{i 2} - \bar{x_{2}})}^{2} = 0$ ･･････(17)

(13)の両辺を $n$ で割る．

$\frac{1}{n} \sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) (y_{i} - \bar{y})$ $- a_{1} \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}})$ $- a_{2} \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 2} - \bar{x_{2}})}^{2} = 0$

ここで

σ_{x_{2} y} = \frac{1}{n} \sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) (y_{i} - \bar{y})

･･･(18)

σ_{x_{2}}^{2} = \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 2} - \bar{x_{2}})}^{2}

･･･(19)

とおく．（18)，(15)，(19)より

$σ_{x_{2} y} - a_{1} σ_{x_{1} x_{2}} - a_{2} σ_{x_{2}}^{2} = 0$

$σ_{x_{2} y} = a_{1} σ_{x_{1} x_{2}} + a_{2} σ_{x_{2}}^{2}$ ･･････(20)

(16)と(20)の連立方程式

$\{\begin{cases} σ_{x_{1} y} = a_{1} σ_{x_{1}}^{2} + a_{2} σ_{x_{1} x_{2}} \\ σ_{x_{2} y} = a_{1} σ_{x_{1} x_{2}} + a_{2} σ_{x_{2}}^{2} \end{cases}$

を行列を使って表わすと

$(\begin{matrix} σ_{x_{1} y} \\ σ_{x_{2} y} \end{matrix}) = (\begin{matrix} σ_{x_{1}}^{2} & σ_{x_{1} x_{2}} \\ σ_{x_{1} x_{2}} & σ_{x_{2}}^{2} \end{matrix}) (\begin{matrix} a_{1} \\ a_{2} \end{matrix})$

となる．クラメルの公式を用いると

$a_{1} = \frac{|\begin{matrix} σ_{x_{1} y} & σ_{x_{1} x_{2}} \\ σ_{x_{2} y} & σ_{x_{2}}^{2} \end{matrix}|}{|\begin{matrix} σ_{x_{1}}^{2} & σ_{x_{1} x_{2}} \\ σ_{x_{1} x_{2}} & σ_{x_{2}}^{2} \end{matrix}|}$ ， $a_{2} = \frac{|\begin{matrix} σ_{x_{1} y} & σ_{x_{1} y} \\ σ_{x_{2} y} & σ_{x_{2} y} \end{matrix}|}{|\begin{matrix} σ_{x_{1}}^{2} & σ_{x_{1} x_{2}} \\ σ_{x_{1} x_{2}} & σ_{x_{2}}^{2} \end{matrix}|}$

となる．

ホーム>>カテゴリー分類>>確率>>統計>>線形回帰

　最終更新日： 2026年5月12日