線形重回帰分析の回帰係数の導出(説明変数が2個の場合)

■説明変数が2つの場合

データNo.	データ $X_{1}$ （説明変数1）	データ $X_{2}$ （説明変数2）	データ $Y$ （目的変数）
1	$x_{11}$	$x_{12}$	$y_{1}$
2	$x_{21}$	$x_{22}$	$y_{2}$
3	$x_{31}$	$x_{32}$	$y_{3}$
$⋮$	$⋮$	$⋮$	$⋮$
$n$	$x_{n 1}$	$x_{n 2}$	$y_{n}$

線形重回帰式

$\hat{y} = β_{0} + β_{1} x_{1} + β_{2} x_{2}$ ･･････(1)

の偏回帰係数 $β_{0}$ ， $β_{1}$ ， $β_{2}$ (定数項 $β_{0}$ も偏回帰係数に含める)を求める．

残差平方和 $S_{S E}$ は

$S_{S E} = \sum_{i = 1}^{n} ε_{i}^{2}$

$= \sum_{i = 1}^{n} {(y_{i} - \hat{y_{i}})}^{2}$

$= \sum_{i = 1}^{n} {\{y_{i} - (β_{0} + β_{1} x_{i 1} + β_{2} x_{i 2})\}}^{2}$ ･･････(2)

となる．

回帰式では， $S_{S E}$ が最小(極小)となるとなるように $β_{0}$ ， $β_{1}$ ， $β_{2}$ が求められている．よって， $S_{S E}$ が極小となるときに成り立つ式

\frac{\partial S_{S E}}{\partial β_{0}} = 0

･･････(3)

\frac{\partial S_{S E}}{\partial β_{1}} = 0

･･････(4)

\frac{\partial S_{S E}}{\partial β_{2}} = 0

･･････(5)

を満たす $β_{0}$ ， $β_{1}$ ， $β_{2}$ を求めればよい．

備考：残差平方和の特徴として，極小は存在するが，極大は存在しない．よって，(3)，(4)，(5)を満たすのは極小のときのみである．

(3)と(2)より(偏微分の計算はここを参照)

$\frac{\partial}{\partial β_{0}} \sum_{i = 1}^{n} {\{y_{i} - (β_{0} + β_{1} x_{i 1} + β_{2} x_{i 2})\}}^{2} = 0$

$\sum_{i = 1}^{n} \frac{\partial}{\partial β_{0}} {\{y_{i} - (β_{0} + β_{1} x_{i 1} + β_{2} x_{i 2})\}}^{2} = 0$

$\sum_{i = 1}^{n} 2 \{y_{i} - (β_{0} + β_{1} x_{i 1} + β_{2} x_{i 2})\} (- 1) = 0$

$\sum_{i = 1}^{n} \{y_{i} - (β_{0} + β_{1} x_{i 1} + β_{2} x_{i 2})\} = 0$ ･･････(6)

が得られる．

(6)の両辺を $n$ で割る．

$\frac{1}{n} \sum_{i = 1}^{n} \{y_{i} - (β_{0} + β_{1} x_{i 1} + β_{2} x_{i 2})\} = 0$

$\frac{1}{n} \sum_{i = 1}^{n} y_{i} - \frac{1}{n} \sum_{i = 1}^{n} β_{0} - β_{1} \frac{1}{n} \sum_{i = 1}^{n} x_{i 1} - β_{2} \frac{1}{n} \sum_{i = 1}^{n} x_{i 2} = 0$

ここで

\bar{y} = \frac{1}{n} \sum_{i = 1}^{n} y_{i}

･･････(7)

\bar{x_{1}} = \frac{1}{n} \sum_{i = 1}^{n} x_{i 1}

･･････(8)

\bar{x_{2}} = \frac{1}{n} \sum_{i = 1}^{n} x_{i 2}

･･････(9)

とおくと

$\bar{y} - β_{0} - β_{1} \bar{x_{1}} - β_{2} \bar{x_{2}} = 0$

$β_{0} = \bar{y} - β_{1} \bar{x_{1}} - β_{2} \bar{x_{2}}$ ･･････(10)

(10)を(2)に代入する．

$S_{S E} = \sum_{i = 1}^{n} {\{y_{i} - (\bar{y} - β_{1} \bar{x_{1}} - β_{2} \bar{x_{2}} + β_{1} x_{i 1} + β_{2} x_{i 2})\}}^{2}$

$= \sum_{i = 1}^{n} {\{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\}}^{2}$ ･･････(11)

(4)と(11)より

$\frac{\partial}{\partial β_{1}} \sum_{i = 1}^{n} {\{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} \frac{\partial}{\partial β_{1}} {\{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} 2 \{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\} \{- (x_{i 1} - \bar{x_{1}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) \{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (y_{i} - \bar{y})$ $- β_{1} \sum_{i = 1}^{n} {(x_{i 1} - \bar{x_{1}})}^{2}$ $- β_{2} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}}) = 0$ ･･････(12)

(12)の両辺を $n$ で割る．

$\frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (y_{i} - \bar{y})$ $- β_{1} \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 1} - \bar{x_{1}})}^{2}$ $- β_{2} \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}}) = 0$

ここで

σ_{x_{1} y} = \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (y_{i} - \bar{y})

･･･(13)

σ_{x_{1}}^{2} = \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 1} - \bar{x_{1}})}^{2}

･･･(14)

σ_{x_{1} x_{2}} = \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}})

･･･(15)

おくと

$σ_{x_{1} y} - β_{1} σ_{x_{1}}^{2} - β_{2} σ_{x_{1} x_{2}} = 0$

$σ_{x_{1} y} = β_{1} σ_{x_{1}}^{2} + β_{2} σ_{x_{1} x_{2}}$ ･･････(16)

(5)と(11)より

$\frac{\partial}{\partial β_{2}} \sum_{i = 1}^{n} {\{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} \frac{\partial}{\partial β_{2}} {\{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\}}^{2} = 0$

$\sum_{i = 1}^{n} 2 \{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\} \{- (x_{i 2} - \bar{x_{2}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) \{y_{i} - \bar{y} - β_{1} (x_{i 1} - \bar{x_{1}}) - β_{2} (x_{i 2} - \bar{x_{2}})\} = 0$

$\sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) (y_{i} - \bar{y})$ $- β_{1} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}})$ $- β_{2} \sum_{i = 1}^{n} {(x_{i 2} - \bar{x_{2}})}^{2} = 0$ ･･････(17)

(13)の両辺を $n$ で割る．

$\frac{1}{n} \sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) (y_{i} - \bar{y})$ $- β_{1} \frac{1}{n} \sum_{i = 1}^{n} (x_{i 1} - \bar{x_{1}}) (x_{i 2} - \bar{x_{2}})$ $- β_{2} \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 2} - \bar{x_{2}})}^{2} = 0$

ここで

σ_{x_{2} y} = \frac{1}{n} \sum_{i = 1}^{n} (x_{i 2} - \bar{x_{2}}) (y_{i} - \bar{y})

･･･(18)

σ_{x_{2}}^{2} = \frac{1}{n} \sum_{i = 1}^{n} {(x_{i 2} - \bar{x_{2}})}^{2}

･･･(19)

とおく．（18)，(15)，(19)より

$σ_{x_{2} y} - β_{1} σ_{x_{1} x_{2}} - β_{2} σ_{x_{2}}^{2} = 0$

$σ_{x_{2} y} = β_{1} σ_{x_{1} x_{2}} + β_{2} σ_{x_{2}}^{2}$ ･･････(20)

(16)と(20)の連立方程式

$\{\begin{cases} σ_{x_{1} y} = β_{1} σ_{x_{1}}^{2} + β_{2} σ_{x_{1} x_{2}} \\ σ_{x_{2} y} = β_{1} σ_{x_{1} x_{2}} + β_{2} σ_{x_{2}}^{2} \end{cases}$

を行列を使って表わすと

$(\begin{matrix} σ_{x_{1} y} \\ σ_{x_{2} y} \end{matrix}) = (\begin{matrix} σ_{x_{1}}^{2} & σ_{x_{1} x_{2}} \\ σ_{x_{1} x_{2}} & σ_{x_{2}}^{2} \end{matrix}) (\begin{matrix} β_{1} \\ β_{2} \end{matrix})$

となる．クラメルの公式を用いると

$β_{1} = \frac{|\begin{matrix} σ_{x_{1} y} & σ_{x_{1} x_{2}} \\ σ_{x_{2} y} & σ_{x_{2}}^{2} \end{matrix}|}{|\begin{matrix} σ_{x_{1}}^{2} & σ_{x_{1} x_{2}} \\ σ_{x_{1} x_{2}} & σ_{x_{2}}^{2} \end{matrix}|}$ ， $β_{2} = \frac{|\begin{matrix} σ_{x_{1} y} & σ_{x_{1} y} \\ σ_{x_{2} y} & σ_{x_{2} y} \end{matrix}|}{|\begin{matrix} σ_{x_{1}}^{2} & σ_{x_{1} x_{2}} \\ σ_{x_{1} x_{2}} & σ_{x_{2}}^{2} \end{matrix}|}$

となる．

ホーム>>カテゴリー分類>>確率>>統計>>線形回帰

　最終更新日： 2026年6月20日