1
$\begingroup$

I understand that in multiple regression $$\sum_{}^{} X_{i,j}\hat{u}_{i} = 0 $$ but I do not understand how my textbook says that if we include the intercept in the regression ($X_{i,0} = 1$)then we get: $$\sum_{}^{} \hat{u}_{i} = 0$$

$\endgroup$

2 Answers 2

3
$\begingroup$

Completing the notation with the indices $$ \forall j: \sum_{i=1}^{n} X_{i,j}\hat{u}_{i} = 0. $$ As you say, if $X_0$ is the constant then $$ \forall i: X_{i,0} = 1. $$ Inputing $j = 0$ into the first equation $$ \begin{align*} \sum_{i=1}^{n} X_{i,0}\hat{u}_{i} & = 0 \\ \\ \sum_{i=1}^{n} 1\hat{u}_{i} & = 0 \\ \\ \sum_{i=1}^{n} \hat{u}_{i} & = 0. \end{align*} $$

$\endgroup$
1
$\begingroup$

When an intercept is included, sum of residuals in multiple regression equals 0.

In multiple regression, $$ \hat{y}_i = \beta_0 + \beta_1x_{i,1} + \beta_2x_{i,2} +…+ \beta_px_{i,p} $$ In Least squares regression, the sum of the squares of the errors is minimized. $$ SSE=\displaystyle\sum\limits_{i=1}^n \left(e_i \right)^2= \sum_{i=1}^n\left(y_i - \hat{y_i} \right)^2= \sum_{i=1}^n\left(y_i -\beta_0- \beta_1x_{i,1}-\beta_2x_{i,2}-…- \beta_px_{i,p} \right)^2 $$ Take the partial derivative of SSE with respect to $\beta_0$ and setting it to zero. $$ \frac{\partial{SSE}}{\partial{\beta_0}} = \sum_{i=1}^n 2\left(y_i -\beta_0- \beta_1x_{i,1}-\beta_2x_{i,2}-…- \beta_px_{i,p} \right)^1 (-1) = -2\displaystyle\sum\limits_{i=1}^n(y_i-\hat{y_i})=-2\displaystyle\sum\limits_{i=1}^ne_i=0 $$ Hence, when an intercept is included, sum of residuals in multiple regression equals 0.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.