1
$\begingroup$

I understand that in multiple regression $$\sum_{}^{} X_{i,j}\hat{u}_{i} = 0 $$ but I do not understand how my textbook says that if we include the intercept in the regression ($X_{i,0} = 1$)then we get: $$\sum_{}^{} \hat{u}_{i} = 0$$

$\endgroup$
3
$\begingroup$

Completing the notation with the indices $$ \forall j: \sum_{i=1}^{n} X_{i,j}\hat{u}_{i} = 0. $$ As you say, if $X_0$ is the constant then $$ \forall i: X_{i,0} = 1. $$ Inputing $j = 0$ into the first equation $$ \begin{align*} \sum_{i=1}^{n} X_{i,0}\hat{u}_{i} & = 0 \\ \\ \sum_{i=1}^{n} 1\hat{u}_{i} & = 0 \\ \\ \sum_{i=1}^{n} \hat{u}_{i} & = 0. \end{align*} $$

|improve this answer|||||
$\endgroup$
1
$\begingroup$

When an intercept is included, sum of residuals in multiple regression equals 0.

In multiple regression, $$ \hat{y}_i = \beta_0 + \beta_1x_{i,1} + \beta_2x_{i,2} +…+ \beta_px_{i,p} $$ In Least squares regression, the sum of the squares of the errors is minimized. $$ SSE=\displaystyle\sum\limits_{i=1}^n \left(e_i \right)^2= \sum_{i=1}^n\left(y_i - \hat{y_i} \right)^2= \sum_{i=1}^n\left(y_i -\beta_0- \beta_1x_{i,1}-\beta_2x_{i,2}-…- \beta_px_{i,p} \right)^2 $$ Take the partial derivative of SSE with respect to $\beta_0$ and setting it to zero. $$ \frac{\partial{SSE}}{\partial{\beta_0}} = \sum_{i=1}^n 2\left(y_i -\beta_0- \beta_1x_{i,1}-\beta_2x_{i,2}-…- \beta_px_{i,p} \right)^1 (-1) = -2\displaystyle\sum\limits_{i=1}^n(y_i-\hat{y_i})=-2\displaystyle\sum\limits_{i=1}^ne_i=0 $$ Hence, when an intercept is included, sum of residuals in multiple regression equals 0.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.