I was talking to a colleague a few days ago and the following question came up:

Why can't we just use projection matrices and the FWL Theorem to "gain" a few degrees of freedom in case of very few observations?

Imagine a case where you want to estimate

$$y_t = \beta_0 + z\beta_1 + x_2\beta_2 + x_3\beta_3 + u_t$$

but unfortunately, you just have 5 observations (extreme example). Luckily, you are just interested in $\beta_1$. Now you could just create some projection matrices in the form

$$ M_x = I - x'(x'x)^{-1}x'$$

and project off your "unwanted" $x_2$ and $x_3$ (following Davidson and MacKinnon). You would be left with the regression

$$M_x y = M_x z \beta_1 + residuals$$

where $\beta_1$ is numerically the same as in the regression above. But now you are able to estimate just one (beta-)parameter with 5 observations compared to three parameters before. So, obviously, the estimates improve, don't they?

Where is our mistake?

  • $\begingroup$ Zen, by gaining degrees of freedom, do you mean when using an F-test to test the significance of $\beta_1$? $\endgroup$ May 25, 2017 at 18:37
  • $\begingroup$ Degrees of freedom in an F-test can be increased in this way as well as it seems to be possible to get estimates with higher precision this way. I don't really have an analytical argument for it, but especially in cases with few observations it seems that decreasing parameters to estimate might have a big effect on the quality of your regression and this, in a way, feels like "cheating" somehow. $\endgroup$
    – yrx1702
    May 25, 2017 at 18:53
  • $\begingroup$ I think the last equation doesn't satisfy the usual assumptions, namely that of no autocorrelation. The each residuals is a linear combination of all $u_t$. So, even if $u_t$ are independent, each $\hat u_t$ is not. $\endgroup$ May 25, 2017 at 19:02
  • $\begingroup$ This would better fit Cross Validated than Economics Stack Exchange. $\endgroup$ May 30, 2017 at 16:45
  • $\begingroup$ I think that the question fits just fine here. See here for previous discussions regarding this: economics.meta.stackexchange.com/questions/1484/… $\endgroup$
    – jmbejara
    May 30, 2017 at 22:03

1 Answer 1


We know that $M_X\mathbf{u}= \mathbf{\hat u}$, where $\mathbf{y}=\mathbf{x\beta}+\mathbf{u}$. So even if the error terms are independent, the residuals will not be independent of the other residuals. This is makes the usual assumption of no autocorrelation invalid.

Of course, if we use a HAC estimator then we can circumvent the existence of autocorrelation. However, the HAC estimators that I know of are just of asymptotic behaviour. So, it wouldn't be useful for a small sample size.

  • $\begingroup$ I don't think I really get what you mean. This is the case in every FWL-Regression. However, this does not change the numerical value of beta. In addition, precision of beta should go up substantially in the extreme example above. Can you elaborate a little bit? $\endgroup$
    – yrx1702
    May 25, 2017 at 19:39
  • $\begingroup$ What I'm trying to say is that we can use the last equation to estimate the betas by the usual formula, but not to do inference with, since it doesn't comply with the usual hypothesis. In this way, the formula for the estimate works, even though the last equation doesn't comply with the hypothesis of no correlation. At least this is my take on your question. $\endgroup$ May 25, 2017 at 20:01
  • $\begingroup$ Thanks for your answer! So that would mean that the FWL theorem has mainly theoretical applications? $\endgroup$
    – yrx1702
    May 25, 2017 at 20:31
  • $\begingroup$ Just wanted to leave it open in case someone else wants to give it a try. Thanks! $\endgroup$
    – yrx1702
    May 26, 2017 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.