# OLS, Frisch-Waugh-Lovell Theorem and few observations

I was talking to a colleague a few days ago and the following question came up:

Why can't we just use projection matrices and the FWL Theorem to "gain" a few degrees of freedom in case of very few observations?

Imagine a case where you want to estimate

$$y_t = \beta_0 + z\beta_1 + x_2\beta_2 + x_3\beta_3 + u_t$$

but unfortunately, you just have 5 observations (extreme example). Luckily, you are just interested in $\beta_1$. Now you could just create some projection matrices in the form

$$M_x = I - x'(x'x)^{-1}x'$$

and project off your "unwanted" $x_2$ and $x_3$ (following Davidson and MacKinnon). You would be left with the regression

$$M_x y = M_x z \beta_1 + residuals$$

where $\beta_1$ is numerically the same as in the regression above. But now you are able to estimate just one (beta-)parameter with 5 observations compared to three parameters before. So, obviously, the estimates improve, don't they?

Where is our mistake?

• Zen, by gaining degrees of freedom, do you mean when using an F-test to test the significance of $\beta_1$? – An old man in the sea. May 25 '17 at 18:37
• Degrees of freedom in an F-test can be increased in this way as well as it seems to be possible to get estimates with higher precision this way. I don't really have an analytical argument for it, but especially in cases with few observations it seems that decreasing parameters to estimate might have a big effect on the quality of your regression and this, in a way, feels like "cheating" somehow. – Mr. Zen May 25 '17 at 18:53
• I think the last equation doesn't satisfy the usual assumptions, namely that of no autocorrelation. The each residuals is a linear combination of all $u_t$. So, even if $u_t$ are independent, each $\hat u_t$ is not. – An old man in the sea. May 25 '17 at 19:02
• This would better fit Cross Validated than Economics Stack Exchange. – Richard Hardy May 30 '17 at 16:45
• I think that the question fits just fine here. See here for previous discussions regarding this: economics.meta.stackexchange.com/questions/1484/… – jmbejara May 30 '17 at 22:03

We know that $M_X\mathbf{u}= \mathbf{\hat u}$, where $\mathbf{y}=\mathbf{x\beta}+\mathbf{u}$. So even if the error terms are independent, the residuals will not be independent of the other residuals. This is makes the usual assumption of no autocorrelation invalid.