2
$\begingroup$

I was talking to a colleague a few days ago and the following question came up:

Why can't we just use projection matrices and the FWL Theorem to "gain" a few degrees of freedom in case of very few observations?

Imagine a case where you want to estimate

$$y_t = \beta_0 + z\beta_1 + x_2\beta_2 + x_3\beta_3 + u_t$$

but unfortunately, you just have 5 observations (extreme example). Luckily, you are just interested in $\beta_1$. Now you could just create some projection matrices in the form

$$ M_x = I - x'(x'x)^{-1}x'$$

and project off your "unwanted" $x_2$ and $x_3$ (following Davidson and MacKinnon). You would be left with the regression

$$M_x y = M_x z \beta_1 + residuals$$

where $\beta_1$ is numerically the same as in the regression above. But now you are able to estimate just one (beta-)parameter with 5 observations compared to three parameters before. So, obviously, the estimates improve, don't they?

Where is our mistake?

Improve this question
$\endgroup$
  • $\begingroup$ Zen, by gaining degrees of freedom, do you mean when using an F-test to test the significance of $\beta_1$? $\endgroup$ – An old man in the sea. May 25 '17 at 18:37
  • $\begingroup$ Degrees of freedom in an F-test can be increased in this way as well as it seems to be possible to get estimates with higher precision this way. I don't really have an analytical argument for it, but especially in cases with few observations it seems that decreasing parameters to estimate might have a big effect on the quality of your regression and this, in a way, feels like "cheating" somehow. $\endgroup$ – Mr. Z May 25 '17 at 18:53
  • $\begingroup$ I think the last equation doesn't satisfy the usual assumptions, namely that of no autocorrelation. The each residuals is a linear combination of all $u_t$. So, even if $u_t$ are independent, each $\hat u_t$ is not. $\endgroup$ – An old man in the sea. May 25 '17 at 19:02
  • $\begingroup$ This would better fit Cross Validated than Economics Stack Exchange. $\endgroup$ – Richard Hardy May 30 '17 at 16:45
  • $\begingroup$ I think that the question fits just fine here. See here for previous discussions regarding this: economics.meta.stackexchange.com/questions/1484/… $\endgroup$ – jmbejara May 30 '17 at 22:03
0
$\begingroup$

We know that $M_X\mathbf{u}= \mathbf{\hat u}$, where $\mathbf{y}=\mathbf{x\beta}+\mathbf{u}$. So even if the error terms are independent, the residuals will not be independent of the other residuals. This is makes the usual assumption of no autocorrelation invalid.

Of course, if we use a HAC estimator then we can circumvent the existence of autocorrelation. However, the HAC estimators that I know of are just of asymptotic behaviour. So, it wouldn't be useful for a small sample size.

Improve this answer
$\endgroup$
  • $\begingroup$ I don't think I really get what you mean. This is the case in every FWL-Regression. However, this does not change the numerical value of beta. In addition, precision of beta should go up substantially in the extreme example above. Can you elaborate a little bit? $\endgroup$ – Mr. Z May 25 '17 at 19:39
  • $\begingroup$ What I'm trying to say is that we can use the last equation to estimate the betas by the usual formula, but not to do inference with, since it doesn't comply with the usual hypothesis. In this way, the formula for the estimate works, even though the last equation doesn't comply with the hypothesis of no correlation. At least this is my take on your question. $\endgroup$ – An old man in the sea. May 25 '17 at 20:01
  • $\begingroup$ Thanks for your answer! So that would mean that the FWL theorem has mainly theoretical applications? $\endgroup$ – Mr. Z May 25 '17 at 20:31
  • $\begingroup$ @Mr.Zen That I wouldn't know... By the way, it seems that you're new to this community. If you're satisfied with an answer, it's practice to accept it by pressing the tick sign in the respective answer. Just in case you might not know what to do. Leaving a question with a unaccepted answer has some negative weight in some statistics by which these communities are assessed. But only accept if you're truly satisfied with the answer. $\endgroup$ – An old man in the sea. May 25 '17 at 20:49
  • $\begingroup$ Just wanted to leave it open in case someone else wants to give it a try. Thanks! $\endgroup$ – Mr. Z May 26 '17 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.