1
$\begingroup$

To test whether it is appropriate to even use certain types of fixed effects, would it ever be appropriate to use a cross-validation methodology? Let's say that I have this type of mental model:

$$\text{wage} = \text{years of schooling} + \text{household FE} + \text{industry FE} + \text{year FE}$$

Suppose, however, that I use cross-validation to test this time series model out of sample, and somehow find out that if I don't include household FE, that my predictions of wage are more accurate (as measured by, say, lower out of sample RMSE). In that case, would it be more appropriate to use this model instead?

$$\text{wage} = \text{years of schooling} + \text{industry FE} + \text{year FE}$$

Why or why not?

My own intuition suggests that if there truly is household-level effects that are confounding estimation, then adding household FE should increase the accuracy of out-of-sample predictions. At the same time, those predictions seem to rely on accurate estimation of each household's fixed effects, which may not be accurate for an individual household. On the other hand, I know that one would never naively throw variables to predict an outcome to get a causal model.

As another way of putting it, are the following statements true or false:

'if there truly is a causal relationship with one variable, then adding that variable should help improve prediction.'

'if there is a causal relationship, AND my prediction is more inaccurate when using those specific causal variables, then there is no way I can accurately identify the causal parameter of interest' (in this case the slope on years of schooling)

$\endgroup$
4
  • $\begingroup$ Why do you want to predict wage? This sounds like data science but not economics. $\endgroup$ Feb 23, 2022 at 11:38
  • $\begingroup$ I guess my question is more in line with: 'if there is truly a causal relationship, then it should help improve prediction.' $\endgroup$
    – Daycent
    Feb 23, 2022 at 14:47
  • $\begingroup$ What casual relationship here? Some unknown characteristics varying across households affect wage? $\endgroup$ Feb 23, 2022 at 14:58
  • $\begingroup$ yes, (e.g. household network, household language, etc.) $\endgroup$
    – Daycent
    Feb 23, 2022 at 15:02

1 Answer 1

2
$\begingroup$

How many observations do you have for each household? If it's a small number, then the household FE won't be estimated well. This will "average out" across households, and the estimate of the effect of years of schooling won't be biased because of it, but if you predict out of sample the imprecise and volatile HH FE point estimates will cause poor prediction out of sample.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.