Fama Macbeth and double clustering presents inconsistent results

Question

I have a large unbalanced panel data with 460 firms and 1259 days. The model I would like to run is below

$$ Y_{it} = \beta X_{it} + \alpha Z_{t} + \epsilon_{it} $$

where $Y_{it}$ is stock return, and $Z_{t}$ are Fama French 3 factors, and $X_{it}$ are variables of interest.

I run Fama Macbeth (FM) and double clustering to correct for the standard error, but two models give inconsistent results,i.e., $\beta$ is significant in one model, and not in the other.

I understand that Fama-MacBeth technique was developed to account for correlation between observations on different firms in the same time point, not to account for correlation between observations on the same firm in different time points. Traditionally, it should run cross section regression at each time point, then average estimates along time $T$. But in my case, due to the inclusion of $Z_t$, I have to run time series regression first since otherwise, $Z_{t}$ are not identifiable. In this case, does FM actually correct for correlation between observations on the same firm in different time points?

More importantly, does the inconsistent result mean that my results are not robust? Under my case, can I argue one is more appropriate than the other? Can the unbalanced data structure contribute to the inconsistent results?

I'm using fm and cluster2 command on this page Stata command

Answer 1

This is a large comment.

Fama-MacBeth only corrects for cross-sectional correlation in a panel and suffers from the error-in-variables problem (your $Z_t$, also check Chordia, Goyal and Shanken (2015)).

Depending on the exact nature of your dataset's variables the answer to your questions is "it depends"; please check Goyal (2012), Mitchell (2009) and Ibragimov and Mueller (2010).

1. Goyal (2012) accurately explains the F-M procedure (section 2.5). He states that "whether there is a bias in the traditional Fama–MacBeth approach if expected returns vary with time-varying characteristics is still unexplored.". Moreover, he says that "autocorrelation in returns (negligible at monthly frequency) leads to autocorrelation in risk premium estimates. This is easily accounted for by Newey– West type corrections to variance formulas".

2. Mitchell (2009) discusses the SEs in financial panel data and clearly states that we should identify the presence of a firm or time effect to see whether the Fama-MacBeth standard errors are unbiased.

3. Ibragimov and Mueller (2010) "find that as long as year coefficient estimators are approximately normal (or scale mixtures of normals) and independent, the Fama–MacBeth method results in valid inference even for a short panel that is heterogenous over time."

You might also find Jagannathan, Skoulakis and Wang. (2014) enlightening to make proper arguments on the appropriateness of the methods if your study is asset pricing related; in corporate finance things are more "flexible".

Last, I am far from an expert in this area, but I think the "pre-made" Stata commands are not exhaustive in dealing with variables with different statistical characteristics (e.g. autocorrelated returns).