In my regression using OLS using Difference-in-Differences regression, when I add a dummy variable into my regression, the Adjusted R-squared reduce from 0.6117 to 0.6111. Is it a critical result and what should I talk about the impact of adding this dummy variable. Many thanks in advance.
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ You can use the F-test to compare the two models. $\endgroup$– Herr K.Commented Sep 23, 2021 at 1:39
-
2$\begingroup$ I would not pay too much attention to $R^2_{adj.}$. See e.g. "Justification for and optimality of $R^2_{adj.}$ as a model selection criterion". $\endgroup$– Richard HardyCommented Sep 23, 2021 at 5:11
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It is not a critical result at all.
First, the difference is very small. Both round to 0.61 and we don't usually look at further digits.
Most importantly though, the R-squared is largely irrelevant to difference-in-difference analysis.
You should be most worried about whether your identifying assumptions (e.g. parallel trends) hold. Secondly, you can consider whether the dummy may change the interpretation of your results. Lastly, consider if it could be a "bad" control in the theoretical sense, as discussed, e.g. in Angrist and Pischke's book "Mostly harmless econometrics".
Other considerations will not matter for your attempt to causally identify the coefficient of interest, which is the goal of difference-in-difference.
All your result means (albeit oversimplified) is that the dummy has a bit less predictive power than the other variables on average. But you are not trying to predict/forecast the outcome variable. You are trying to causally identify an effect (of another variable). So even if the drop in R-squared were larger, it would not itself matter.
And if you are trying to forecast, then difference-in-difference is not the right method for you.
