3
$\begingroup$

I'm running a difference-in-differences (DID) regression on panel data as follows:

$$ Y_{it} = \beta_{0} + \beta_{1}Treat_{i}\times Post_{t} + \mathbf{\Pi}\mathbf{X}_{it} + \tau_{i} + \gamma_{t} +\varepsilon_{it}$$

where $\tau_{i}$ and $\gamma_{t}$ are company and year fixed effects, respectively. $Treat_i$ equals 1 for treated companies and 0 for control companies and $Post_t$ equals 1 for years 2010-2012 and 0 for years 2007-2009. These main effects are subsumed by the fixed effects. $\mathbf{X}_{it}$ is a vector of control variables. Standard errors are robust and clustered by company. $\beta_1$ is the difference-in-differences estimator that I'm interested in.

When I run the regression as is, the parallel trends assumption appears satisfied, i.e., when I replace the $Post_{t}$ in $Treat_{i}\times Post_{t}$ with separate year indicators, the coefficients for the pre-intervention years are not significantly different from zero. The post-intervention years are also not significantly different from zero (consistent with the fact that the estimated $\beta_1$ in the original regression is not significant).

However, the pre-intervention dependent variable and control variables differ substantially between the treatment and control groups. This motivates me to utilize a reweighting technique which results in an improved counterfactual. Specifically, I use entropy balancing (Hainmueller, 2012), although I don't think the particular matching method will be relevant to my question (e.g., it might be easier to imagine that I use propensity score weights). I include the average pre-intervention values of the dependent variable as well as other covariates as the conditioning variables. When I run the same regression as above but include the entropy weights, the parallel trends assumption is also satisfied but now the estimate of $\beta_1$ is significant.

My question is this:

Since the parallel trends assumption seems to be satisfied in both the original DID and in the DID combined with matching, is there a reliable source that justifies using the matching DID over the original DID? The first three papers cited below seem to suggest that the DID combined with matching provides more accurate estimates because it reduces bias, but I am not familiar enough with the literature to be sure that I am interpreting these papers correctly.

Edit: I'm digesting Ferraro and Miranda (2017) some more and my sense is that an insignificant result may be the result of heterogeneous treatment effects (p310): "...the constant treatment effect is hard to believe when the treated and untreated units, despite having similar trends, have very different levels of fixed characteristics (like home size and age) that affect water use." Any thoughts on this would be greatly appreciated.

Citations:

Imbens, Guido W. and Jeffrey M. Wooldridge. 2009. Recent developments in the econometrics of program evaluation. Journal of Economic Literature 47, no. 1: 5-86.

Ferraro, Paul J., and Juan José Miranda. "Panel data designs and estimators as substitutes for randomized controlled trials in the evaluation of public programs." Journal of the Association of Environmental and Resource Economists 4, no. 1 (2017): 281-317.

Heckman, James J., Hidehiko Ichimura, and Petra Todd. 1997. Matching as an econometric evaluation estimator: Evidence from evaluating a job training program. Review of Economic Studies 64 (4): 605–54.

Hainmueller, Jens. "Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies." Political Analysis 20, no. 1 (2012): 25-46.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.