1
$\begingroup$

If I have data that is on some level, say industry-county-year, and a certain treatment, say a regulation change in some states in certain years. Is it possible to run a Dif-in-Dif on the industry-county-year level and cluster by states? Or do I overestimate my statistical significance if I don't aggregate my data to the state level first? If it is not necessary, are there any advantages to aggregating?

And would clustering be a problem if only one state is treated? Thus, if there would only be 1 treatment cluster and ~50 control clusters.

$\endgroup$
4
$\begingroup$

Yes, you can run diff-in-diff with aggreggate treatment in this case.

Yes, standard errors should be clustered at the treatment (here=state) level.

No, the number of clusters is not an issue, as the relevant number is the total number of clusters and 50 should reasonably avoid the small cluster problem.

The way you can think about this is to forget states exits. You simply have some (US) observations (firms?) that are treated and others that are not. In principle, that gives you diff-in-diff if you can satisfy all the assumptions on the control group (similar behavior in absence of treatment).

However, three other general problems arise.

  1. Ideally, in this case you'd also want to show results at the aggregated treatment level, which you can't do because of small sample bias, since that gives you only 50 observations in each time period, but that's fine.

  2. Finding the right control units/states (satisfying assumptions, e.g. parallel trends) is hard. The fact that only one state is treated may mean that state is very unique in the context of your research question. Maybe you want to use propensity score matching or synthetic control in addition to diff-in-diff.

  3. Treatments at state level can be very general and you need to ensure other diverging policies also aren't happening etc.

That being said, these types of analyses can be performed and are often done. I recommend finding a paper that does diff-in-diff with treatment at the state level and follow their steps to get you started.

$\endgroup$
2
  • $\begingroup$ Thanks, for a great answer! But is there an advantage in running the regs on the aggregate level to running them on the county-industry level, if I cluster on the state level in both cases? $\endgroup$
    – ArOk
    Jan 12 at 21:27
  • 1
    $\begingroup$ So keep in mind that clustering only affects standard errors and not identificaiton/bias. To have a stronger case against bias causing problems, it is in principle good to see results at the level of treatment. But you cannot do that anyway, because then you would have regs with 50 observations in each time period, which gives you small sample bias. $\endgroup$
    – BB King
    Jan 12 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.