0
$\begingroup$

For parallel trend assumption in Difference-in-Differences (DiD), normally which benchmark we normally use to judge whether the parallel assumption is being satisfied?

From this answer from @1muflon1, it seems that the p-value of the joint null test of coefficients before the event date higher than 0,1 is the benchmark to say the parallel test is satisfied (there is no difference between treatment and control group before the event date).

I am wondering it is right-thinking? And I am wondering if there is any reference for this benchmark.

$\endgroup$
1
$\begingroup$

There is no universal rule, in principle you could get away even with the joint test showing significant differences at 10% but not at 5%. It all depends on specifics of your research. For example, if you have large sample, coefficients will be estimated with much higher precision so using 10% level would not be reasonable (although when it comes to testing parallel trend assumption it is rare to have more than 5-10 years of pre treatment data to check for the trend).

In addition, there is no bullet proof way of testing parallel trends, you can do not just one test but battery of tests to be more confident (see literature review on various ways of testing for parallel trends in Roth 2019a or Rambachan & Roth 2020).

But generally there are no widely accepted benchmarks, some people still get away with showcasing plots that show the variables sort of move together before intervention and without rigorous testing. It varies by subfield, and is context depended, my recommendation is to either look at what other people are doing in your subfield and apply similar tests, or potentially a bit more than that if you want to go extra mile.

$\endgroup$
10
  • 1
    $\begingroup$ @Louise because the more precision you have the likelier it is that even small difference between two trends that is economically not very meaningful will happen to be statistically significantly different from zero unless you go for more strict level like 5 or 1%. Also yes I meant p values, you check p value against your desired significance level which in social science is usually either 10, 5 or 1% or in decimal form 0.1, 0,05 and 0.01 $\endgroup$
    – 1muflon1
    Sep 10 at 10:53
  • 1
    $\begingroup$ @Louise no, I am saying that in a large dataset with 350k observations if the difference between trend is only significant at 10% or 0.1 then it might not be significant at all. With that many data points if some test is not significant at at least 5% or heck even 1% it is not really significant. Significance value tells you what is probability of committing type I error. With 350k observations your estimates should be estimated with enough precision that you should simply not tolerate chance of 10% type I error. But in smaller dataset you might be willing to do that since there is a lot of $\endgroup$
    – 1muflon1
    Sep 10 at 21:12
  • 1
    $\begingroup$ noise there so something statistically significant at 10% might actually truly indicate actual difference $\endgroup$
    – 1muflon1
    Sep 10 at 21:13
  • 1
    $\begingroup$ @Louise what exactly you want reference for? The fact that precision of the estimates increases with sample size is well known and you can find it any textbook, for example the formula for t-test is given by $t= \beta/(s/\sqrt{n})$ which is clearly increasing in number of observations n, so for example if beta is 0.5 s=1 with 4 observations the t stat would be 1 which would not be significant at any level, but with the same numbers just with higher number of observations let’s say n=25, the t statistics would 2.5 which would be significant at 5% level. Yet nothing changed here we have the same $\endgroup$
    – 1muflon1
    Sep 10 at 22:43
  • 1
    $\begingroup$ Point estimate, the same variability of that estimate just more data. Hence you need to take this fact in account no matter what empirical research you do, generally for smaller sample sizes using 10% level might be ok but for larger ones you should use 5% or even 1% definitely not 10% $\endgroup$
    – 1muflon1
    Sep 10 at 22:45
1
$\begingroup$

There's no widely accepted benchmark. The idea is to present as much evidence as you can to convince your audience that the parallel trend is satisfied. The least you can do (and most of the time is done in economics papers) is to plot the trend and see if there's any discernible divergent trend before the intervention. It also depends on your unit of analysis; if your unit of analysis is smaller than the unit of treatment (for instance, if you have individual data, but your treatment is happening at the city level), you can present a balance table showing that your treatment and control groups are similar on observable characteristics. The joint test you posted can also be helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.