Is Difference-in-Differences still valid when treat or control is determined by subjects?

Question

I have a question about the difference-in-differences method. I am considering using it to evaluate a policy, the UK's Help-to-Buy (H2B) equity loan scheme. The housebuilders can choose to join the H2B or not. Then, my idea is that the builders who participated in the H2B are in the treatment group, while others are in the control group. My question is: Would the choice of builders be an endogenous factor that invalidates the DiD setting?

My guess is not, as long as the parallel trends assumption is not obviously violated.

Answer 1

It is definitely an issue that needs to be considered.

So, you are right that if the parallel trends assumption is not violated the approach is fine. Note the assumption in other words is that "no time-varying differences exist between the treatment and control groups."

However, this assumption cannot be proven, so we never know if it is definitely violated or not. So even if the assumption is not "obviously" violated, a doubt that it could be violated is still enough to cast doubt on the final result.

Notably, saying your approach is fine, if the parallel trends assumption is not violated, is effectively assuming away the problem. That's because the self-selection into treatment and control group is likely to invalidate the assumption.

Because people who choose to participate are likely different from those that choose not to participate, it is possible that time-varying differences exist between the two groups. You would have to argue and show that these differences are unlikely. There are four ways to do this (source):

1. Compare changes in the outcomes for the treatment and control groups repeatedly before the program is implemented (i.e. in t-3, t-2, t-1). If the outcome trend moves in parallel before the program began, it likely would have continued moving in tandem in the absence of the program.

2. Perform a placebo test using a fake treatment group. The fake treatment group should be a group that was not affected by the program. A placebo test that reveals zero impact supports the equal-trend assumption.

3. Perform a placebo test using a fake outcome. A placebo test that reveals zero impact supports the equal-trend assumption.

4. Perform the difference-in-differences estimation using different comparison groups. Similar estimates of the impact of program confirms the equal-trend assumption.

Note that you should ideally use as many of these options as possible. In principle, it is not enough to only show past parallel trends, because the assumption is about counterfactual (future) parallel trends after treatment, in the absence of treatment.