2
$\begingroup$

We know that selection bias occurs when the treatment and control groups are not comparable, leading to differences in the outcome that are not solely due to the treatment.

How does one address the problem due to selection bias in a diff-in-diff setting? Or is DiD not suitable for any study that faces selection bias problem? What other econometric methods can give credible causal estimates of the intervention in the above case?

First edit: I think i didn't make my question precise. I am interested in effects of a policy on an outcome. The treatment group in my case is nonrandom (hence selection bias) in the sense that it is decided before the actual treatment based on their observed covariates like income. The interventation didn't create a treatment group; the group was formed based on its characteristics. How does one address this nonrandom-assignment-led selection bias? Is DiD useful? What are alternatives if not?

$\endgroup$
0

2 Answers 2

2
$\begingroup$

As the other answer mentioned, if the Diff-in-diff assumptions hold, it should not be a problem.

However, if you think selection might be invalidating your diff-in-diff assumptions, you have two routes:

  1. Argue that the selection is not invalidating your assumptions. For more see here: Is Difference-in-Differences still valid when treat or control is determined by subjects?

  2. Try to correct the selection in the econometric specification.

To do this, some options are:

  1. Try to add controls to the regression to ensure the assumptions work, conditional on those controls, as the other answer says.

  2. Use matching methods in combination with diff-in-diff to make sure the assumption still holds. E.g. one short thread is here: Is matching combined with Diff-in-Diff a bad idea?, but there are a lot of resources on this

  3. Try to model the selection and use a Heckman correction, but this has fallen out of fashion, has numerous problems and I would not recommend it. For more on this, see e.g. here: https://stats.stackexchange.com/questions/38853/heckman-selection-model-with-difference-in-differences-specification

$\endgroup$
8
  • $\begingroup$ Thanks, this is helpful. For matching, I need to use Propensity Score Matching, Synth Control. What else? $\endgroup$
    – funcard
    Mar 27, 2023 at 20:22
  • $\begingroup$ Those are the main two that you can focus on, I wouldn't go with anything else on matching, unless you really want to. $\endgroup$
    – BB King
    Mar 27, 2023 at 20:29
  • $\begingroup$ What additional methods of robutness checks (beyond PSM and Synth Control) would be necessary to ensure that nonrandom treatment assignment is not biasing the results? $\endgroup$
    – funcard
    Mar 27, 2023 at 20:40
  • $\begingroup$ As I mentioned, there is not really anything else. But robustness checks does not just mean removing bias. In practice, a lot of it is showing bias is not really there. There are 4 broad ways to do that in the answer I linked under my very first bullet: "Is Difference-in-Differences still valid when treat or control is determined by subjects?". $\endgroup$
    – BB King
    Mar 27, 2023 at 20:44
  • $\begingroup$ Got it. Trying to make sure I am fully clear on the next step. I'm not fully sure if the question/problem (seems to be self-selection bias problem to me) of the post you mentioned is the same as my problem. In my case, it's more like: the intervention affected only people having 3 or more children. So, I am saying that the people in this category are in treatment group, and people who don't have children are in the 'donor pool' to form a comparison group. Would you still say matching is the way forward? $\endgroup$
    – funcard
    Mar 27, 2023 at 21:00
4
$\begingroup$

The parallel trends assumption of Diff-in-diff is essentially an assumption that there is no selection bias. If you are curious about the parallel trends assumption, I encourage you to ask about it in a separate question.

In context of diff-in-diff, we may believe parallel trends is true only conditional on controls, in which case we can add in controls to the regression when we estimate the diff-in-diff.

Apart from diff-in-diff, there are other methods: Instrumental variables, regression discontinuity, synthetic controls, and fixed effects/first differences.

$\endgroup$
3
  • $\begingroup$ I upvoted this answer because its mostly correct, but I think the first paragraph should be bit adjusted, parallel trend does not mean there is no selection bias, that can be still present in the level of a variable but with diff-in-diff you look at differences in differences, so even if there is selection bias in the level as long as the variable behaves similarly for treated and untreated there won't be bias when you compare the changes at point of treatment $\endgroup$
    – 1muflon1
    Mar 27, 2023 at 9:20
  • $\begingroup$ Yes, I understand both the answer and the comment. I think i didn't make my question precise. I am interested in effects of a policy on an outcome. The treatment group in my case is nonrandom (hence selection bias) in the sense that it is decided before the actual treatment based on their observed covariates like income. The interventation didn't create a treatment group; the group was formed based on its characteristics. How does one address this nonrandom-assignment-led selection bias? Is DiD useful? What are alternatives if not? $\endgroup$
    – funcard
    Mar 27, 2023 at 18:28
  • $\begingroup$ 1muflon1 we may be using different terminology. I believe you are saying that, even if parallel trends holds, there can be selection into treatment in the sense that a variable can be of different levels between treated and control. Sure. I wouldn't call that "selection bias" though, because "bias" implies the point estimate has bias. At any rate, I think we both understand the math and econometrics underlying the idea. $\endgroup$ Mar 28, 2023 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.