Take the 2-minute tour ×
Economics Stack Exchange is a question and answer site for professional and academic economists and analysts. It's 100% free, no registration required.

Suppose you are running a randomized experiment to assess the effect of $X$, say some training program for unemployed people, on $Y$, say the chance of finding a job in the coming year. Suppose also that $X$ takes time : maybe it lasts for several month.

Because you randomize, you do not need to worry about self-selection bias initially. But during the course of $X$, some people will likely realize that $X$ is beneficial to them, and others may realize that they are wasting their time.

As a result, one might expect that among people who drop from the program, there is a higher proportion of agents for which the treatment effect would have been smaller. This might induce an over-estimation bias of the treatment effect.

My questions are :

  • Is this kind of bias discussed in the literature on randomized experiments?
  • Does it have a canonical name ?
  • Do researcher try to control for this, and if yes, how?
share|improve this question

3 Answers 3

up vote 9 down vote accepted

Apparently this is called attrition bias. It's very similar to survivorship bias. This paper suggest correcting for it using Heckman correction. Propensity score matching may also help somewhat. My experience with both has been mixed, but they are commonly used. You should figure out what exact approach is most appropriate for your setting.

One last edit: These two papers, which talk about bounding the average treatment effect, may also be of use to you.

share|improve this answer
Great answer, thanks! –  Martin Van der Linden Nov 20 '14 at 3:03

I think this paper might be useful to you. It's a job market paper by one of Heckman's students at UChicago, named Rodrigo Pinto. The paper is titled "Selection Bias in a Controlled Experiment: The Case of Moving to Opportunity." In the MTO experiment, the voucher assignment mechanism was random but only approximately half that received the voucher ended up actually moving. He shows how to decompose the typical treatment-on-the-treated parameter into components that have unambiguous interpretations.

share|improve this answer

Another thing you can look at is "Intention-to-treat analysis". From Wikipedia,

An intention-to-treat (ITT) analysis of the results of an experiment is based on the initial treatment assignment and not on the treatment eventually received. ITT analysis is intended to avoid various misleading artifacts that can arise in intervention research such as non-random attrition of participants from the study or crossover.

This appears to match what you were looking for: your treatment is randomized initially and people drop out non-randomly.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.