3
$\begingroup$

One of the concerns that one can have regarding the use of RCTs for evaluating policies is that while when done in small scales, the RCTs may show the result in one direction, scaling the policies may involve general equilibrium effects that may possibly make the relationship of interest nil or even the opposite of what was found in the small scale RCTs.

Several possible examples, intuitively, can be thought of immediately.

But I would like to have some real world examples where this has happened. Links to any relevant papers that demonstrate this would be helpful.

Improve this question
$\endgroup$
5
  • $\begingroup$ What examples do you have in mind as you have said that "Several possible examples, intuitively, can be thought of immediately"? $\endgroup$
    – Amit
    Apr 29 at 11:49
  • $\begingroup$ Umm one example can be: we do an RCT where the treatment is a job training programme. The treatment group's income increases as a result of the treatment when it is small, but as we scale it economy-wide, trainees lose the competitive advantage they have. $\endgroup$
    – Ishan Kashyap Hazarika
    Apr 29 at 19:23
  • $\begingroup$ Another can be, when a new farming technique when tried on a small treatment group leads to increase in yields, but when implemented in the entire region, the land quality may degrade leading to probably even a fall in the average yield $\endgroup$
    – Ishan Kashyap Hazarika
    Apr 29 at 19:27
  • $\begingroup$ Or a sales idea that when implemented by a small treatment group leads to rise in sales but when everybody adopts it, we are back to square one $\endgroup$
    – Ishan Kashyap Hazarika
    Apr 29 at 19:30
  • $\begingroup$ You're right - such a thing can happen. The treatment effects of a policy on a small sample can be misleading if we aim to use the results to implement the policy on a large scale. To avoid this, we need to take into account the equilibrium while measuring these effects. $\endgroup$
    – Amit
    Apr 30 at 23:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.