1
$\begingroup$

in an RCT, I have 3 different treatment groups and one control group. The size of the control group is around 1000 while the size of other groups are just above 300.

To test balance I used ANOVA and Welch test, which show that several variables are unbalanced. Can I draw a smaller random sample (400) from the control group so that the sizes of all groups are relatively equal?

Actually after doing that only one variable is unbalanced. So would it cause any problems if I want to draw a smaller sample just for more balance? Thanks!

$\endgroup$
2
  • $\begingroup$ If you haven't already, recommend also posting this question on the statistics site: stats.stackexchange.com. $\endgroup$
    – AlexK
    Mar 25, 2019 at 6:26
  • $\begingroup$ There is already a similar question there with an answer that discourages this practice: stats.stackexchange.com/q/71933/241093 $\endgroup$
    – AlexK
    Mar 25, 2019 at 6:32

1 Answer 1

0
$\begingroup$

I would not do that. Your problem is that the treatment assignment is not independent of your covariates. As such, it is unlikely that the potential outcomes $Y_0$ (my outcome in the world "I take the treatment") and $Y_1$ (my outcome in the world "I do not take the treatment") are independent of treatment assignment $D\in\{0,1\}$.

What you can do is use estimation techniques that would allow you to validate the conditional independence assumption (i.e., given covariates $X$, treatment assignment $D$ is independent of potential outcomes $Y_0,Y_1$). Bear in mind that you can never claim you've reached independence as there will always be covariates you don't measure.

These techniques include: running an OLS with controls, matching treatment with control units with similar propensity scores, weighting with the propensity score etc.

Good luck!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.