I have always thought that variation in regressors are a good thing. In fact, one can show that the precision of the estimated coefficients is increasing in the variance of the regessors. I have also heard that simple randomized controlled trials (RCTs) are very good to gauge causal effects. Consider the following regression:$$ Y_{i}=\alpha+\beta X_{i}+\epsilon_{i} $$ where $X_{i}$ is a dummy variable that takes on the value 1 if the individaul $i$ is treated and 0 otherwise. My question is as follows: in this setting, $var(X)=p(1-p)$, This value takes on a maximum at 0.25. I do not think this variance is that 'large.' Can someone please clarify if this is indeed then a shortcoming? Given that we do not observe that much variation at all, how are we able to gauge so much ? Is it because the variation is random? Thanks!

  • 2
    $\begingroup$ What criteria are you using in order to conclude that the variance "is not that large"? No number can be evaluated alone, but only in relation to something else. $\endgroup$ Oct 25, 2015 at 21:09
  • $\begingroup$ Thanks. I guess my question will then boil down to comparing variances of discrete variables with those of continuous variables. Are they really comparable? Also, in a general bivariate regression, we force the regressor to have a single coefficient for its entire domain. In this case, the model is really completely non parametric because we allow for a separate regressor for a different value of the same variable! Rephrasing my question- can we compare a regression with a dummy variable to that with a continuous one? $\endgroup$
    – ChinG
    Oct 25, 2015 at 21:27
  • $\begingroup$ @ChinG If you want to rephrase your question consider editing it. Yes, you can compare the variations. It is probably meaningless, as variation is dependent on the unit of measurement (km, mile, cm). A similar question might be: you think data sets with "high" variation of regressors provide better fit. (This is not always true. I can construct data with high variation of $X$ and $R^2 =0$.) How can we measure "high" variation given that the regressors measure totally different things? And I think the maximum variation of the dummy $X_i$ is at $p = 0.5$, not $p=0.25$. $\endgroup$
    – Giskard
    Oct 26, 2015 at 9:32
  • $\begingroup$ Sure- I agree with your R2 argument. I should have mentioned- it can be shown that holding the estimated variance of the residuals constant, the estimated variance of the point estimate is decreasing in the variance of the regressor in question. And yes the maximum variation is at p=0.5 which yields a maximum variance of 0.25. I was a bit unclear in my wording. $\endgroup$
    – ChinG
    Oct 26, 2015 at 14:18


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.