For instance, consider the following regression equation $College Score=\beta_0+\beta_1HighSchool GPA+u$. We interpret this $\beta_1$as the effect of a higher High School GPA holding all other variables constant. The second regression model adds a new term and becomes $College Score=\beta_0+\beta_1HighSchool GPA+\beta_2ACTSCORE+u$. The interpretation of $\beta_1$ is still the effect of a higher High School GPA holding all other variables constant. So what is the difference?(and the values in two regressions are different). The textbook explanation is that, in the first equation, ACT score is assumed to be fixed. In the second equation, however, ACT score is explicitly controlled. I don't quite understand the real difference between 'assumed fixed' and 'explicitly held fixed'.
-
$\begingroup$ This is a deep question. I now think $E(y|x_1)=\beta_0+\beta_1 x_1$ and $E(y|x_1,x_2) = \beta_0 + \beta_1 x_1 + \beta_2 x_2$ are a lot easier to swallow. To me, interpreting $\beta_1$ of the first model as "holding other variables constant ..." is not very meaningful because it's impossible to hold $u$ fixed. In fact, we don't even know what $u$ is in some (many) cases. $\endgroup$– chan1142Commented Jan 17, 2020 at 5:13
1 Answer
The difference is that the first regression is unbiased only if you can assume that high school GPA and ACT score are orthogonal on each other $cov(x,z)=0$ where $x$ is shortcut for high school GPA and $z$ for ACT score. Or if you can assume the second variable ATC score does not affect the dependent variable at all $\beta_2=0$. This is because in simple regression the $\beta$ coefficient bias is given by:
$$\hat{\beta_1} = \beta_1 + \beta_2 \frac{\text{COV}(x,z) }{VAR(x)} $$
where $\hat{\beta_1} $ is the estimated beta effect of GPA on col. score (beta reported by your statistical program like R), $\beta_1$ is the true unobserved coefficient, $\beta_2$ is the true unobserved effect of ACT on score, and the fraction is just covariance between $x$ and $z$ over variance of $x$.
So unless you know that the two variables are completely unrelated you will have bias. Now an example of variable that is completely unrelated to another is variable thats hold fixed. For example if in $z(x) =100$ no matter which $x$ we examine then we can be confident that $cov(x,z)=0$. Regressions should be applied only if you think they are unbiased so the first equation implicitly assumes that the second variable is hold constant given $x$. However, once you include the second covariate explicitly you can calculate both expected beta coefficients conditional on each other which is equivalent to actually keeping the other variable constant.
So there is actually a great deal of difference between the two regressions (except for special cases where $cov(x,z)=0$ or $\beta_2=0$).
-
1$\begingroup$ thank u! that really answers my question. $\endgroup$ Commented Jan 17, 2020 at 13:26
-
$\begingroup$ @ChenliZhou I am glad that I could help. If you are satisfied with the answer please don’t forget to accept it $\endgroup$– 1muflon1 ♦Commented Jan 17, 2020 at 13:30