I have a set of results from a multiple regression table with four columns. In each column, a new variable is added. Why, however, is the initial variable coefficient different in each column? Is it due to the coefficients being jointly estimated?
1 Answer
Yes, it is because they are estimated jointly and the point estimates are conditional on other point estimates and also because of omitted variable bias. To be more specific, contrast an example of univariate OLS with bivariate OLS, both of which are estimated by minimizing the sum of squared residuals, but resulting formulas for estimated coefficients are different.
In univariate OLS by minimizing squared residuals we get
$$\min_{b_0,b_1}\sum e^2 = \sum (y_i-b_0-b_1x_{1i})^2 \implies \hat{b_1} = \frac{\sum (x_{1i}-\bar{x_1})(y_i-\bar{y})}{\sum (x_{1i}-\bar{x_1})^2}$$
In a bivariate regression the result will be different
$$\min_{b_0,b_1, b_2}\sum e^2 = \sum (y_i-b_0-b_1x_{1i}-b_2x_{2i})^2 \implies \hat{b_1} = \frac{\sum (x_{2i}-\bar{x_2})^2\sum (x_{1i}-\bar{x_1})(y_i-\bar{y})- \sum (x_{1i}-\bar{x_1})(x_{2i}-\bar{x_2})\sum (x_{2i}-\bar{x_2})(y_i-\bar{y})}{\sum (x_{1i}-\bar{x_1})^2 \sum (x_{2i}-\bar{x_2})^2- \sum (x_{1i}-\bar{x_1})(x_{2i}-\bar{x_2})}$$
Hence we can see that the difference is that in the second expression the calculation of $b_1$ critically depends on what the covariance of $x_1$ and $x_2$ is (i.e. $\sum (x_{1i}-\bar{x_1})(x_{2i}-\bar{x_2})$), what variance of $x_2$ is (i.e. $\sum (x_{2i}-\bar{x_2})^2$ and also what covariance of $x_2$ and $y$ is (i.e. $\sum (x_{2i}-\bar{x_2})(y_i-\bar{y})$) and hence $\hat{b_1}$, hence including additional variable will change the $b_1$ (and also other coefficient) formula. This will extend to multiple variables as well with formula getting progressively more complex.
Moreover, the coefficients will change substantially especially if there is some omitted variable bias present. If we assume that the true model is given by:
$$y= \beta_0 + \beta_1 x_1 +\beta_2 x_2 + u$$
but you will try to fit univariate model regardless of the above.
$$\hat{y} = \hat{\beta_0} + \hat{\beta_1} x_1$$
In such conditions it can be shown that even though we fit univariate regression the expected estimated beta will be (see Wooldridge Introduction to Econometrics):
$$E[\hat{\beta_1}]= \beta_1 + \beta_2 \frac{\sum (x_{1i}-\bar{x_1})(x_{2i}-\bar{x_2})}{\sum (x_{1i}-\bar{x_1})^2}$$
Where the second term is the omitted variable bias. The intuition here is that if there is any covariance between $x_1$ and $x_2$ and if $y$ actually depends also on $x_2$ (i.e. $\beta_2 \neq 0$) then if you apply univariate regression on bivariate relationship the estimate of $\beta_1$ will be biased as it partially captures the relationship of $y$ and $x_2$. For example, if $\beta_1=0.1$, $\beta_2= -2$ and $cov(x_1,x_2)=0.5$ and $var(x_1)=2$ then if you estimate univariate regression the estimated $\hat{beta_1} = -0.4$ whereas in bivariate regression it would be $\hat{\beta_1}=0.1$ and $\hat{\beta_2}=-2$
Hence to sum up, the coefficients are different because they are in multivariate regression calculated conditional on other coefficients and also if you are omitting an relevant independent variable its effect will be 'hidden' in the estimates of non-omitted variable (if they are correlated).
-
$\begingroup$ Thank you so much for your response! It was very clear. If I could also ask a follow-up question based on your response, that would be extremely helpful. In the second column of my regression results, the first new variable added is a 'female dummy variable'. The other variable from the first column increased in value though. I would have expected a parameter estimate of a given variable to decrease if a dummy variable was added. Am I misunderstanding something? $\endgroup$ Aug 10, 2020 at 6:19
-
$\begingroup$ @EstherBlaese you are welcome. Yes this can happen for example the hidden effect of omitted variable can be negative but if also the covariace between first and second variable is negative the two minuses cancel each other out. For example, if true b_1=2 true b_2 =-1, cov(x1,x2)=-1, and var(x1)=1 the estimated b_1 with omitting second variable would become 3. This is why omitted variable bias is so nasty - it messes with your estimates in unexpected and unintuitive ways $\endgroup$– 1muflon1 ♦Aug 10, 2020 at 11:06
-