# Including an endogenous covariate in a regression model as a control to estimate the effect of another variable of interest

I am interested in the effect of an independent variable $$x$$ on a dependent variable $$y$$, like so

$$y = \beta_0 + \beta_1 x + e$$

where $$e$$ is the error term. Now $$x$$ includes two effects $$z_1$$ and $$z_2$$. For simplicity, let say $$x = z_1 + z_2$$. If I include $$z_1$$ in the model, like this

$$y = \beta_0 + \beta_1 x + \beta_2 z_1 + e$$

Does that mean that $$\beta_1$$ is predominantly capturing the effect of $$z_2$$?

And a follow up question is: if $$z_1$$ is correlated with the error term, will including it in the model bias the estimate of $$\beta_1$$?

If I include $$z_1$$ in the model, like this: $$> y = \beta_0 + \beta_1 x + \beta_2 z_1 + e, >$$ Does that mean that $$\beta_1$$ is predominantly capturing the effect of $$z_2$$?

Yes. This can be seen using the Frish-Waugh-Lovell theorem:

If you regress: $$y = \beta_0 + \beta_1 x + \beta_2 z_1 + e,$$ then $$\beta_1$$ will be the same as the corresponding coefficient of a modified regression: $$\hat y = \gamma_0 + \beta_1 \hat x + \hat e \tag{1}$$ where $$\hat x$$ is the residual from regressing $$x$$ on $$z_1$$ and the same for $$\hat y$$. Now if we regress $$x$$ on $$z_1$$ then the residual is equal to: $$M_{z_1} x,$$ where $$M_{z_1} = 1 - z_1(z_1'z_1)^{-1}z_1'$$ is the annihilator matrix. If $$x = z_1 + z_2$$ then: $$M_{z_1} x = (1 - z_1(z_1'z_1)z_1')(z_1 + z_2) = M_{z_1}z_2$$ As such, substituting in $$(1$$), we have: $$\hat y = \gamma_0 + \beta_1 \hat z_2 + \hat e,$$ where $$\hat z_2$$ is now the residual from regressing $$z_2$$ on $$z_1$$. Using the Frish-Waugh-Lovell theorem, in the reverse direction, this gives that $$\beta_1$$ is also equal to the coefficient in the following regression: $$y = \delta_0 + \beta_1 z_2 + \delta_2 z_1 + \varepsilon.$$ In other words, $$\beta_1$$ will also be equal to the coefficient for $$z_2$$ for a regression of $$y$$ on both $$z_2$$ and $$z_1$$. Notice, however, that in general $$\beta_2 \ne \delta_2$$ (so the coefficients for $$z_1$$ will not be equal in the two regressions).

Another way to see this is by immediately substituting $$x = z_1 + z_2$$ into the regression $$(1)$$ then: \begin{align*} y &= \beta_0 + \beta_1(z_1 + z_2) + \beta_2 z_1 + e,\\ &= \beta_0 + \beta_1 z_2 + (\beta_1 + \beta_2) z_1 + e. \end{align*} So the coefficient on $$z_2$$ in the new regression is identical to the coefficient on $$x$$ in the original one $$(\beta_1)$$ , while the coefficient on $$z_1$$ in the new regression is the sum of the coefficients on $$x$$ and $$z_1$$ in the original regression $$(\beta_1 + \beta_2)$$.

And a follow up question is: if $$z_1$$ is correlated with the error term, will including it in the model bias the estimate of $$\beta_1$$?

The opposite is true. Including $$z_1$$ into the regression will make the estimate of $$\beta_1$$ unbiased. Consider the following data generating process: $$y = \beta_0 + \beta_1 x + e, \tag{2}$$ and assume that $$e$$ is correlated with $$z_1$$. Then we can write: $$e = \gamma z_1 + \varepsilon.$$ where $$\varepsilon$$ is now uncorrelated with $$z_1$$. (and where $$\gamma = \mathbb{E}(e z_1)/\mathbb{E}((z_1)^2) \ne 0$$. Assume for simplicity that $$e$$ is uncorrelated with $$z_2$$.

Then the estimate of $$\beta_1$$ will be biased as the orthogonality condition $$\mathbb{E}(e x) = 0$$ is not satisfied. Indeed: \begin{align*} \mathbb{E}(ex) &= \mathbb{E}(e z_2) + \mathbb{E}(\gamma z_1 z_1) + \mathbb{E}(\varepsilon z_1),\\ &= \mathbb{E}(\gamma (z_1)^2) \ne 0 \end{align*} If we include $$z_1$$ into the regression. Then substituting $$e = \gamma z_1 + \varepsilon$$, into $$(2)$$ we can write: $$y = \beta_0 + \beta_1 x + \gamma z_1 + \varepsilon.$$ And $$\mathbb{E}(\varepsilon) = \mathbb{E}(\varepsilon x) = \mathbb{E}(\varepsilon z_1) = 0$$. So by including $$z_1$$ into the regression, we can guarantee the residual $$\varepsilon$$ to be uncorrelated with all covariates. This means that $$\beta_1$$ is identified and its estimate will be unbiased.

• Regarding Another way to see this..., if I simulate the data according to $y=\beta_0+\beta_1x+e$ with $x=z_1+z_2$ and then regress $y$ on $x$ and $z_1$, the estimated coefficient on $z_1$ is around zero and insignificant -- and far from $\beta_1+\beta_2=\beta_1+0=\beta_1$. Is that in line with what you are saying? Jun 29, 2021 at 12:00
• @Richard Hardy What I am saying is that if you first regress $y$ on $x$ and $z_1$ and then $y$ on $z_2$ and $z_1$, then the coefficient on $z_1$ for the second regression is the sum of the coefficients for $x$ and $z_1$ of the first regression. In addition, the coefficient on $x$ in the first regression should be the same as the coefficient on $z_2$ in the second regression. In your case the coefficient on $z_1$ is zero as it is zero in your data generating process.
– tdm
Jun 29, 2021 at 12:53
• OK, thanks! I think my case is the same as OP's case, as I am following the setup in there. But even if that is not the case, you explanation is helpful. Jun 29, 2021 at 13:52