Multiple regression and "holding variables fixed"

Question

Suppose that we regress a variable $Y$ on two independent variables $X_1$ and $X_2$. Since we have included $X_2$ in the regression, the obtained $\hat{\beta_1}$ coefficient is often said to be the effect of $X_1$ on $Y$ ‘holding $X_2$ fixed’ or 'controlling for $X_2$'. This is then said to eliminate any omitted variable bias that is due to $X_2$.

To investigate this more closely, observe that we can hold $X_2$ fixed at a variety of different values. Restrict attention to those values that reappear at least once across the dataset (e.g. because there are two datapoints for which $X_{2i} = 5$). In such cases, we can conduct a simple regression of $Y$ on $X_1$ solely using those datapoints for which (say) $X_{2i} = 5$. We can then repeat this regression for all possible $X_2$ values (assume they reappear at least once). It is natural to conjecture that our obtained $\beta_2$ estimate should be similar (identical?) to the average of the $\beta_2$ coefficients obtained from our simple regressions. Is this indeed the case?

Answer 1

The language "holding fixed other regressors" is heuristic (but not problematic). Consider $$y_i =\beta_0 +\beta_1 x_{1i} +\beta_2 x_{2i}+ u_i$$

The OLS estimate of $\beta_1$ is the same as from the regression,

$$y_i =\beta_0 +\beta_1 \widetilde{x_{1i}}+ u_i$$

where $\widetilde{x_{1i}}$ are residuals from $x_{1i} =\delta_0 +\delta_1 x_{2i}+\widetilde{x_{1i}}$. This is an application of the Frisch-Waugh-Lovell theorem.

Because OLS residuals are by construction uncorrelated with regressors, we know that $Cov(\widetilde{x_{1i}}, x_{2i})=0$. Thus, what OLS is really doing is creating a version of a regressor that is uncorrelated with the controls ($\widetilde{x_{1i}}$ above), and using variation in that version. When that version of the variable changes, the other controls don't change on average (due to uncorrelatedness) and are "held fixed".

Some formal derivations exist that are more in the style of what you are asking. In chapter 3 of Mostly Harmless Econometrics, Angrist and Pischke consider the setting in which $x_{1i}$ is binary. Define $\delta_v$ as the average effect of $x_1$ on $y$ when $x_{2i}=v$

$$\hat{\beta_1} = \frac{\sum_{v}\delta_v[P(x_{1i}=1|x_{2i}=v)(1-P(x_{1i}=1|x_{2i}=v))]P(x_{2i}=v)}{\sum_{v}[P(x_{1i}=1|x_{2i}=v)(1-P(x_{1i}=1|x_{2i}=v))]P(x_{2i}=v)}$$