2
$\begingroup$

Consider the standard linear regression model: $y_i = \alpha + \beta D_i + e_i$ where the coefficients are defined by linear projections and $D_i$ is a dummy variable. In the population, the coefficients are given by:

$$\alpha = E[y_i \mid D_i =0] \ \text{and} \ \beta = E[y_i \mid D_i = 1] - E[y_i \mid D_i =0]$$

Using OLS to estimate the coefficients, we get:

$$\widehat{\alpha} = \overline{y}_{D_i=0} $$

$$\widehat{\beta} = \overline{y}_{D_i=1}-\overline{y}_{D_i=0} $$

In other words, $\widehat{\alpha}$ is just the sample mean of $y_i$ in the subsample with $D_i=0$ and $\widehat{\beta}$ is the difference in sample means of the two groups. The expressions seem very obvious because they are just sample versions of the population, but my question is, how can we arrive at the above coefficient estimates by using the standard OLS formulas? That is using:

$$\widehat{\alpha} = \overline{y} - \overline{D}\widehat{\beta} \ \ \text{and} \ \ \widehat{\beta} = \frac{\sum_{i=1}^{N}(D_i - \overline{D})(y_i - \overline{y})}{\sum_{i=1}^{N}(D_i - \overline{D})^2}$$

$\endgroup$
2
$\begingroup$

To start, let's derive $\hat{\beta}$. As you note:

$$\hat{\beta} = \frac{\sum(D_i - \bar{D})(y_i - \bar{y})}{\sum (D_i - \bar{D})^2} $$

The key to evaluating this is to recognise that $D_i$ is a binary variable that equals $0$ or $1$. Thus, we can use standard results on Bernoulli random variables.

To evaluate the numerator, note that if $D$ is Bernoulli distributed:

$$ \mathrm{Cov}[D, y] = \mathbb{E}[Dy] - \mathbb{E}[D]\mathbb{E}[y] = \mathrm{P(D=1)}\mathbb{E}[y|D=1] - \mathrm{P(D=1)}\mathbb{E}[y] $$

By the sample analogue of this:

$$ \frac{\sum(D_i - \bar{D})(y_i - \bar{y})}{n} = \bar{D}(\bar{y}_{D_i=1}-\bar{y})$$

To evaluate the denominator, note that if $D$ is a Bernouilli random variable:

$$\mathrm{Var[D]} = \mathrm{P(D=1)}[1-\mathrm{P(D=1)]}$$

Sample analogue:

$$\frac{\sum (D_i - \bar{D})^2}{n} = \bar{D}(1 - \bar{D})$$

We therefore see that:

$$\hat{\beta}=\frac{\bar{D}(\bar{y}_{D_i=1}-\bar{y})}{\bar{D}(1 - \bar{D})} = \frac{\bar{y}_{D_i=1}-\bar{y}}{1 - \bar{D}} \\ = \frac{\bar{y}_{D_i=1}-(\bar{y}_{D_i=1}\bar{D}+\bar{y}_{D_i=0}(1-\bar{D}))}{1 - \bar{D}} \\ = \bar{y}_{D_i=1} - \bar{y}_{D_i=0}$$

which is what we wanted to show.

It is then trivial to derive:

$$\hat{\alpha} = \bar{y} - \bar{D}\hat{\beta} =\bar{y}_{D_i=1}\bar{D}+\bar{y}_{D_i=0}(1-\bar{D}) - \bar{D}(\bar{y}_{D_i=1} - \bar{y}_{D_i=0}) = \bar{y}_{D_i=0}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.