1
$\begingroup$

Consider the following linear model: $Y = X\beta + u$ .

If Gauss-Markov assumptions hold, I'm trying to prove that $\hat{\sigma}^2 = \frac{\hat{u}'\hat{u}}{N-K-1}$ is an unbiased estimator for the error variance $\sigma^2$. How can I prove this?

My attempt:

I tried with:

$E(\hat{\sigma}^2) = \frac{E(\hat{u}'\hat{u})}{N-K-1} =\frac{E[(Y- X\hat{\beta})'(Y- X\hat{\beta})]}{N-K-1} = \frac{E(Y'Y) - E(Y'X \hat{\beta}) -E(\hat{\beta}X'Y + E(\hat{\beta}X'X \hat{\beta})}{N-K-1}$.

How can I calculate those expectations?

$\endgroup$
1
  • 1
    $\begingroup$ Pls, see the updated question above $\endgroup$
    – Mark
    Dec 22, 2023 at 10:10

1 Answer 1

3
$\begingroup$

To prove the unbiasedness of $\hat{\sigma}^2$ when conditioned on $X$, that is, $\mathbb{E}(\hat{\sigma}^2|X)=\sigma^2$, consider $\hat{u}= y- X \hat{\beta}=y-X(X'X)^{-1} X'y = [I_n -X(X'X)^{-1} X'X]y = My$, and using $y=X\beta +u$, write $\hat{u}=Mu$, since $MX=0$.

Just a remark about the $M$ matrix: $M$ is a matrix that is both symmetric ($M=M'$) and idempotent ($M=M^2$). The matrix $M$, named the residual maker, can be interpreted as a matrix producing the (column) vector of least squares residuals in the regression of $y$ on $X$ when it premultiplies any vector $y$.

Then, \begin{equation*} \hat{u}' \hat{u} = u'M' M u = u' M u \end{equation*} Thus, $u' M u$ is a scalar and equals its trace that we denote as $tr(u' M u)$.

Then, recall that tr(ABC)=tr(BCA), and under the homoskedasticity assumption $\mathbb{E}(uu'|X)=\sigma^2$, \begin{align*} \mathbb{E}(u'M u |X) = & \mathbb{E}(tr(u'Mu|X)) = \mathbb{E}[tr(Muu')|X] \\ = & tr[M \mathbb{E}(u u'|X)] = tr(M \sigma^2 I_n) \\ = & \sigma^2 tr(M)= \sigma^2 (n-k-1) \end{align*} The last equality comes from the fact that $tr(M)=tr(I_n)-tr[X(X'X)^{-1} X'] = n - tr[(X'X)^{-1}X'X]$, and this yelds $n-tr(I_{k+1})=n-k-1$. Eventually, \begin{equation} \mathbb{E}(\hat{\sigma}^2|X)= \mathbb{E}(u'Mu|X)/(n-k-1)=\sigma^2 \end{equation}

To find references where the properties of trace in linear algebra are discussed in a simple manner, consult the appendix of "Introductory Econometrics: A Modern Approach" by Wooldridge (latest edition)

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.