# Linear Regression Model and estimators

Consider the following linear model: $$Y = X\beta + u$$ .

If Gauss-Markov assumptions hold, I'm trying to prove that $$\hat{\sigma}^2 = \frac{\hat{u}'\hat{u}}{N-K-1}$$ is an unbiased estimator for the error variance $$\sigma^2$$. How can I prove this?

My attempt:

I tried with:

$$E(\hat{\sigma}^2) = \frac{E(\hat{u}'\hat{u})}{N-K-1} =\frac{E[(Y- X\hat{\beta})'(Y- X\hat{\beta})]}{N-K-1} = \frac{E(Y'Y) - E(Y'X \hat{\beta}) -E(\hat{\beta}X'Y + E(\hat{\beta}X'X \hat{\beta})}{N-K-1}$$.

How can I calculate those expectations?

• Pls, see the updated question above
– Mark
Dec 22, 2023 at 10:10

## 1 Answer

To prove the unbiasedness of $$\hat{\sigma}^2$$ when conditioned on $$X$$, that is, $$\mathbb{E}(\hat{\sigma}^2|X)=\sigma^2$$, consider $$\hat{u}= y- X \hat{\beta}=y-X(X'X)^{-1} X'y = [I_n -X(X'X)^{-1} X'X]y = My$$, and using $$y=X\beta +u$$, write $$\hat{u}=Mu$$, since $$MX=0$$.

Just a remark about the $$M$$ matrix: $$M$$ is a matrix that is both symmetric ($$M=M'$$) and idempotent ($$M=M^2$$). The matrix $$M$$, named the residual maker, can be interpreted as a matrix producing the (column) vector of least squares residuals in the regression of $$y$$ on $$X$$ when it premultiplies any vector $$y$$.

Then, $$\begin{equation*} \hat{u}' \hat{u} = u'M' M u = u' M u \end{equation*}$$ Thus, $$u' M u$$ is a scalar and equals its trace that we denote as $$tr(u' M u)$$.

Then, recall that tr(ABC)=tr(BCA), and under the homoskedasticity assumption $$\mathbb{E}(uu'|X)=\sigma^2$$, \begin{align*} \mathbb{E}(u'M u |X) = & \mathbb{E}(tr(u'Mu|X)) = \mathbb{E}[tr(Muu')|X] \\ = & tr[M \mathbb{E}(u u'|X)] = tr(M \sigma^2 I_n) \\ = & \sigma^2 tr(M)= \sigma^2 (n-k-1) \end{align*} The last equality comes from the fact that $$tr(M)=tr(I_n)-tr[X(X'X)^{-1} X'] = n - tr[(X'X)^{-1}X'X]$$, and this yelds $$n-tr(I_{k+1})=n-k-1$$. Eventually, $$$$\mathbb{E}(\hat{\sigma}^2|X)= \mathbb{E}(u'Mu|X)/(n-k-1)=\sigma^2$$$$

To find references where the properties of trace in linear algebra are discussed in a simple manner, consult the appendix of "Introductory Econometrics: A Modern Approach" by Wooldridge (latest edition)