Consider a regression model on the form:

$y_{i} = \alpha + \beta_{1}X_{i} + u_{i}$ (1)

I am given $var(u_{i}|X_{i}) = 10$ and I know the OLS estimates for $\alpha$ and $\beta$ and using this I have to calculate the standard errors of the parameter estimates.

I am following Hayashi's book and I know that

$se(b_{k}) = \sqrt(s^{2}(X'X)^{-1}_{kk})$

Where $b_{k}$ is the parameter estimate of interest. I know $(X'X)^{-1}$ but what is confusing me is going from the given model to expressing it in matrix notation and that this does not use the variance of the error term.

I can express (1) as $Y = X\beta + u$ and since $var(u_{i}|X_{i}) = var(y_{i} - \alpha - \beta_{2}X_{i}|X_{i})$ but is this equal to $var(Y - X\beta|X)$?

If so, I can express $var(u_{i}|X_{i}) = 10$ as $\sigma^{2}I_{n}$ where $\sigma^{2} = 10$. And then could I use the fact that

$var(b_{k}|X) = (X'X)^{-1}X' \sigma^{2} I_{n} X(X'X)^{-1}$

To find the standard errors of interest.


1 Answer 1


Your notation is a bit all over the place, so I'm going to try and standardize it for a general case.

Let $X$ be the regression matrix (which includes a column of 1s for the intercept term) and $\beta$ be the vector of coefficients to be estimated via OLS (which includes the intercept term). $Var(u_i|X) = 10 = \sigma^2$, for all $i$. Thus the errors are homoscedastic (constant variance) and the variance-covariance matrix of the errors $\Omega$ has diagonal entries that are all equal to $\sigma^2$. Furthermore, if you assume that the error terms are not serially correlated, then the off-diagonal covariance terms $Cov(u_i, u_j|X)$ for all $i \neq j$ in the variance-covariance matrix $\Omega$ are zero.

Those are two of the standard Gauss-Markov assumptions used to establish the BLUEness of the OLS estimator. Under these assumptions, $\Omega$ is

$$ E[uu'|X] = \left[ \begin{array}{ccccc} \sigma^2 \\ & \sigma^2 & & \huge0 \\ & & \ddots \\ & \huge0 & & \sigma^2 \\ & & & & \sigma^2 \end{array} \right] = \sigma^2I. $$

To get the variance of the OLS estimates $b = \hat{\beta}$, first note that

$$ \begin{align} b &= (X'X)^{-1}X'y \\ &= (X'X)^{-1}X'(X\beta + u) \\ &= \beta + (X'X)^{-1}X'u \\ \implies b - \beta &= (X'X)^{-1}X'u, \end{align} $$

using $y = X\beta + u$. Then,

$$ \begin{align} Var(b|X) &= E[(b-\beta)(b-\beta)'|X] \\ &= E[(X'X)^{-1}X'u((X'X)^{-1}X'u)'|X] \\ &= E[(X'X)^{-1}X'uu'X(X'X)^{-1}|X] \\ &= (X'X)^{-1}X'E[uu'|X]X(X'X)^{-1}. \end{align} $$

But recall that we derived $E[uu'|X] = \sigma^2 I$ via the Gauss-Markov assumption of spherical errors. Thus, by substitution,

$$ \begin{align} Var(b|X) &= (X'X)^{-1}X'E[uu'|X]X(X'X)^{-1} \\ &= (X'X)^{-1}X'(\sigma^2 I)X(X'X)^{-1} \\ &= \sigma^2 (X'X)^{-1} X'X(X'X)^{-1} \\ \implies Var(b) &= \sigma^2 (X'X)^{-1}. \end{align} $$

The standard deviation of $b$ is just the square root of the variance, or

$$ sd(b) = \sqrt{\sigma^2 (X'X)^{-1}}. $$

To find the standard deviation of the $k^{th}$ estimated coefficient in $b$, $b_{k}$, we simply extract the $k^{th}$ diagonal element of $(X'X)^{-1}$, denoted as $(X'X)_{kk}^{-1}$:

$$ sd(b_k) = \sqrt{\sigma^2 (X'X)_{kk}^{-1}}. $$

$\sigma^2$ is the (common) variance of the errors and, unfortunately, this value is unobserved in our sample. In your question, however, it appears that this value is just given to you directly — it's $10$. In general, however, $\sigma^2$ must be estimated using the data. It turns out that given homoscedastic errors, an unbiased estimator of $\sigma^2$ is

$$ s^2 = \frac{e'e}{n-P}, $$

where $n$ is the number of observations, $P$ is the number of columns in $X$, and $e = \hat{u}$; that is, the vector of residuals $y - Xb$. $s$, incidentally, is called the "standard error of the regression", not to be confused with the standard errors of the OLS estimates.

Thus, the standard error of $\mathbf{b_k}$ is estimated as

$$ \boldsymbol{\widehat{se}(b_k) = \sqrt{s^2 (X'X)_{kk}^{-1}}}, $$

which is Hayashi's result.

  • 1
    $\begingroup$ Thank you so much for the thorough explanation! Really cleared things up. $\endgroup$
    – user11767
    Oct 27, 2018 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.