The asymptotic sampling distribution, after taking plim, of the OLS estimator is given by $\sqrt{N}(\hat{\beta}-\beta) = E[X_iX_i^T]^{-1} \left(1/\sqrt{N} \sum_{I=1}^NX_ie_i \right) $

It must be shown that the asymptotic variance can be written as: $ Var[E[X_iX_i^T]^{-1} 1/\sqrt{N} \sum_{I=1}^NX_ie_i] = \left( E[X_iX_i^T]^{-1} \right) \left( E[X_iX_i^T e_i^2]\right) \left( E[X_i X_i^T]^{-1} \right) $

Because the variance is linear in parameters, my idea was to start like this:

$V[\sqrt{N}(\hat{\beta}-\beta)] = \sqrt{N}Var[\hat{\beta}] + 0$

Because the $Var[\beta]=0$. Thus one could use the fact that $Var[\hat{\beta}]=E[(\hat{\beta}-\beta)(\hat{\beta}-\beta)^T]$.

However, while there seems to be a relationship I stuck at this point, mostly because of the $\sqrt{N}$. Thanks for help.

  • $\begingroup$ That is unreadable to me. Try using LaTeX surrounded by $ signs. And perhaps also what you have attempted yourself $\endgroup$ – Henry Oct 3 '19 at 0:26
  • $\begingroup$ I'm downvoting this because I don't understand why you're looking at the bias of $\boldsymbol{\hat\beta}$. $\endgroup$ – ahorn Oct 9 '19 at 16:22

First, $$\sqrt{N}(\hat\beta - \beta) = \left( \frac{1}{N} \sum_{i=1}^N X_i X_i^T \right)^{-1} \frac{1}{\sqrt{N}} \sum_{i=1}^N X_i e_i.$$ (No. You can't take plim.)

Next, apply LLN to the "denominator", and apply CLT to the "numerator". Then, under the assumption that $plim \frac{1}{N} \sum_{i=1}^n X_iX_i^T$ is nonsingular and that $\frac{1}{\sqrt{N}} \sum_{i=1}^N X_i e_i$ converges in distribution to a centered Gaussian distribution, you will be able to show that $$\sqrt{N}(\hat\beta - \beta) \rightarrow N(0, C)$$ for some $C$, where $C$ is called the asymptotic variance of $\sqrt{N}(\hat\beta - \beta)$. (Please derive $C$ yourself for this OLS case using "if $plim A_n = A$ and $b_n \rightarrow N(0,B)$, then $A_n b_n \rightarrow N(0, ABA^T)$.)

Finally, the "asymptotic variance" of $\hat\beta$ is defined as $$AV(\hat\beta) = \frac{1}{N} AV( \sqrt{N}(\hat\beta - \beta)).$$ (This is my definition. Others may define it differently. I don't know yours.) According to this definition, $AV(\hat\beta) = \frac{1}{N} C$. You have already derived $C$ above. Divide it by $N$.

One step further: I don't know how you define asymptotic variances. Depending on the exact definition of $AV$, you may need to remove $\lim$ and $plim$ from $C$ and cancel $N$'s here and there. Some people also mean by $AV(\hat\beta)$ a consistent estimator of $C$ divided by $N$.

  • $\begingroup$ The question was how to derive $C$. $\endgroup$ – ahorn Oct 9 '19 at 16:17
  • $\begingroup$ Well, if iid, $A = E[X_i X_i^T]^{-1}$ and $B = E[X_i e_i^2 X_i^T]$ as OP writes, and $C=ABA^T$. OP derives $C$ (though a bit clumsy); what OP misses is $AV(\hat\beta)$, I thought. Only OP knows. $\endgroup$ – chan1142 Oct 10 '19 at 5:48

Note that is it customary in econometrics to put the explanatory variables on the columns of $\boldsymbol X$, and the observations on the rows. The question has switched these around; I will use the notation in the question.

The covariance matrix of $\boldsymbol{{\hat\beta}}$ is $$\sigma^2\cdot \mathrm E_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]$$ where an unbiased estimate of $\sigma^2$ is $\frac 1{N-K}\sum_{i=1}^N e_i e_i $. This setting (with the expectation operation used) assumes that $\boldsymbol X$ is stochastic, i.e. that we cannot fix $\boldsymbol X$ in repeated sampling. My point is that this is not a distribution, as claimed in the question. The following is a distribution: $$\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right) \overset a\sim \mathrm N\left(\ 0\,,\ \ N \sigma^2\cdot \mathrm E_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]\right).$$ Note that the above states that $\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right)$ is asymptotically normally distributed and centred on $0$ (if we didn't multiply by $\sqrt N$, the distribution would converge to a spike, because $\boldsymbol{\hat\beta}$ is consistent).

In your question, you've looked at the bias of $\boldsymbol{\hat\beta}$, which is expected to be $0$. Note that $\sum_{i=1}^N \boldsymbol {x_i}e_i$ is expected to be $0$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.