How to derive the asymptotic variance from the sampling distribution of the OLS estimator?

The asymptotic sampling distribution, after taking plim, of the OLS estimator is given by $$\sqrt{N}(\hat{\beta}-\beta) = E[X_iX_i^T]^{-1} \left(1/\sqrt{N} \sum_{I=1}^NX_ie_i \right)$$

It must be shown that the asymptotic variance can be written as: $$Var[E[X_iX_i^T]^{-1} 1/\sqrt{N} \sum_{I=1}^NX_ie_i] = \left( E[X_iX_i^T]^{-1} \right) \left( E[X_iX_i^T e_i^2]\right) \left( E[X_i X_i^T]^{-1} \right)$$

Because the variance is linear in parameters, my idea was to start like this:

$$V[\sqrt{N}(\hat{\beta}-\beta)] = \sqrt{N}Var[\hat{\beta}] + 0$$

Because the $$Var[\beta]=0$$. Thus one could use the fact that $$Var[\hat{\beta}]=E[(\hat{\beta}-\beta)(\hat{\beta}-\beta)^T]$$.

However, while there seems to be a relationship I stuck at this point, mostly because of the $$\sqrt{N}$$. Thanks for help.

• That is unreadable to me. Try using LaTeX surrounded by $signs. And perhaps also what you have attempted yourself – Henry Oct 3 '19 at 0:26 • I'm downvoting this because I don't understand why you're looking at the bias of$\boldsymbol{\hat\beta}$. – ahorn Oct 9 '19 at 16:22 2 Answers First, $$\sqrt{N}(\hat\beta - \beta) = \left( \frac{1}{N} \sum_{i=1}^N X_i X_i^T \right)^{-1} \frac{1}{\sqrt{N}} \sum_{i=1}^N X_i e_i.$$ (No. You can't take plim.) Next, apply LLN to the "denominator", and apply CLT to the "numerator". Then, under the assumption that $$plim \frac{1}{N} \sum_{i=1}^n X_iX_i^T$$ is nonsingular and that $$\frac{1}{\sqrt{N}} \sum_{i=1}^N X_i e_i$$ converges in distribution to a centered Gaussian distribution, you will be able to show that $$\sqrt{N}(\hat\beta - \beta) \rightarrow N(0, C)$$ for some $$C$$, where $$C$$ is called the asymptotic variance of $$\sqrt{N}(\hat\beta - \beta)$$. (Please derive $$C$$ yourself for this OLS case using "if $$plim A_n = A$$ and $$b_n \rightarrow N(0,B)$$, then $$A_n b_n \rightarrow N(0, ABA^T)$$.) Finally, the "asymptotic variance" of $$\hat\beta$$ is defined as $$AV(\hat\beta) = \frac{1}{N} AV( \sqrt{N}(\hat\beta - \beta)).$$ (This is my definition. Others may define it differently. I don't know yours.) According to this definition, $$AV(\hat\beta) = \frac{1}{N} C$$. You have already derived $$C$$ above. Divide it by $$N$$. One step further: I don't know how you define asymptotic variances. Depending on the exact definition of $$AV$$, you may need to remove $$\lim$$ and $$plim$$ from $$C$$ and cancel $$N$$'s here and there. Some people also mean by $$AV(\hat\beta)$$ a consistent estimator of $$C$$ divided by $$N$$. • The question was how to derive$C$. – ahorn Oct 9 '19 at 16:17 • Well, if iid,$A = E[X_i X_i^T]^{-1}$and$B = E[X_i e_i^2 X_i^T]$as OP writes, and$C=ABA^T$. OP derives$C$(though a bit clumsy); what OP misses is$AV(\hat\beta)\$, I thought. Only OP knows. – chan1142 Oct 10 '19 at 5:48

Note that is it customary in econometrics to put the explanatory variables on the columns of $$\boldsymbol X$$, and the observations on the rows. The question has switched these around; I will use the notation in the question.

The covariance matrix of $$\boldsymbol{{\hat\beta}}$$ is $$\sigma^2\cdot \mathrm E_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]$$ where an unbiased estimate of $$\sigma^2$$ is $$\frac 1{N-K}\sum_{i=1}^N e_i e_i$$. This setting (with the expectation operation used) assumes that $$\boldsymbol X$$ is stochastic, i.e. that we cannot fix $$\boldsymbol X$$ in repeated sampling. My point is that this is not a distribution, as claimed in the question. The following is a distribution: $$\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right) \overset a\sim \mathrm N\left(\ 0\,,\ \ N \sigma^2\cdot \mathrm E_{\boldsymbol X}\left[\left(\boldsymbol {XX^T}\right)^{-1}\right]\right).$$ Note that the above states that $$\sqrt N \left(\boldsymbol{\hat\beta} - \boldsymbol\beta\right)$$ is asymptotically normally distributed and centred on $$0$$ (if we didn't multiply by $$\sqrt N$$, the distribution would converge to a spike, because $$\boldsymbol{\hat\beta}$$ is consistent).

In your question, you've looked at the bias of $$\boldsymbol{\hat\beta}$$, which is expected to be $$0$$. Note that $$\sum_{i=1}^N \boldsymbol {x_i}e_i$$ is expected to be $$0$$.