# Show that the estimator of GMM weigh matrix is consistent

(Hansen Exercise 13.3) Take the model $$Y = X'\beta + e$$ with $$\mathbb{E}[Ze] = 0$$. Let $$\tilde e_i = Y_i - X_i'\tilde \beta$$ where $$\tilde\beta$$ is consistent for $$\beta$$ (e.g. a GMM estimator with some weight matrix). An estimator of the optimal GMM weigh matrix is $$\hat W = \left( \frac1n \sum_{i=1}^n Z_i Z_i' \tilde e_i^2 \right)^{-1}.$$ Show that $$\hat W \to_p \Omega^{-1}$$ where $$\Omega = \mathbb{E}[ZZ'e^2]$$.

Since $$\tilde \beta$$ is consistent, $$\tilde e_i$$ also converges in probability to $$e_i$$ for all $$i$$. Also, by LLN, $$\frac1n \sum_{i=1}^n Z_i Z_i' \tilde e_i^2 \to E[Z_iZ_i' \tilde e_i^2].$$ I think I need to combine these two results in some way to get the final result. Can you give some help?

• At the level of a single random term, $\tilde e_i$ will not converge in probability to $e_i$. Averaging is necessary (LLN). Jul 18 '21 at 21:31
• Note $\tilde{e}_i = e_i - X_i' (\tilde{\beta}-\beta)$ so $\tilde{e}_i^2 = e_i^2 - 2e_i x_i' (\tilde\beta - \beta) + (\tilde\beta - \beta)' X_i X_i' (\tilde\beta - \beta)$. Multiply $Z_iZ_i'$, add up, and then you will see that the second term and the third term converge in probability to zero due to the consistency of $\tilde\beta$. The first term converges in probability to $E[Z_i Z_i' e_i^2] = \Omega$ by LLN. Note that it's not $E[Z_iZ_i' \tilde{e}_i^2]$. Jul 19 '21 at 10:13
• @Bertrand Well, $\tilde{e}_{i}$ does converge in probability to $e_{i}$, it's just from consistency of $\tilde{\beta}$ and slutsky lemma. But this method won't work because the infinite sum of little op is not necessary a little op, $\tilde{e}_{i}=e_{i}+o_{p}\left(1\right)$ doesn't yield $\frac{1}{n}\sum_{i=1}^{n}Z_{i}Z_{i}^{\top}\tilde{e}_{i}=\frac{1}{n}\sum_{i=1}^{n}Z_{i}Z_{i}^{\top}e_{i}+o_{p}\left(1\right)$.
– Q9y5
Jul 19 '21 at 10:49
• In my previous comment, "add up" is suppose to be "average". Jul 19 '21 at 12:10
• Thanks for your clarification. Jul 20 '21 at 9:51

We have \begin{align} \frac1n \sum_{i=1}^n Z_i \tilde{e}_i^2 Z_i' &= \frac1n \sum_{i=1}^n Z_i \Big[e_i^2 - 2e_i X_i' (\tilde\beta - \beta) + (\tilde\beta - \beta)' X_i X_i' (\tilde\beta - \beta)\Big] Z_i' \\ &= A_1 - 2A_2 + A_3. \end{align} because $$\tilde{e}_i = y_i - X_i'\tilde\beta = (X_i'\beta + e_i) - X_i'\tilde\beta = e_i - X_i' (\tilde\beta - \beta).$$
First, $$A_1 = \frac1n \sum_{i=1}^n Z_i e_i^2 Z_i' \to_p E[Z_i e_i^2 Z_i'] = \Omega \text{ by LLN}.$$ Next, $$A_2 = \frac1n \sum_{i=1}^n Z_i e_i X_i' (\tilde\beta - \beta) Z_i',$$ the $$(j,k)$$ element of which is $$A_{2,jk} = \frac1n \sum_{i=1}^n z_{ij} e_i X_i' (\tilde\beta - \beta) z_{ik} = \left( \frac1n \sum_{i=1}^n z_{ik} z_{ij} e_i X_i' \right) (\tilde\beta - \beta).$$ The first term (in the brackets) is $$O_p(1)$$; it can be $$o_p(1)$$ if $$X_i$$ is exogenous but it does not matter. The second term is $$o_p(1)$$ because $$\tilde\beta$$ is consistent. Thus, $$A_2 = o_p(1)$$. Finally, $$A_3 = \frac1n \sum_{i=1}^n Z_i (\tilde\beta - \beta)' X_iX_i' (\tilde\beta - \beta) Z_i',$$ the $$(j,k)$$ element of which is \begin{align} A_{3,jk} &= \frac1n \sum_{i=1}^n z_{ij} (\tilde\beta - \beta) X_i X_i' (\tilde\beta - \beta) z_{ik}\\ &= (\tilde\beta - \beta)' \left[ \frac1n \sum_{i=1}^n X_i z_{ij} z_{ik} X_i' \right] (\tilde\beta - \beta)\\ &= o_p(1) O_p(1) o_p(1) = o_p(1), \end{align} and thus $$A_3 = o_p(1)$$.
We have shown that $$\hat{W}^{-1} \to_p \Omega$$. Under the assumption that $$\Omega$$ is invertible, $$\hat{W} \to_p \Omega^{-1}$$.