How to show that GLS estimator is consistent in regression model?

$$\mathbf Y=\mathbf X\beta + \mathbf U$$ $$\mathbf E[\mathbf U\mathbf U']=\Omega$$

$$\hat \beta - \beta=(\mathbf X'\Omega^{-1}\mathbf X)^{-1}\mathbf X'\Omega^{-1}\mathbf U$$

let $\mathbf X^{*}=\Omega^{-0.5}\mathbf X$, and $\mathbf U^{*}=\Omega^{-0.5}\mathbf U$. it is obviously that $\mathbf u^{*}_i$,which is the i-th element of $\mathbf U^{*}$, is iid follows the standard normal distribution, if we assume that $\mathbf U$ follows normal distribution. But $x_i^{*'}$, the i-th row of $\mathbf X^{*}$ is not necessarily independent, for it is the linear combination of all rows in $\mathbf X$ (which is assumed iid for they stand for observations from the same distribution).

Thus the followings are not necessarily true under independent Law of Large number and Central limit theorem:

1. $plim\frac{\mathbf X'\Omega^{-1}\mathbf U}{n}=plim\frac{\mathbf X^{*'}\mathbf U^{*}}{n}= plim\frac{1}{n}\Sigma x_i^{*}u_i^{*} =0$

2. $\frac{\mathbf X'\Omega^{-1}\mathbf U}{\sqrt n}=\frac{\mathbf X^{*'}\mathbf U^{*}}{\sqrt n}= \frac{1}{\sqrt n}\Sigma x_i^{*}u_i^{*}$, converges to normal distribution

I understand that when $\mathbf \Omega$ is a diagonal matrix then $x_i^{*}$ is independent and it is not a problem, what if when $\mathbf \Omega$ is not a diagonal matrix? What kind of dependent LLN and CLT can be applied here? should we make other assumptions?

• If you have iid sample, the regressors are exogen and $\Omega$ (which is really a sequence indexed by sample size) is uniformly positive definite, and has a block diagonal structure, then consistency follows from LLN for, e.g. mixing processes. The moment restrictions, e.g. finite second moment must exists, etc, for the iid case may need to be strengthened accordingly. – Michael Apr 21 '16 at 6:43
• @Michael Assume iid sample and exogeneity. It sounds like there must be some assumptions applied to the $\Omega$ before we could use any LLN and CLT isn't it? – Wenkui Liu Apr 21 '16 at 10:59
• Yes, uniformly positive definite and block diagonal should do it. Although block diagonal may be too strong, it's easier to state, I will try to post an answer if someone doesn't beat me to it. – Michael Apr 22 '16 at 3:13
• @Michael I just come up with a solution by the Chebyshev's inequality. P[|$\frac{\mathbf X'\Omega^{-1}\mathbf U}{n}$ |>$\epsilon$]<= $\frac{E[\mathbf X'\Omega^{-1}\mathbf U\mathbf U'\Omega^{-1}\mathbf X']}{n^2}$ =$\frac{E[\mathbf X'\Omega^{-1}\mathbf X']}{n^2}$ . This goes to zero if we only assume finite second moment of X. Thus the consistency is proved. Am I right? – Wenkui Liu Apr 29 '16 at 7:29

If the regressors are strictly exogenous to the error term (which is the initial standard assumption made), meaning, if the error vector is mean-independent of the regressor matrix

$$E(\mathbf u \mid \mathbf X) = \mathbf 0$$

then consistency follows immediately. Set for convenience $\mathbf \Omega ^{-1/2} \equiv \mathbf C$. Then

$$\text{plim}\left(n^{-1} (\mathbf C\mathbf X)'\mathbf u \right)= \text{plim}\left(n^{-1}\mathbf X'\mathbf C'\mathbf u\right)$$

This is a $k \times 1$ matrix. What would a typical element will be, say the first?

The matrix $\mathbf C'\mathbf u$ is a $n\times 1$ vector,

$$\mathbf C'\mathbf u = \left(\sum_{i=1}^nc_{1i}u_i,..., \sum_{i=1}^nc_{ni}u_i\right)'$$

Mutliplied by the first row of $\mathbf X' = (x_{11}\;...\;x_{1n})$ gives

$$\left [\mathbf X'\mathbf C'\mathbf u\right]_1 =x_{11}\cdot \sum_{i=1}^nc_{1i}u_i\;+...\;+ x_{1n}\cdot\sum_{i=1}^nc_{ni}u_i$$

Now strict exogeneity comes into play: it is stronger and so implies that

$$E(\mathbf u \mid \mathbf X) = \mathbf 0 \implies E(x_{1i}u_j)=0\;\;\forall i,j$$

So the above sum of sums, scaled by $n^{-1}$ will tend to zero asymptotically, since all products in each sum will tend to a zero expected value. This proves consistency.

Note that strict exogeneity is really needed, for the case where the matrix $\Omega$ is a function of the regressors. In such a case, we would be looking at products of the form

$$x_{1i}\cdot h(\mathbf x) \cdot u_j$$

which would still go to zero in expected value terms, because mean-independence implies orthogonality between the error term and any function of the regressors.

If $\Omega$ does not depend on the regressors, then we could weaken strict exogeneity to the RHS condition, which is stronger than "contemporaneous uncorrelatedness", the latter needed for OLS.

I leave the case for asymptotic normality to anyone interested.

• Misleading to claim the above is an argument for consistency. "...So the above sum of sums, scaled by $n^{−1}$ will tend to zero asymptotically..." is clearly not true in general, even in the i.i.d. case. It's a sum of $n$ sums, and $n \rightarrow \infty$. Expectation and infinite series already doesn't commute in general, much less probability limit and infinite series. Counterexamples are easy to construct with diagonal $\Omega$. As OP already pointed out in the comments, this requires boundedness-from-below type of assumptions on $\frac{\Omega^{-1}}{n}$ . – Michael Jul 13 at 14:59
• @Michael Hmm, I guess I left certain "usual" assumptions implied and not explicitly stated. Let me see how I can upgrade the quality of this answer. – Alecos Papadopoulos Jul 14 at 16:52