Is this use of a Law of Large Numbers correct?

Question

The usual LLN I've seen, the theorem states that $1/n \sum u_t \rightarrow^p E(u_t)=\mu$,where the expected value is independent of t. However, I cannot apply it to the exercise below.

In the picture, the author seems to be using $ \displaystyle \frac{1}{n} \sum u_t \rightarrow^p \lim_{n \rightarrow \infty}\frac{1}{n} \sum E(u_t)$

Result (3.44) is $Var(\hat u_t^2)=E(\hat u_t^2)=(1-h_t)\sigma^2$, and it's also the result they refer to as 'in the previous exercise', and (3.46) is just the first equation, without the plims.

So, what LLN can I apply?

Any help would be appreciated.

Answer 1

Are the $\hat{u}_t^2$ independent? If yes, the result is indeed a version of the law of large numbers due to Kolmogorov, and that deals with independent (but not necessarily identically distributed) processes.

Here is the formal statement. Suppose that $\{u_t\}_{t=1}^{\infty}$ is a sequence of independent real-valued random variables that satisfies \begin{equation*} \sum_{t=1}^{\infty}{\dfrac{Var(u_t)}{t^2}}<\infty \end{equation*}

Then the random variable \begin{equation*} \dfrac{1}{n} \sum_{t=1}^{n}{[u_t-\mathbb{E}(u_t)]} \end{equation*} converges to 0 almost surely.

Edit: the assumption of independence seems stronger than needed, and the following result (an adapted version of the law of large numbers for non-independent processes) might be useful. If $\sigma_{n,m}=Cov(u_n,u_m)$ exists, and if there is $0< \alpha <1$ and $M \in \mathbb{R}$ such that $\sigma_{n,m} < M \alpha^{|n-m|}$ for all $(n,m)$, then \begin{equation*} \mathbb{P}(|\dfrac{1}{n}\sum_{t=1}^{n}u_t- \dfrac{1}{n}\sum_{t=1}^{n}\mathbb{E}(u_t)|>\epsilon] < \dfrac{M}{n \epsilon^2} \end{equation*} for any $\epsilon$.

Answer 2

A bit more intuitive formulation of a sufficient condition for the (weak) law of large numbers to hold(which is the one associated with the consistency property), for the average of a collection of non-independent, non-identically distributed random variables with finite variances and covariances, is the following (the "Markov condition"):

$$\text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) \rightarrow 0 $$

This simply says that it is sufficient that the variance of the average goes to zero, which has intuition. Decomposing,

$$\text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) = \frac 1{n^2}\sum_{t=1}^n \text {Var}(x_t) +\frac 1{n^2}\sum_{t\neq s} \text {Cov}(x_t,x_s) \rightarrow 0$$

Since all variances are finite, the first sum goes to zero. As regards the second sum, if each element is correlated with all others irrespective of how many there are, then this (double) sum has $n^2-n$ strictly non-zero elements, i.e of order $O(n^2)$. If this is the case, then it doesn't go to zero, and the sufficient condition does not hold.

The easiest way to see this is to assume that all rv's are equicorrelated: if

$$\text {Cov}(x_t,x_s) = c\;\forall t,s \implies \text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) = \frac 1{n^2}\sum_{t=1}^n \text {Var}(x_t) +\left(1-\frac 1n\right) c\rightarrow c$$

By the way this is how one can glimpse why, with "everybody with everybody" correlation, the sample mean remains a random variable irrespective of the sample size.

So what it takes to obtain the weak $\text{LLN}$?

We may assume $m-$dependence, namely that each rv is correlated only with $m$ others, with $m$ being a fixed number. This will send the variance of the sample mean to zero.

We may assume that as the sample size increases, the number of non-zero covariances increases with it, but not at the same rate: $m(n)/n \rightarrow 0$.

If we want to maintain that each rv is correlated with every other (which is the case of the OP since it deals with estimation residuals), then we arrive at the condition stated in @Oliv's answer: it has intuitive meaning (and justification) as "covariance diminishes as distance increases" only when there is a natural ordering of the sample (temporal or spatial). When the sample is bona-fide cross-sectional and one can re-order the rv's at will, the condition is just mathematical.