2
$\begingroup$

The usual LLN I've seen, the theorem states that $1/n \sum u_t \rightarrow^p E(u_t)=\mu$,where the expected value is independent of t. However, I cannot apply it to the exercise below.

In the picture, the author seems to be using $ \displaystyle \frac{1}{n} \sum u_t \rightarrow^p \lim_{n \rightarrow \infty}\frac{1}{n} \sum E(u_t)$

enter image description here

Result (3.44) is $Var(\hat u_t^2)=E(\hat u_t^2)=(1-h_t)\sigma^2$, and it's also the result they refer to as 'in the previous exercise', and (3.46) is just the first equation, without the plims.

So, what LLN can I apply?

Any help would be appreciated.

|improve this question
$\endgroup$
2
$\begingroup$

Are the $\hat{u}_t^2$ independent? If yes, the result is indeed a version of the law of large numbers due to Kolmogorov, and that deals with independent (but not necessarily identically distributed) processes.

Here is the formal statement. Suppose that $\{u_t\}_{t=1}^{\infty}$ is a sequence of independent real-valued random variables that satisfies \begin{equation*} \sum_{t=1}^{\infty}{\dfrac{Var(u_t)}{t^2}}<\infty \end{equation*}

Then the random variable \begin{equation*} \dfrac{1}{n} \sum_{t=1}^{n}{[u_t-\mathbb{E}(u_t)]} \end{equation*} converges to 0 almost surely.

Edit: the assumption of independence seems stronger than needed, and the following result (an adapted version of the law of large numbers for non-independent processes) might be useful. If $\sigma_{n,m}=Cov(u_n,u_m)$ exists, and if there is $0< \alpha <1$ and $M \in \mathbb{R}$ such that $\sigma_{n,m} < M \alpha^{|n-m|}$ for all $(n,m)$, then \begin{equation*} \mathbb{P}(|\dfrac{1}{n}\sum_{t=1}^{n}u_t- \dfrac{1}{n}\sum_{t=1}^{n}\mathbb{E}(u_t)|>\epsilon] < \dfrac{M}{n \epsilon^2} \end{equation*} for any $\epsilon$.

|improve this answer
$\endgroup$
  • $\begingroup$ That's it. Many thanks for the answer. The $û_t$ are not independent, but I think they are 'asymptotically independent'. Either way, I don't like much this way of proving that the variance estimator for the error terms is consistent. I prefer others. I just wanted to know the theorem that would allow that equality to be written. Thanks once again. ;) $\endgroup$ – An old man in the sea. Mar 5 '16 at 10:29
  • $\begingroup$ @Anoldmaninthesea. my pleasure ! :) $\endgroup$ – Oliv Mar 5 '16 at 10:32
  • $\begingroup$ If you happen to know how we could ensure that the theorem may be applied to this I would much obliged. ;) $\endgroup$ – An old man in the sea. Mar 5 '16 at 10:33
  • $\begingroup$ @Anoldmaninthesea. I apologize, but I am not sure I understand. What is the point you are missing? If you replace $u_t$ by $\hat{u}_t^2$ in the statement of the theorem, you get exactly the inequality $\lim_{n \rightarrow +\infty} \frac{1}{n}\sum_{t=1}^{n}{\hat{u}_t^2} = \lim_{n \rightarrow +\infty} \frac{1}{n} \sum_{t=1}^{n}{\mathbb{E}(\hat{u}_t^2)}$, don't you? $\endgroup$ – Oliv Mar 5 '16 at 10:44
  • $\begingroup$ Yes. The thing is the textbook says to assume that I can apply the theorem to $\{\hat u_t\}$ even though they are not independent. I just asked that if you knew why the textbook states that we can use the that theorem, then I would be thankful if you could tell me. ;) $\endgroup$ – An old man in the sea. Mar 5 '16 at 10:48
0
$\begingroup$

A bit more intuitive formulation of a sufficient condition for the (weak) law of large numbers to hold(which is the one associated with the consistency property), for the average of a collection of non-independent, non-identically distributed random variables with finite variances and covariances, is the following (the "Markov condition"):

$$\text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) \rightarrow 0 $$

This simply says that it is sufficient that the variance of the average goes to zero, which has intuition. Decomposing,

$$\text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) = \frac 1{n^2}\sum_{t=1}^n \text {Var}(x_t) +\frac 1{n^2}\sum_{t\neq s} \text {Cov}(x_t,x_s) \rightarrow 0$$

Since all variances are finite, the first sum goes to zero. As regards the second sum, if each element is correlated with all others irrespective of how many there are, then this (double) sum has $n^2-n$ strictly non-zero elements, i.e of order $O(n^2)$. If this is the case, then it doesn't go to zero, and the sufficient condition does not hold.

The easiest way to see this is to assume that all rv's are equicorrelated: if

$$\text {Cov}(x_t,x_s) = c\;\forall t,s \implies \text {Var}\left (\frac 1{n}\sum_{t=1}^n x_t\right) = \frac 1{n^2}\sum_{t=1}^n \text {Var}(x_t) +\left(1-\frac 1n\right) c\rightarrow c$$

By the way this is how one can glimpse why, with "everybody with everybody" correlation, the sample mean remains a random variable irrespective of the sample size.

So what it takes to obtain the weak $\text{LLN}$?

We may assume $m-$dependence, namely that each rv is correlated only with $m$ others, with $m$ being a fixed number. This will send the variance of the sample mean to zero.

We may assume that as the sample size increases, the number of non-zero covariances increases with it, but not at the same rate: $m(n)/n \rightarrow 0$.

If we want to maintain that each rv is correlated with every other (which is the case of the OP since it deals with estimation residuals), then we arrive at the condition stated in @Oliv's answer: it has intuitive meaning (and justification) as "covariance diminishes as distance increases" only when there is a natural ordering of the sample (temporal or spatial). When the sample is bona-fide cross-sectional and one can re-order the rv's at will, the condition is just mathematical.

|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.