# Prove that an estimator is consistent

The following exercise is from Wooldrige:

Show that $\hat{\beta} =\frac{1}{N} \sum\limits_{i=1}^{N}\hat{u_i^{2}}x'x$ is a consistent estimator for $E(u^2x'x)$

by showing:

$$\frac{1}{N} \sum\limits_{i=1}^{N}\hat{u_i^{2}}x'x = \frac{1}{N} \sum\limits_{i=1}^{N}{u_i^{2}}x'x + o_p(1)$$

Where $o_p(1)$ is read "little oh p 1"

And we use the hints that:

1.) $\hat{u}_i^2$ = $u_i^2 - 2x_iu_i(\hat{\beta}-\beta) + [x_i(\hat{\beta}-\beta)]^2$

2.) $\hat{\beta} - \beta$ = $o_p(1)$

3.)Sample averages are $O_p(1)$ read as "Big Oh P 1"

4.) we assume all necessary expectations exist and are finite

I'm getting stuck with this. I know I must be missing some simple sort of substitution or there is some gap in my knowledge or understanding preventing me from making the necessary manipulations.

I would love if someone could walk me through this and explain the intuition a bit here.

• I'm voting to close this question as off-topic because this is an exercise and there is no effort towards solution shown. – Giskard Sep 8 '15 at 7:40
• I've been working toward a solution on my whiteboard for the past two hours. It's a bit ridiculous for you to assume a person addressing a question of this level is the type of person who wouldn't want to solve it himself or herself. – 123 Sep 8 '15 at 7:48
• I think the standard text was "homework question", not "exercise". For some reason (perhaps this is to be discussed in meta), I personally don't require more advanced questions to have the same effort in solving it shown. Perhaps it is because advanced questions are more interesting intrinsically. – FooBar Sep 8 '15 at 7:53
• I omitted my efforts largely because I doubt any reader will gain much from them. And my chosen path is relatively obvious: I follow the hints, of course. – 123 Sep 8 '15 at 7:59
• Very well. Perhaps I misjudged the level if this exercise. In this case there will be more votes against than for closing, which is fine. I think this is one of the reasons why the system requires 5 votes to close. – Giskard Sep 8 '15 at 8:18

I will provide another hint, more "primitive" than the one offered by @Anoldmaninthesea, for those that are not still very familiar with the "big-Oh/little oh" notation and arithmetic.

What we are examining here is a sum, which, following the first obvious "hint" given, is to be decomposed in three separate sums.

Now, we are interested in what happens to the value of these sums as the number of summands goes to infinity... In such cases, essentially we are looking if "we have enough "$N$"'s for the infinite sum to go in value to something finite (given the a priori assumptions), and if yes, whether this finite limit is zero or not.

Consider the middle sum

$$\frac{1}{n} \sum\limits_{i=1}^{n}[-2x_iu_i(\hat{\beta_n}-\beta)\mathbf x_i'\mathbf x_i] = -2(\hat{\beta_n}-\beta)\frac{1}{n} \sum\limits_{i=1}^{n}(x_iu_i\mathbf x_i'\mathbf x_i)$$

$(\hat{\beta_n}-\beta)$ was taken out of the sum because it does not depend on the index $i$. It does depend on $n$, since it is an estimator function and not a specific estimate, but not on $i$. The assumptions of the model tell us what happens to $(\hat{\beta_n}-\beta)$, if it remains unscaled, as $n\rightarrow \infty$. As for the summands, $(\mathbf x_i'\mathbf x_i)$ is a $k\times k$ matrix, that does not include itself any sum. So it is finite (the fact that the random variables involved have possibly infinite support does not bother us -the probability that they will take the value "infinity" is zero). So it is only the whole expression $(x_iu_i\mathbf x_i'\mathbf x_i)$ that multiplies in cardinality as $n$ increases - and by the assumptions of the model we know things about $x_i$ and $u_i$ and what happens to the average of their product. Etc.

An unrelated example to show what this "how many $n$'s" business means is to consider, for example, a random vector like

$$\frac{1}{\sqrt n}\left(\mathbf X'\mathbf X (\hat \beta_n - \beta)\right)$$

We "know" that this random vector converges to a multivariate distribution, i.e. it neither converges to a constant, nor does it diverge. Can we see this in our primitive way? We would want a finite asymptotic variance. The asymptotic mean will be zero so we need only to examine

$$V_n=E\left[\frac{1}{\sqrt n}\mathbf X'\mathbf X(\hat \beta_n - \beta)(\hat \beta_n - \beta)'\mathbf X'\mathbf X\frac{1}{\sqrt n}\right] = E\left[\left(\frac{1}{n}\mathbf X'\mathbf X\right)[(\hat \beta_n - \beta)(\hat \beta_n - \beta)']\left(\mathbf X' \mathbf X\right)\right]$$

What is the problem with the above expectation? The first term wll converge to something finite and non-zero (per initial assumptions), the second goes to zero, the third goes to infinity(because it includes sums whose cardinality will go to infinity). So the expectation is still of undetermined fate. But if we multiply and divide by $n$ (we can do that, we have not yet send $n$ to infinity) we obtain

$$V_n= E\left[\left(\frac{1}{n}\mathbf X'\mathbf X\right)[n\cdot(\hat \beta_n - \beta)(\hat \beta_n - \beta)']\left(\frac{1}{n}\mathbf X' \mathbf X\right)\right]$$

Now things begin to clear: the first and second terms will converge to expected values, so they can be taken out of the external expected value:

$$V_n\rightarrow \left(\lim_{n\rightarrow \infty} E\frac{1}{n}\mathbf X'\mathbf X\right)\lim_{n\rightarrow \infty} E\left[n\cdot(\hat \beta_n - \beta)(\hat \beta_n - \beta)'\right]\left(\lim_{n\rightarrow \infty}E\frac{1}{n}\mathbf X' \mathbf X\right)$$

The middle term inside the expectation is the (multivariate) "square" of $\sqrt n (\hat \beta_n - \beta)$. We know that this last one converges to a zero-mean random vector, so the limit of the expected value of its "square" is its asymptotic variance, which is finite. So we have the product of three asymptotically finite expected values, and so the whole expression is finite, and so the variance of the expression we started with is finite, and moreover, non-zero (by the usual initial assumptions of the model).

In hint 1 substitute $\hat{\beta}-\beta$ by little o, and remember that the sum of little o 's is still little o's. Then substitute on the LHS of your equality abouve. and you get what you want. (multiplication of $x'x$ by little o is also little o) I hope this is enough. If not, I'll show more later. I'm pressed for time right now... sorry