The famous Newey 94 paper on the asymptotic convergence of semiparametric estimators with a first non parametric step and a second parametric one, http://www.jstor.org/stable/2951752, establishes that it does not matter the rate of convergence of the particular non parametric estimator, as long a a number of regularity assumptions are fulfilled then the estimator of the second step is $\sqrt{N}$ convergent to a normal distribution. Here I am asking for intuition why does this a complicated stochastic process, say Naradaya-Watson asymptotic distribution converges to this nice distribution.

  • 1
    $\begingroup$ If nobody answers you in 2 days, I would try either CV or MSE. Don't forget to link the questions. good luck $\endgroup$ Dec 29, 2014 at 9:39

1 Answer 1


The usual proof of the classical Central Limit Theorem (CLT), I believe provides the most intuition there is about this phenomenon. And it is not much of an intuition anyway.

This "usual" proof is through characteristic functions.

Consider a random variable $X$ with characteristic function

$$\phi_X(t) = E(e^{i tX}), \;\;\;i^2 = -1$$

Now consider its centered and scaled version

$$Y = \frac {X-\mu}{\sigma} = \frac 1{\sigma}X - \frac {\mu}{\sigma}$$.

with $E(Y) = 0 , {\rm Var}(Y) E(Y^2)= 1$. Moreover, $Y$ is the sum of two independent random variables, the second degenerate (being a constant, and so also independent of everything). So, by the properties of the characteristic function for the sum of two independent random variables

$$\phi_Y(t) = \phi_Y\left(\frac 1{\sigma}t\right)\cdot e^{-i\frac {\mu}{\sigma}t} = E\left[\exp{\left\{it\frac 1{\sigma}X - i\frac {\mu}{\sigma}t\right\}}\right] = E(e^{i tY})$$

Now take the second-order Taylor expansion of $\phi_Y(t)$, with respect to $Y$ and with center of expansion $E(Y) =0$: $$\phi_Y(t) = E\left[e^{it\frac 1{\sigma}\cdot 0} + ite^{it\cdot 0}Y + \frac 12 i^2t^2e^{it\cdot 0}Y^2 + o(t^2)= \right]$$

The first term is zero, the second term vanishes because $E(Y) =0$, and for the third term we use $i^2=-1, E(Y^2) =1$ to arrive at

$$\phi_Y(t) = 1 - \frac {t^2}2 + o(t^2)$$

The "phenomenon" is already here because

$$ \frac {t^2}2 = \ln MGF_{Z}(t),\;\; Z\sim {\rm N}(0,1)$$

In words: The characteristic function of any random variable with finite mean and variance that is centered and scaled accordingly, has a strong connection with the moment generating function of the Standard Normal distribution ($\ln MGF_{Z}(t)$ is actually the cumulant generating function).

How can this happen? Have we uncovered a "law of nature" here, this fundamental connection of different "types of uncertain behavior" to the specific type labelled "standard normal distribution", or it is just our mathematical system, through which we have modeled this thing called "uncertainty", producing some artificial connection that perhaps reveals some aspect of its own internal structure, but it has nothing to do with the real world?

Let's first complete the proof of the CLT: consider the random variable $$Z_n = \frac 1{\sqrt{n}}\sum_{i=1}^nY_i$$

where the $Y_i$'s are independently and identically distributed. Again by the properties of the characteristic function we have

$$\phi_{Z_n}(t) = \prod_{i=1}^n\phi_{Y}(t\sqrt{n}) = \Big[1 - \frac {t^2}{2n} + o(\frac {t^2}{n})\Big]^n \xrightarrow{n\rightarrow \infty} e^{-t^2/2} = \phi_{Z}(t),\;\; Z\sim {\rm N}(0,1)$$

and so

$$Z_n \xrightarrow{d} {\rm N}(0,1)$$

...and now we have arrived at the Standard Normal distribution proper. And this is a law of nature: mathematics aside, computer simulations aside, real world data consistently validate this result. So there is no further "why?" here -as it cannot be with any law of nature. We just discover them -and we feel like gaining added intuition when we discover the interconnections between these laws (but this is not really insightful, it is just more discoveries).

Of course this is the first and most simple case, in a long line of Central Limit Theorems that increasingly deal with more complicated, interdependent, functions of random variables, stochastic processes etc as the one mentioned in the OP. And there are also the generalizations of CLT to stable distributions and not just the normal, and there is extreme value theory... all these are different aspects of the same conclusion: that collective behavior (even in the simple sense of pooled behavior) is much more homogeneous than individual behaviors, even though it is just the pool of the latter -and this should be one of the most counter-intuitive results ever encountered.

P.S. An inspired attempt at bottom-up intuition is provided by @whuber in this Cross Validated thread: https://stats.stackexchange.com/questions/3734/what-intuitive-explanation-is-there-for-the-central-limit-theorem


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.