Let's suppose I have $\sqrt{N}$-consistent estimate, $\beta^{*}$(vector, but couldn't put it in bold), then doing the Gauss-Newton Regression, i.e. $\mathbf{y}-\mathbf{x}(\beta^{*})=D_\mathbf{x}(\beta^{*})\mathbf{b}+\mathbf{v}$.

Then we obtain the estimate $\mathbf{\hat b}=\left(D_\mathbf{x}(\beta^{*})^T D_\mathbf{x}(\beta^{*})\right)^{-1}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))$.

By adding this last estimate to $\beta^{*}$, we get the one-step estimator $\bar\beta=\beta^{*}+\mathbf{\hat b}$.

My first question is how the initial estimate is obtained? Do we use the usual method for estimations in this setting (Quasi-Newton methods, etc.)?

Second, how do we prove that this one-step estimator is asymptotically efficient?

I'm following Davidson and Mackinnon (2008) book, but the way they 'derive' it is not really a proof.

So, by trying to mimic their line of reasoning I've done the following:

From a Taylor Expansion/Mean-Value theorem, $$n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))=n^{-1/2}D_\mathbf{x}(\beta_0)^T(\mathbf{y}-\mathbf{x}(\beta_0))+\underbrace{\left(o_p(1)-n^{-1}D_\mathbf{x}(\beta)^TD_\mathbf{x}(\beta)\right)}_{\Delta(\beta)}n^{-1/2}(\beta^{*}-\beta_0)$$, where $\beta_0$ is the DGP value of beta, and we've done an abuse of notation by not specifying a different beta for each component of $\Delta(\beta)$. See the answers to this question, if you're interested in more details on this expansion.

Also, we know from the previous link, that for the Non-linear Least Squares estimator $\hat\beta$, we have $n^{-1/2}(\hat\beta-\beta_0)=-\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta_0)^T\mathbf{u}$.

Hence we can conclude

$n^{-1/2}(\beta^{*}-\beta_0)=\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))+n^{-1/2}(\hat\beta-\beta_0)$, implying that


By definition, and some usual assumptions, we know that $plim \Delta(\beta)=S_{D_\mathbf{x}^TD_\mathbf{x}}$.

And I'm stuck here, because what the book does is not translatable when I try be formal in the derivation. So, what should I do to prove that the one-step estimator is efficient?

  • $\begingroup$ NLLS is an M-estimator, so we can get asymptotic variance by simply using the formula for asymptotic variance of an M-estimator:$$\frac{1}{n}A^{-1}BA^{-1}$$ where $A$ is the expected value of the hessian matrix of the objective function and $B$ is the expected value of the score (jacobian) matrix squared. Because we have a least squares estimator, the objective function is trying to minimize squared errors, so that should give you the objective function. $\endgroup$
    – DornerA
    Commented Apr 19, 2016 at 13:24
  • $\begingroup$ @DornerA I was expecting an answer without resorting to that perspective of M-estimators... $\endgroup$ Commented Apr 19, 2016 at 13:41
  • $\begingroup$ It was just a suggestion because it is an easy way to find the asymptotic distribution $\endgroup$
    – DornerA
    Commented Apr 19, 2016 at 16:18
  • $\begingroup$ @DornerA I think I may have seemed like a bit rude. Sorry. Thanks for your comment. I know that we could use the usual derivations for M-estimators, but I'm interested in this particular case, out of curiosity. Anyhow, thanks for your comment. $\endgroup$ Commented Apr 19, 2016 at 18:38


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.