# One-Step estimators for non-linear regression

Let's suppose I have $\sqrt{N}$-consistent estimate, $\beta^{*}$(vector, but couldn't put it in bold), then doing the Gauss-Newton Regression, i.e. $\mathbf{y}-\mathbf{x}(\beta^{*})=D_\mathbf{x}(\beta^{*})\mathbf{b}+\mathbf{v}$.

Then we obtain the estimate $\mathbf{\hat b}=\left(D_\mathbf{x}(\beta^{*})^T D_\mathbf{x}(\beta^{*})\right)^{-1}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))$.

By adding this last estimate to $\beta^{*}$, we get the one-step estimator $\bar\beta=\beta^{*}+\mathbf{\hat b}$.

My first question is how the initial estimate is obtained? Do we use the usual method for estimations in this setting (Quasi-Newton methods, etc.)?

Second, how do we prove that this one-step estimator is asymptotically efficient?

I'm following Davidson and Mackinnon (2008) book, but the way they 'derive' it is not really a proof.

So, by trying to mimic their line of reasoning I've done the following:

From a Taylor Expansion/Mean-Value theorem, $$n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))=n^{-1/2}D_\mathbf{x}(\beta_0)^T(\mathbf{y}-\mathbf{x}(\beta_0))+\underbrace{\left(o_p(1)-n^{-1}D_\mathbf{x}(\beta)^TD_\mathbf{x}(\beta)\right)}_{\Delta(\beta)}n^{-1/2}(\beta^{*}-\beta_0)$$, where $\beta_0$ is the DGP value of beta, and we've done an abuse of notation by not specifying a different beta for each component of $\Delta(\beta)$. See the answers to this question, if you're interested in more details on this expansion.

Also, we know from the previous link, that for the Non-linear Least Squares estimator $\hat\beta$, we have $n^{-1/2}(\hat\beta-\beta_0)=-\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta_0)^T\mathbf{u}$.

Hence we can conclude

$n^{-1/2}(\beta^{*}-\beta_0)=\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))+n^{-1/2}(\hat\beta-\beta_0)$, implying that

$n^{-1/2}(\beta^{*}-\hat\beta)=\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))$.

By definition, and some usual assumptions, we know that $plim \Delta(\beta)=S_{D_\mathbf{x}^TD_\mathbf{x}}$.

And I'm stuck here, because what the book does is not translatable when I try be formal in the derivation. So, what should I do to prove that the one-step estimator is efficient?

• NLLS is an M-estimator, so we can get asymptotic variance by simply using the formula for asymptotic variance of an M-estimator:$$\frac{1}{n}A^{-1}BA^{-1}$$ where $A$ is the expected value of the hessian matrix of the objective function and $B$ is the expected value of the score (jacobian) matrix squared. Because we have a least squares estimator, the objective function is trying to minimize squared errors, so that should give you the objective function. – DornerA Apr 19 '16 at 13:24
• @DornerA I was expecting an answer without resorting to that perspective of M-estimators... – An old man in the sea. Apr 19 '16 at 13:41
• It was just a suggestion because it is an easy way to find the asymptotic distribution – DornerA Apr 19 '16 at 16:18
• @DornerA I think I may have seemed like a bit rude. Sorry. Thanks for your comment. I know that we could use the usual derivations for M-estimators, but I'm interested in this particular case, out of curiosity. Anyhow, thanks for your comment. – An old man in the sea. Apr 19 '16 at 18:38