# Remainder term in Linear Approximations going to 0

A number of proofs in optimisation use the idea that the remainder term in either the differential or the Taylor Approximation go to zero. For example:

1. Some envelope theorem proofs:.
2. Necessity and Sufficiency of FOC & SOC in optimisation proofs.

I'm struggling to understand the Remainder term going to 0 as explained below.

1) Removing the remainder term from the differential

Used in an envelope theorem proof:

Lets say we have a constraint function $$G(\mathbf{(x(θ), θ)} =b$$

The differential for $$G$$ when $$θ _ι$$  changes is:

$$∆G = \sum_{j=1}^{n}[\frac{\partial{G}}{\partial{x_j}}∆x_j] + \frac{\partial{G}}{\partial{θ_i}}∆θ_i +R = 0$$ Where R is our remainder term:

Then dividing through by $$∆θ_i$$ and taking the $$\lim \limits_{∆θ_i \to 0}$$ we get:

$$\lim \limits_{∆θ_i \to 0} \frac{∆G}{∆θ_i} = \lim \limits_{∆θ_i \to 0} \sum_{j=1}^{n}[\frac{\partial{G}}{\partial{x_j}}\frac{∆x_j}{∆θ_i}] + \frac{\partial{G}}{\partial{θ_i}} +\frac{R}{∆θ_i} = 0$$

• Question: How do I show that: $$\lim \limits_{∆θ_i \to 0} \frac{R}{∆θ_i} = 0$$

I understand why R would go to 0. Because as the change approaches 0 then $$f(x) = P(x)$$ Where $$P(x)$$ is our Taylor Polynomial, and hence the difference $$R(x)$$ goes to 0.

But then don't we have $$\frac{0}{0}$$ and i don't know how to resolve this in this case?

2) Removing the remainder term from the second order Taylor Polynomial

I've seen this written as Taylors Theorem I've seen this be used to prove the necessity and sufficiency of FOC and SOC.

If we define $$R_2(∆x) = f(x) - P_2(x)$$ where $$P_2(x)$$ is the second order Taylor polynomial around $$a$$.

• Question: How do we show that: $$\lim \limits_{∆x_i \to 0} \frac{R_2(∆x)}{(∆x)^2} = 0$$

Defining $$∆x = x - a$$ My best effort so far from expanding $$f(x)$$ and $$P_2(x)$$, we get:

$$\lim \limits_{∆x_i \to 0} [\frac{f'(a)}{∆x} - \frac{f'(a)}{∆x} - \frac{1}{2}f''(a)]$$ = $$- \frac{1}{2}f''(a)$$ = $$- \frac{1}{2}f''(x) ≠ 0$$

If anyone can help with these it would solve many riddles for me. Expanding to $$\mathbb{R^n}$$ and other generalised cases I should then be able to do myself, thanks!

• Question: How do I show that: $$\lim \limits_{∆θ_i \to 0} \frac{R}{∆θ_i} = 0$$

[...]

• Question: How do we show that: $$\lim \limits_{∆x_i \to 0} \frac{R_2(∆x)}{(∆x)^2} = 0$$

As you can see from your questions, it must be proved not only that the remainders $$R_1$$ or $$R_2$$ go to zero, but the ratio between the remainders and $$(\Delta x) ^2$$ or $$\Delta \theta$$ goes to zero.

That is, the remainder $$R_n$$ in Taylor's formula is an infinitesimal of greater order with respect to $$(\Delta x)^n$$. This is the important theorem that must be proved with regard to Taylor's formula.

Intuitively, this means that the remainder $$R_n$$, of a Taylor's polynomial of order $$n$$,$$^1$$ goes to zero more 'quickly' tha $$(\Delta x)^n$$.

Therefore, it is not sufficient to prove that $$R_n$$ goes to zero, because it can go to zero 'slowly', so that the ratio $$\frac {R_n}{\Delta x} \nrightarrow 0$$.

This is important, as a Taylor's polynomial is an approximation of the function at a point: for example, intuitively, if when $$\Delta x$$ is very small the remainder is very high, we haven't a good approximation.

As you rightly noticed, this ratio, as $$\Delta x \rightarrow 0$$, is the indeterminate form $$0/0$$. Therefore, the proof uses L'Hôpital's rule.

I give you below a proof that assumes that the derivatives of the function are continuos, but a similar proof exists for the general case.$$^2$$

$$***$$

THEOREM: TAYLOR'S FORMULA. - Let $$f: \mathbb{R} \rightarrow \mathbb{R}$$ be a function and the derivatives $$f^{(n)}$$ exist and are continuous at a point $$x_0$$.

We have

$$f(x)= \sum_{k=0}^{n}\frac { f^{(k)}(x_0)} {k!}(x-x_o)^k+R_n(x) \qquad (1)$$

and

$$\lim _{x\rightarrow x_0} \frac {R_n(x)}{(x- x_0)^n}=0 \qquad (2).$$

PROOF.

According to the definition of $$R_n$$ we must prove that:

$$\lim _{x\rightarrow x_0} \frac {f(x) -[f(x_0) + f'(x_0) (x-x_0)+ ...+f^{(n)} (x_0)(x-x_0)^n/n!]}{(x- x_0)^n}=0 \qquad (3)$$

We now use l'Hôpital rule. The derivative is respect to $$x$$, so the derivative of $$f(x_0)$$ is zero, whereas the derivative of $$\frac {f^{(n)}(x-x_0) ^n}{n!}$$ is

$$\frac {f^{(n)}(x_0)n (x-x_0) ^{n-1}}{n!}= \frac {f^{(n)}(x_0) (x-x_0) ^{n-1}}{(n-1)!}\qquad (4)$$

Therefore, the limit $$(3)$$, having applied l'Hôpital, is the same as

$$\lim _{x\rightarrow x_0} \frac {f'(x) -[f'(x_0) + ...+f^{(n)} (x_0)(x-x_0)^{n-1}/(n-1)!]}{n(x- x_0)^{n-1}}. \qquad (5)$$

If $$n>1$$, we have again an indeterminate form $$0/0$$. After applying $$n$$ times l'Hôpital rule, we have:

$$\lim _{x\rightarrow x_0} \frac {f^{(n)} (x)-f^{(n)} (x_0)}{n!} \qquad (6).$$

This last limit is zero, as $$f^{(n)} (x)$$ is continuous, by assumption, at $$x_0$$.

$$\Box$$

If reading the proof with a generic $$n$$ is too cumbersome, you can re-read the method of the proof with $$n=2$$.

$$^1$$ This result holds for every order $$n$$ of the Taylor's polynomial.

$$^2$$ You can find this proof and the overall theory of the Taylor's polynomial in a good book of calculus/ undergraduate text of mathematical analysis.

• This is another great answer @BakerStreet, glad to see that it was something that simply required some extra steps I.e. L'Hpoital, rather than a total misunderstanding of the material! Thanks again for taking the time as usual, and apologies for my delay in responding (Last week was busy). Feb 27 at 13:29
• I will take what you have given me and see if I can use it to make sense of my method for $\lim \limits_{∆x_i \to 0} \frac{R_2(∆x)}{(∆x)^2} = 0$. As in you have already shown me that it works for the nth case...but is it obvious where my approach expanding $f(x)$ and $P_2(x)$ might have gone wrong? Feb 27 at 13:31
• You're welcome, and don't mind the delay, it is normal to be busy! And thank you very much for your kindness. You didn't misunderstand the matter, you just hadn't seen before the proof of Taylor's formula. What expansions do you refer to? Which equations in your question? Feb 27 at 14:37
• Actually, that is the usual habit of mathematicians, otherwise they become crazy to refer to equations. :-) Feb 27 at 15:08
• Yes, I've been contract professor at university for several yesrs, I have been teaching Economics to Law students, who are intelligent students, but they haven't a great aptitude to Economics :-), so a little bit of patience is required. But they were clever and kind students, really nice. Feb 27 at 15:33