A number of proofs in optimisation use the idea that the remainder term in either the differential or the Taylor Approximation go to zero. For example:
- Some envelope theorem proofs:.
- Necessity and Sufficiency of FOC & SOC in optimisation proofs.
I'm struggling to understand the Remainder term going to 0 as explained below.
1) Removing the remainder term from the differential
Used in an envelope theorem proof:
Lets say we have a constraint function $G(\mathbf{(x(θ), θ)} =b$
The differential for $G$ when $θ _ι$ changes is:
$∆G = \sum_{j=1}^{n}[\frac{\partial{G}}{\partial{x_j}}∆x_j] + \frac{\partial{G}}{\partial{θ_i}}∆θ_i +R = 0$ Where R is our remainder term:
Then dividing through by $∆θ_i$ and taking the $\lim \limits_{∆θ_i \to 0}$ we get:
$\lim \limits_{∆θ_i \to 0} \frac{∆G}{∆θ_i} = \lim \limits_{∆θ_i \to 0} \sum_{j=1}^{n}[\frac{\partial{G}}{\partial{x_j}}\frac{∆x_j}{∆θ_i}] + \frac{\partial{G}}{\partial{θ_i}} +\frac{R}{∆θ_i} = 0$
- Question: How do I show that: $\lim \limits_{∆θ_i \to 0} \frac{R}{∆θ_i} = 0$
I understand why R would go to 0. Because as the change approaches 0 then $f(x) = P(x)$ Where $P(x)$ is our Taylor Polynomial, and hence the difference $R(x)$ goes to 0.
But then don't we have $\frac{0}{0}$ and i don't know how to resolve this in this case?
2) Removing the remainder term from the second order Taylor Polynomial
I've seen this written as Taylors Theorem I've seen this be used to prove the necessity and sufficiency of FOC and SOC.
If we define $R_2(∆x) = f(x) - P_2(x)$ where $P_2(x)$ is the second order Taylor polynomial around $a$.
- Question: How do we show that: $\lim \limits_{∆x_i \to 0} \frac{R_2(∆x)}{(∆x)^2} = 0$
Defining $∆x = x - a$ My best effort so far from expanding $f(x)$ and $P_2(x)$, we get:
$\lim \limits_{∆x_i \to 0} [\frac{f'(a)}{∆x} - \frac{f'(a)}{∆x} - \frac{1}{2}f''(a)]$ = $- \frac{1}{2}f''(a)$ = $- \frac{1}{2}f''(x) ≠ 0$
If anyone can help with these it would solve many riddles for me. Expanding to $\mathbb{R^n}$ and other generalised cases I should then be able to do myself, thanks!