1
$\begingroup$

Edit: I have updated the link so that it works!

I was watching a lecture for a proof that if $x^*$ is a local maximiser of $f$ then necessarily the hessian is negative semi definite.

However i've got stuck on this notation: Time stamped Source

$∆y = f(λ∆x) = f(x^* + λ∆x) - f(x^*)$

  • The $x's$ are vectors:
  • He mentions later that "Dividing by $λ^2$ is basically dividing through by the square of the length of the vector $∆x$," which i guess implies that $∆x$ is very small and hence deriving most of it's length from λ?
  1. I'm more used to defining a $∆x = x^* - x$ and $∆y = f(x^*) - f(x)$
  2. I suppose given $y = f(x)$ we could take $y$ as $∆y$ and $x$ as $∆x$ hence we'd have $f(∆x)$ and then really $λ∆x = ∆x$ for some arbitrary change?
  3. Then setting setting this = $f(x^* + λ∆x) - f(x^*)$ is equivalent to something like $f(x^* + hx) - f(x^*)$

But for example if you actually sub in $∆x = x^* - x$ you get something that looks like the convex combination of $x^*$ and $x$, so I'm really not convinced i know whats going on.

If anyone could walk me through this notation that would be great...

Additional Context

For context he then goes onto use the second degree Taylor polynomial: $\nabla fλ∆x^* + \frac{1}{2}λ∆xH(x^*)λ∆x  + R_2(λ∆x)$ Where the final term is just his notation for the remainder term for the second degree Taylor series with respect to our augment, and then divides by $λ^2$ as mentioned, to reduce the remainder to 0.

$\endgroup$
13
  • $\begingroup$ I've also noticed in this video he writes $∆y = ∆f(∆x) = f(x) - f(x^*)$. In general I've found this lecturer to be very good, and usually he even responds to comments, so I'm nervous to call the discrepancy a typo...but I don't really understand what's going on here: youtu.be/KesZslq5uVg?t=1158 $\endgroup$
    – CormJack
    Commented Feb 15, 2023 at 2:43
  • $\begingroup$ AS for $f(\Delta y)$, I saw a very short part of the video of this lesson at minute 19: , it is strange, you are right, one had to see the video from the beginning, but there is something unusual. Coluld be a typo. $\endgroup$ Commented Feb 15, 2023 at 11:02
  • $\begingroup$ Hi BakerStreet, your comment reads as if thee was another part to it before the As?...Perhaps you were referencing the Brocken link in the video which I have now fixed! $\endgroup$
    – CormJack
    Commented Feb 15, 2023 at 13:49
  • $\begingroup$ Yes, the minute 19 of the video, where the professor writes $\Delta f( \Delta x)$ $\endgroup$ Commented Feb 15, 2023 at 13:52
  • $\begingroup$ Ah okay that link in the comments works, you'll notice there is a different link which was Brocken (but now works) in the video which links to the notation I was trying to understand in the question! Subtly different to the notation in the comment...Look forward to your response whenever you get the time! $\endgroup$
    – CormJack
    Commented Feb 15, 2023 at 13:54

1 Answer 1

0
$\begingroup$

Defining the following terms where $x$ and $y$ are vectors:

$∆x = x - x^*$

$\therefore x = ∆x + x^*$

$∆y = ∆f(x) = f(x) - f(x^*) = f(x^* + ∆x) - f(x^*)$

We can draw a fresh axis in terms of $∆x$ and $∆y$ essentially treating $x^*$ and $f(x^*)$ as the origin. This makes sense as $∆x$ and $∆f(x)$ evaluated at $x^* = 0$. We then call this function $F(∆x)$ (See attached diagram)

This gives the required: $∆y = F(∆x) = f(x) - f(x^*) = f(x^* + ∆x) - f(x^*)$

The $∆x$ and $∆y$ axis imposed on the original graph can be seen at this point in the video, the use of $F$ instead of $f$ can be seen here.

Answering Questions

  1. I believe that $∆F(∆x)$ seen here was a typo and this should have either been $∆f(x)$ or $F(∆x)$
  2. What's the benefit of $F(∆x)$? I'm not entirely sure, but I believe the author does this to allow him to talk about Limits as $(∆x) \to 0$
  3. What's going on with $∆y = F(λ{∆x}) = f(x^* + λ{∆x}) - f(x^*)$? Here he notes we are using a "fixed" $∆x$ and then taking λ scalar multiples of it. I suppose this is still just some arbitrary change representing $∆y$.
  4. He uses λ so that he can then divide through by $λ^2$ which he says is the approximate length squared of the vector $(∆x)$ so that we get: $\frac{1}{λ^2}[\nabla f(x^*)•λ(∆x) + \frac{1}{2}λ(∆x)^TH(x^*)λ(∆x)  + R_2(λ{∆x})] = \frac{1}{2}(∆x)^TH(x^*)(∆x)$
  • I have some questions about how we remove the remainder term $\lim \limits_{λ \to 0} \frac{R_2(λ∆x)}{(λ)^2} = 0$ , but I have added these in a new post!
  • Any other thoughts about the notation in general and the use of $F(λ{∆x})$ are welcome.

$∆x$ and $∆y$ axis enter image description here

Clearer explanation of the links:

  1. The First Link is showing an example of the axis using $∆x$ and $∆y$.
  2. The Second Link if you look at the top line (LHS) of his notation we see the use of $F(∆x)$ which is different to $∆f(∆x)$ which is shown in the next link. These two notation are implied to mean the same thing and is a source of confusion discussed here.
  3. The Third Link shows the use of $∆f(∆x)$ notation.
  • Here is an additional example of $(∆x), (∆y)$ axis, when he first uses it. However it's more clear what's happening in the first link (because he gives arrows to the new axis there).
$\endgroup$
3
  • $\begingroup$ I find it , $ F(\Delta x)$, a quite strange notation, but it isn't possible to say anything more precise without seeing the video entirely. We have usaually the Taylor's formula with $x*=0$, that is called the formula of MacLaurin, but there isn't $ F(\Delta x)$, nor a translation af axes, even if could be the same. Maybe the professor here wants you to have a geometric insight, but it is impossible to say it without seeing the video. By the way, the video you linked in the other question maybe is wrong, it isn't the video to which the question refers to. $\endgroup$ Commented Feb 21, 2023 at 13:01
  • $\begingroup$ Hello there! Yes it is unusual I agree, the $F(∆x)$ makes some sense to me now, and I suppose he wanted to write things as a function of $∆x$. That being said I am confused still on $F(∆x)$ vs $∆f(∆x)$...I have made the video links explicit at the end, so that you understand the purpose of them and what to look for in the video. He's a very very good professor, like your explanations usually his are excellent...so perhaps if you do find the time you will get something from it! I'm not too worried now, but the explicit guide to the links at the bottom should help if you're curious! Many thanks! $\endgroup$
    – CormJack
    Commented Feb 27, 2023 at 13:51
  • 1
    $\begingroup$ I will try to listen to the video, I hope it colud be enough , because when there is a particular notation it is explained at the beginning of the lessons. I've never seen in my books such a notation when explaining maximum conditions. $\endgroup$ Commented Feb 27, 2023 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.