4
$\begingroup$

Deterministic Optimal Control Problem
$ V(a_t) = \underset{c}{\max} \int_{\tau =t}^{\tau = \infty} e^{-\rho (\tau - t) } u(c_{\tau}) d\tau $
s.t. $ \frac{da}{d\tau} = \left( r a_{\tau} - c_{\tau} \right) $

CV Hamiltonian: $H^{CV} \equiv u(c_t) + \lambda_{t} \times (r a_{t} - c_{t})$
$H^{CV}_c=0$, $H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$, $H^{CV}_\lambda = \frac{da}{dt}$
Boils down to system of boundary-value DAEs.
If you can eliminate $\lambda$, system of BV-ODEs.

HJB: $\rho V(a_t) = \underset{c}{\max} \{ u(c_{t}) + V_a \times (r a_{t} - c_{t}) \}$

Authors often define the term in the HJB which depends on the control as "Hamiltonian": $H\left( V_a\right) \equiv \underset{c}{\max} \{ u(c) - c_{t} \times V_a \} \Rightarrow c(a_t) =u'^{-1} \left( V_a(a_t) \right)$
Hence $\Leftrightarrow \rho V(a_t) = H\left( V_a\right) + V_a \times r a_{t}$

Adrien Bilal (p9) and Neil Walton & others, call the term inside the HJB $H\left( V_a\right)$ a Hamiltonian.
If we set $\lambda_{t} \equiv V_a$ then $H^{CV} = H\left( V_a\right) + \lambda \times r a_{t}$.

Q1: why do they add the term $H\left( V_a\right)$ inside the HJB?
-guess: it separates the non-linear part of HJB
-guess: it separates the part of the HJB that depends on the control
Q2: why do they call $H\left( V_a\right)$ a "hamiltonian"?

Note: I am NOT asking if the solution to the HJB is the same as to the Hamiltonian.
Sol to HJB: $V(a_t)$ gives policy $c(a_t) = u'^{-1}(V_a)$
Combine policy w/ transition eqn: you get $c^{HJB}(t)$
Sol to CV Hamiltonian: $c^{H}(t)$
I can show (under appropriate assumptions) that $c^{HJB}(t)=c^{H}(t)$.

(side-node) Recall: $H^{CV}_c=0$ & $H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$
$H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$: implies $\lambda r = \rho \lambda - \frac{d\lambda}{dt}$ implies $V_a (\rho-r) = V_{aa} (r a_{t} - c_{t})$

$\endgroup$
1
$\begingroup$

Since you mention Walton, here is something from his notes page 111 Definition 4:

The Hamiltonian and is defined $H(t,x,\rho) := min _v \{c(t,x,v) - \rho v\}$

Notice the analogue to the Hamiltonian you have written.

$v$ in here is the control (your $c$), $c(t,x,v)$ in here is the $u(c)$ function you have and $-\rho = \partial_xL(t,x)$ in here where $L$ is the minimized loss, analogous to your maximized value.

Answer to your second question: That is simply how a Hamiltonian is defined.

Answer to your first question: When the HJB is derived it has that specific form. For example see Benjamin Moll's notes, slides 8-10.

I hope this helps.

References:

Walton: https://appliedprobability.files.wordpress.com/2021/01/stochastic_control_jan29.pdf

Moll: https://benjaminmoll.com/wp-content/uploads/2019/07/Lecture3_2149.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.