# What is the relationship between the HJB and “Hamiltonian”? Why is the Hamiltonian H(p) inside the HJB?

Deterministic Optimal Control Problem
$$V(a_t) = \underset{c}{\max} \int_{\tau =t}^{\tau = \infty} e^{-\rho (\tau - t) } u(c_{\tau}) d\tau$$
s.t. $$\frac{da}{d\tau} = \left( r a_{\tau} - c_{\tau} \right)$$

CV Hamiltonian: $$H^{CV} \equiv u(c_t) + \lambda_{t} \times (r a_{t} - c_{t})$$
$$H^{CV}_c=0$$, $$H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$$, $$H^{CV}_\lambda = \frac{da}{dt}$$
Boils down to system of boundary-value DAEs.
If you can eliminate $$\lambda$$, system of BV-ODEs.

HJB: $$\rho V(a_t) = \underset{c}{\max} \{ u(c_{t}) + V_a \times (r a_{t} - c_{t}) \}$$

Authors often define the term in the HJB which depends on the control as "Hamiltonian": $$H\left( V_a\right) \equiv \underset{c}{\max} \{ u(c) - c_{t} \times V_a \} \Rightarrow c(a_t) =u'^{-1} \left( V_a(a_t) \right)$$
Hence $$\Leftrightarrow \rho V(a_t) = H\left( V_a\right) + V_a \times r a_{t}$$

Adrien Bilal (p9) and Neil Walton & others, call the term inside the HJB $$H\left( V_a\right)$$ a Hamiltonian.
If we set $$\lambda_{t} \equiv V_a$$ then $$H^{CV} = H\left( V_a\right) + \lambda \times r a_{t}$$.

Q1: why do they add the term $$H\left( V_a\right)$$ inside the HJB?
-guess: it separates the non-linear part of HJB
-guess: it separates the part of the HJB that depends on the control
Q2: why do they call $$H\left( V_a\right)$$ a "hamiltonian"?

Note: I am NOT asking if the solution to the HJB is the same as to the Hamiltonian.
Sol to HJB: $$V(a_t)$$ gives policy $$c(a_t) = u'^{-1}(V_a)$$
Combine policy w/ transition eqn: you get $$c^{HJB}(t)$$
Sol to CV Hamiltonian: $$c^{H}(t)$$
I can show (under appropriate assumptions) that $$c^{HJB}(t)=c^{H}(t)$$.

(side-node) Recall: $$H^{CV}_c=0$$ & $$H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$$
$$H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$$: implies $$\lambda r = \rho \lambda - \frac{d\lambda}{dt}$$ implies $$V_a (\rho-r) = V_{aa} (r a_{t} - c_{t})$$

Since you mention Walton, here is something from his notes page 111 Definition 4:

The Hamiltonian and is defined $$H(t,x,\rho) := min _v \{c(t,x,v) - \rho v\}$$

Notice the analogue to the Hamiltonian you have written.

$$v$$ in here is the control (your $$c$$), $$c(t,x,v)$$ in here is the $$u(c)$$ function you have and $$-\rho = \partial_xL(t,x)$$ in here where $$L$$ is the minimized loss, analogous to your maximized value.

Answer to your second question: That is simply how a Hamiltonian is defined.

Answer to your first question: When the HJB is derived it has that specific form. For example see Benjamin Moll's notes, slides 8-10.

I hope this helps.

References: