3
$\begingroup$

Consider a routine continuous time optimization problem:
$ V(t,a_{t}) := \max \int_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)} u(c_{\tau})d\tau $ $\text{ s.t. }$
$\dot{a}_{t} = y + ra_{t} - c_{t}$,
$a_{0} \text{ given, } a_{T}=0.$
Assume y & r are constants and $u(c)=\frac{c^{1-\gamma}}{1-\gamma}$.

Optimal control approach:
$H := u(c_{t}) + \lambda (y + ra_{t} - c_{t})$
Boils down to a BVP (two ODEs w/ unkown $c_{t}, a_{t}$):
$\left[\begin{array}{l} \dot{c}=\left(\frac{r-\rho}{\gamma} \right) c \\ \dot{a}_{t} = y + ra_{t} - c_{t} \\ a(0)=a_0 \\ a_{T}=0 \end{array} \right]$.
This can be plugged into a DiffEq solver bc we have two first-order ODEs w/ two boundary conditions.

HJB approach:
$\rho V(t,a_{t}) = \max_{c} \left\{u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) \right\}$
FOC: $c(t,a_{t})=u'^{-1}(V_{a}(t,a_{t})) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}}$
This boils down to a PDE w/ unknown function $V(t,a_{t})$:
$\left[\begin{array}{l} \rho V(t,a_{t}) = u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) \\ c(t,a_{t}) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}} \end{array} \right]$
To solve this first-order PDE we need one boundary condition for $V(t,a_{t})$.
Issue: we have boundary conditions for $a_{t}$ $(a(0)=a_0, a_{T}=0)$ but not for $V(t,a_{t})$.
Question: How do we get the boundary conditions for HJBs in general?

Partial answer (case 1): finite-horizon deterministic problems
$V(T,a_{T}) = \psi(a_{T}) = 0$ (no bequest here by assumption) $\left[\begin{array}{l} \rho V(t,a_{t}) = u(c_{t}) + V_{a}(t,a_{t})\times \left(y + ra_{t} - c_{t} \right) + V_{t}(t,a_{t}) \\ c(t,a_{t}) = (V_{a}(t,a_{t}))^{-\frac{1}{\gamma}} \\ V(T,a) = 0 \text{ , } \forall a \end{array} \right]$

Generically, many problems in economics can be written:
$V(t,x_{t}) := \max \int_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)}u(\tau,x_{\tau},c_{\tau})d\tau + \Omega(T,x_{T})$ $\text{ s.t. }$
$\dot{x}_{t} = g(t,x_{t},c_{t})$,
$x_{0} \text{ given, } x_{T}=0.$

The generic optimal control problem is straightforward, w/ boundary conditions $x_{0}, x_{T}=0$.

Generic deterministic HJB:
$\left[\begin{array}{l} \rho V(t,x_{t}) = \max_{c} \left\{u(t,x_{t},c_{t}) + V_{x}(t,x_{t}) \dot{x} + V_{t}(t,x_{t}) \right\} \\ V(T,x_{T}) = \Omega(T,x) \text{ , } \forall x \end{array} \right]$

Case 2: finite horizon stochastic problems (w/ Ito processes)
$V(t,x_{t}) := \max \int_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)} u(\tau,x_{\tau},c_{\tau})d\tau + \Omega(T,x_{T})$ $\text{ s.t. }$
$dx=\mu_{t}dt + \sigma_{t} dz_{t}$, where $\mu_{t} = g(t,x_{t},c_{t})$,
$x_{0} \text{ given, } x_{T}=0.$
$\rho V(t,x_{t}) = \max_{c} \left\{ u(t,x_{t},c_{t}) + E_{t}\frac{dV}{dt}\right\} $.
Ito's Lemma: if $X \sim dX=\mu_{t}dt + \sigma_{t} dZ_{t}$ $\Rightarrow$ $E_{t}\frac{dV(t,x_{t})}{dt} = V_{t}(t,x_{t}) + \mu_{t}V_{x}(t,x_{t}) + \frac{\sigma_{t}^2}{2} V_{xx}(t,x_{t}) $
$\left[\begin{array}{l} \rho V(t,x_{t}) = \max_{c} \left\{ u(t, x_{t},c_{t}) + V_{t}(t,x_{t}) + \mu_{t}V_{x}(t,x_{t}) + \frac{\sigma_{t}^2}{2} V_{xx}(t,x_{t}) \right\} \\ V(T,x_{T}) = \Omega(T,x) \text{ , } \forall x \end{array} \right]$
Question 2: a generic HJB for a stochastic problem (w/ Ito risk) is a second-order PDE and we need two boundary conditions.
What is the second boundary condition for $V(t,x_{t})$?
My guess:
$\left[\begin{array}{l} \rho V(t,x_{t}) = \max_{c} \left\{ u(x_{t},c_{t},t) + V_{t}(t,x_{t}) + \mu_{t}V_{x}(t,x_{t}) + \frac{\sigma_{t}^2}{2} V_{xx}(t,x_{t}) \right\} \\ V(T,x_{T}) = \Omega(T,x) \text{ , } \forall x \\ V_{x}(T,x_{T}) = \Omega_{x}(T,x) \text{ , } \forall x \end{array} \right]$

The two boundary conditions for the finite-horizon pdes above are usually called:
Value matching: $V(T,x_{T}) = \Omega(T,x_{T}) $
Smooth pasting: $V_{x}(T,x_{T}) = \Omega_{x}(T,x_{T}) $
In the original example $\Omega(T,x_{T}) =0$.

Remaining questions: what are the value matching & smooth pasting conditions for generic infinite horizon problems?
I've seen:
$\lim_{t \to \infty} E[ e^{-\rho t} V(x_{t}) ]=0$
$\lim_{t \to \infty} E[ e^{-\rho t} V_{x}(x_{t}) ]=0$

Case 3: What is the boundary condition for an infinite horizon deterministic HJB?

Case 4: What is the boundary condition for an infinite horizon stochastic HJB?

Case 5: Problems with borrowing constraints etc studied by Moll and Gomez et al?

Lesson 1: PDEs are ALOT more complicated than ODEs.
An nth order ODE needs n boundary conditions.
An nth order hyperbolic PDE (wave eqn) needs n boundary conditions.
An nth order elliptic equation needs n/2 boundary conditions.
Is there a general rule for the HJB PDEs economists usually solve?

Lesson 2: HJB PDEs are usually solved w/ viscosity methods.

See Moll, Lifecycle (finite horizon).
See Moll, Lecture 3, Slide 11.
Achdou, Han, Lasry, Lions, Moll (2020) is the most seminal paper in this literature, but doesn't really answer this question.
This is challenging b/c it's hard to plug a pde into a pde-solver w/o the proper boundary conditions.

$\endgroup$
3
  • $\begingroup$ The terminology "value matching" and "smoothing pasting" are not being used correctly here. They occur in free boundary problems---e.g. exercise of American options or real options. There is no "boundary condition" at infinity for HJB equations. Rather HJB conditions are only necessary. Sufficiency is given by transversality-type conditions at infinity. In stochastic control, such sufficiency results are called verification theorems. One needs to first solve the HJB, then apply verification (second step commonly not done in economics). $\endgroup$
    – Michael
    Nov 7 '20 at 4:26
  • $\begingroup$ Whatever the terminology, those conditions give the correct solution in the first two cases. See Laibson's lecture notes. $\endgroup$ Nov 7 '20 at 5:04
  • $\begingroup$ Evidently you're misreading the terminology from the linked notes. $\endgroup$
    – Michael
    Nov 8 '20 at 2:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.