# Multiple solutions to an HJB, how to pin down the optimal "viscosity" solution?

Consider the deterministic consumption-savings problem:
$$V(a_t) = \underset{c}{\max} \int_{\tau =t}^{\tau = \infty} e^{-\rho (\tau - t) } u(c_{\tau}) d\tau$$ w/ $$u(c)=\frac{c^{1-\gamma}-1}{1-\gamma}$$ and $$\gamma, \rho >0$$
s.t.
$$\frac{da}{d\tau} = \left( r a_{\tau} - c_{\tau} \right)$$
Initial condition: $$a(0)=a_0$$ given
Terminal condition: $$\underset{t\to\infty}{\lim} e^{-\rho t} \lambda(t) a_{t} =\underset{t\to\infty}{\lim} e^{-\rho t} u'(c_{t}) a_{t} =\underset{t\to\infty}{\lim} e^{-\rho t} V_{a}(a_{t}) a_{t} =0$$

Let's call the optimization problem above "Sequential Problem" SP & denote the set of solutions $$\text{Sol}(\textbf{SP}):= \{V(a_t), c(a_t): \text{solve }\textbf{SP} \}$$.
Under routine assumptions $$\text{Sol}(\textbf{SP})$$ is a set w/ one element.

Using the Hamiltonian we can solve a system of two BV-ODEs in closed form.
Define: $$\omega \equiv \left(\frac{r-\rho}{\gamma}\right)$$
$$c(a_{t}) = \left(r - \omega \right) \times a_{t}$$
$$a_{t}=a_{0}e^{\omega t}$$
$$\text{TVC: } \underset{t\to\infty}{\lim} e^{-\rho t} u'(c_{t}) a_{t} = \underset{t\to\infty}{\lim} e^{-\rho t} \left( (r - \omega)a_{t} \right)^{-\gamma} a_{t} = \underset{t\to\infty}{\lim} e^{-\rho t} \left( a_{t} \right)^{1-\gamma} = \underset{t\to\infty}{\lim} e^{-\rho t} \left( a_{0}e^{\omega t} \right)^{1-\gamma} = \underset{t\to\infty}{\lim} e^{-\rho t} e^{(1-\gamma) \omega t} = \underset{t\to\infty}{\lim} e^{-(\rho - (1-\gamma) \omega ) t} =0 \Leftrightarrow \rho - (1-\gamma) \omega >0$$
Note: $$\rho - (1-\gamma) \omega = r - \omega$$ TVC holds $$\Leftrightarrow \rho - (1-\gamma) r >0$$
$$V(a_{t}) = \frac{-1}{(1-\gamma)\rho} + \frac{1}{\rho - (1-\gamma)\omega} \frac{\left( \left(r - \omega \right) \times a_{t} \right)^{1-\gamma} }{1-\gamma}$$
Note: $$V_{a}= \frac{\left(r - \omega \right)}{\rho - (1-\gamma)\omega} \left( \left(r - \omega \right) \times a_{t} \right)^{-\gamma} = \left( \left(r - \omega \right) \times a_{t} \right)^{-\gamma} \Rightarrow c(a_t) =u'^{-1}(V_a)$$

HJB: $$\rho V(a_t) = \underset{c}{\max} \{ u(c_{t}) + V_a \times (r a_{t} - c_{t}) \}$$
Denote the set of solutions $$\text{Sol}(\textbf{HJB})$$.
FONC: $$u'(c_{t}) - V_a =0 \Rightarrow c(a_t) = (V_a)^{-\frac{1}{\gamma}} \Rightarrow V_a > 0$$
Plugin: $$\Rightarrow u(c) - cV_a = \frac{\gamma (V_a)^{1-\frac{1}{\gamma}}}{1-\gamma} - \frac{1}{1-\gamma}$$
SOSC: $$u''(c_{t})<0 \Rightarrow u''\left( (V_a)^{-\frac{1}{\gamma}} \right)<0$$
We can combine these eqns (maximized HJB & FONC)
$$\left[\begin{array}{l} \rho V(a_{t}) = u(c_{t}) + V_{a}(a_{t})\times \left(ra_{t} - c_{t} \right) \\ c(a_{t}) = (V_{a}(a_{t}))^{-\frac{1}{\gamma}} \end{array} \right] \Leftrightarrow \left[\begin{array}{l} \rho V(a_{t}) = \frac{\gamma (V_a)^{1-\frac{1}{\gamma}}}{1-\gamma} - \frac{1}{1-\gamma} + V_a \times r a_{t} \end{array} \right]$$
DE: $$\rho V(a_t) = \frac{\gamma (V_a)^{1-\frac{1}{\gamma}}}{1-\gamma} - \frac{1}{1-\gamma} + V_a \times r a_{t}$$
Denote the set of solutions $$\text{Sol}(\textbf{DE})$$.

DE is a non-linear equation (first-order ODE) w/ multiple solutions $$V(a_t)$$.

• Sol 1: $$V(a)=\frac{1}{\rho (\gamma-1)}$$, solves DE, if $$\gamma>1$$
Note: while Sol 1 solves DE it does not solve the $$\max$$ part of HJB, e.g. adding $$V_a >0$$ to DE will rule out Sol 1, but this condition isn't enough to pin down the sol to SP.
• Sol 2: $$V(a)=B_0 + B_1 a$$, solves DE, if $$r=\rho$$ and $$\rho B_0 = \frac{\gamma (B_1)^{1-\frac{1}{\gamma}}}{1-\gamma} - \frac{1}{1-\gamma}$$.
Note: condition $$V_a>0$$ is satisfied if $$B_1 >0$$.
• Sol 3: Walde 2010 claims this problem also has a strictly convex solution (I think) in a note I don't fully understand
• Sol 4: $$V(a_{t}) = \frac{-1}{(1-\gamma)\rho} + \frac{1}{\rho - (1-\gamma)\omega} \frac{\left( \left(r - \omega \right) \times a_{t} \right)^{1-\gamma} }{1-\gamma}$$, solves DE

We have: $$\text{Sol}(\textbf{SP}) \subseteq \text{Sol}(\textbf{HJB}) \subseteq \text{Sol}(\textbf{DE})$$

Q1: what conditions does $$V(a)$$ need to satisfy s.t. the solution to DE is also the solution to the optimization problem SP?

• My understanding of viscosity for dummies is that while DE has many solutions, the optimal solution is the unique "viscosity solution"

Q2 why do the constant & affine solutions above not satisfy the conditions for a viscosity solution?
-the authors of "viscosity for dummies" don't provide a simple example of an HJB w/ multiple closed form solutions & show only the optimal solution satisfies the properties

If we re-write FONC as a function of $$a$$: $$u'(c(a))-V_{a} =0$$
Differentiate wrt $$a$$ (as in Walde 2010): $$u''(c(a)) c'(a) -V_{aa} =0$$
We know from SOSC that $$u''(c(a))<0$$.
If we assume consumption is increasing in wealth $$c'(a)>0$$, then $$V_{aa}<0$$

$$\left[\begin{array}{l} \rho V(a_{t}) = \frac{\gamma (V_a)^{1-\frac{1}{\gamma}}}{1-\gamma} - \frac{1}{1-\gamma} + V_a \times r a_{t} \\ V_{a}(a_{t}) >0 \text{ FONC: for max to be well defined} \\ V_{aa}(a_{t}) <0 \text{ differentiate FONC & SOSC} \end{array} \right]$$

• I think it's enough for $$V_a>0, V_{aa}<0$$ to hold at a single point
• I believe if we add these two conditions (increasing & concave value) to DE we can pin down the unique optimal sol in this non-generic example.
I cannot prove there is no other sol to DE w/ $$V_a>0, V_{aa}<0$$.
I really wanna know how these conditions generalize to more generic econ problems.
• $$V_a>0$$ rules out Sol 1
• $$V_{aa}<0$$ rules out Sols 2 & 3
• These conditions don't feel as essential/generic as $$a(0)=a_0$$ & TVC.
• Since DE is a first-order non-linear ode, I believe we either need one equation, such as $$V(a_{1})=V_1$$, or two inequalities ($$V_a>0, V_{aa}<0$$) to pin down a unique sol.
(I'm not sure about this. It may only be true for "nice" odes. Can someone who knows help?)
Since DE is first-order, I don't like relying on a boundary condition w/ a 2nd derivative $$V_{aa}<0$$
• Achdou Hans Lasry Lions Moll add a boundary condition that doesn't bind $$a_t \geq \underline{a}$$ where $$\underline{a}<0$$ and use a "state-boundary condition" $$V_a(\underline{a}) \geq u'(r\underline{a})$$.
Issue: if $$\gamma \in (0,1)$$ then $$u'(r\underline{a})$$ may not be a real number.
Is one inequality enough theoretically, even if numerically their method converges to the unique sol?
Does this rule out the affine solutions above?
And how does this generalize to generic problems?
• Most importantly, why do the constant & affine solutions above not satisfy the conditions for a viscosity solution? Why does the optimal sol satisfy them?
• Hey Albert. Cannot really provide the necessary and/or sufficient conditions to have a coincidence of solutions but may add something to your own answer. $V_a>0$ should be a given by non-satiation of the utility function and the FOC. The additional condition on $V_{aa}(a)$, pointed out in Walde (2010), should precisely rule out solutions of the HJB that are not solutions to the maximization problem. If for all state variable levels, $V_{aa}(a) < 0$, then $c'(a)<0$ and we can argue by contradiction. (continued) Jul 27 at 13:34
• For any initial condition $a_0$, take $a_0' > a_0$. $c_t' = c(a_0') < c_t = c(a_0)$ and so we will have that for $t = \epsilon$, $a_{\epsilon}' > a_{\epsilon}$. The same argument applies for any time period, $t \geq 0$. This means that consumption will be lower at all points in time, $c_t' \leq c_t$. We reached our contradiction. Just take in turn the previous consumption plan - also achievable with the new initial condition. By the monotonicity of the integral, the value function would then be higher. Hence $c_t' \quad \forall t \geq 0$, determined by the proposed solution, cannot be optimal. Jul 27 at 13:36
• For now I cannot truly rule out solution 2 since $V_{aa}(a)=0$. If $r=\rho$ and $B_0$ respects the equality imposed, we can then solve for $B_1$ using the boundary conditions. For $B_1$ to respect the terminal condition we must have that $B_1 = (ra_0)^{-\gamma}$. This means that $B_1$ cannot be a constant independent of the boundary conditions. (continued) Jul 27 at 14:36
• This conclusion is intuitive if we just look at the FOC which sets $c(a)$ to a constant. Take a starting $a_0 \leq 0$ for example. The $\lim_{t \rightarrow \infty} a_t$ diverges as $\dot{a_t}<0 \forall t \geq 0$ and the tvm is not respected. In fact, as long as $a_0 \neq a*$, for a certain $a*$, the limit will always diverge and the tvm will not be respected. Jul 27 at 14:40

Maybe I am completely wrong here (given that I don't see the need to talk about viscosity solutions at all) but in the standard representation theorems you have a terminal/limit condition that the solution to the HJB has to satisfy for it to be the value function. This involves checking, for any admissible control, $$\lim_{T \to \infty} E[ e^{-\rho T} V(a_T)] = 0$$ Your affine solution to DE clearly violates this when $$r = \rho$$.

• I'm not sure your limit condition is enough to pin down the optimal solution. Sol 1: $\lim_{T\to \infty} e^{-\rho T} \frac{1}{\rho (\gamma -1)} =0$, but it is not optimal. Sol 2: $\lim_{T\to \infty} e^{-\rho T} (B_0 + B_1 a_{T}) =a_{0} B_{1} \lim_{T\to \infty} e^{-(r-\omega)T} = 0$, but it's not optimal Jul 23 at 21:11
• I agree that this does not rule out Sol 1 - but for Sol 2, it must hold for ANY admissible control, not just the optimal one. So i can take $c_t = 0$ for all $t$ Jul 23 at 21:18