Dynamic Optimization: What if the second order condition does not hold?

Question

Consider the following dynamic optimization problem \begin{align} &\max_u \int^T_0{F(x,u)dt}\\ \text{s.t.}~& \dot{x} = f(x,u) \end{align}

FOCs

The Hamiltonian is given by \begin{align} H(x,u,\lambda) = F(x,u) + \lambda f(x,u) \end{align} The necessary condtions for optimality are given by the maximum principle \begin{align} \frac{\partial H}{\partial u} &= 0\\[2mm] \frac{\partial H}{\partial x} &= -\dot{\lambda} \end{align}

Suppose $u^*=\arg\max_u H(x,u,\lambda)$ is a maximizer, i.e. $H_{uu} < 0$.

SOC

The Arrow Sufficient Theorem states, that the necessary condtions are sufficient if the maximized Hamiltonian \begin{align} H^0(x,\lambda) = \max_u H(x,u,\lambda) \end{align} is concave in $x$, i.e. if $H_{xx} < 0$.

Problem

Suppose the FOCs hold, but the SOC fails to hold.

• What can be said about the optimality of the solution?

Answer 1

There is not a single answer, it will depend on the particulars of each problem. Let's look at a standard example.

Consider the benchmark intertemporal optimization problem for the Ramsey model

$$\begin{align} &\max_u \int^{\infty}_0{e^{-\rho t}u(c)dt}\\ \\ & \text{s.t.}\;\; \dot{k} = i-\delta k\\ & \text{s.t.}\;\; y = f(k)=c+i \end{align}$$

The current value Hamiltonian is

$$\tilde H = u(c) +\lambda [f(k)-c-\delta k]$$

Maximizing over $c$ alone we have

$$\frac {\partial \tilde H}{\partial c} = u'(c) - \lambda =0 \implies u'(c^*) = \lambda \implies c^* = (u')^{-1}(\lambda)$$

and the 2nd-order condition will hold if the utility function is concave, $$\frac {\partial^2 H}{\partial c^2} = u''(c^*) < 0$$

Moreover, from the first-order condition with respect to consumption, $\lambda >0$ if local non-satiation holds. Assume that we do have such "usual" preferences.

The maximized over consumption Hamiltonian is

$$\tilde H^0 = u[(u')^{-1}(\lambda)]+\lambda [f(k)-(u')^{-1}(\lambda)-\delta k]$$

The partial derivatives with respect to the state variable, $k$ are

$$\frac {\partial \tilde H^0}{\partial k} = \lambda[f'(k) - \delta], \;\;\;\; \frac {\partial^2 \tilde H^0}{\partial k^2} = \lambda f''(k)$$

So here, the Arrow-Kurz sufficiency condition boils down to whether the marginal product of capital is decreasing, constant, or increasing (which will depend on the sign of the second derivative of the production function). In the standard case $f''(k) < 0$ and we have the sufficient condition.

In the most famous case of deviation, Romer's $AK$ model that initiated the Endogenous Growth literature, $f''(k) =0$, and the marginal product of capital is a positive constant.

So what can we say in this case?

Here, Seierstad, A., & Sydsaeter, K. (1977). Sufficient conditions in optimal control theory. International Economic Review, 367-391. provide various results that can help us.

In particular, they prove that if the Hamiltonian is jointly concave in $c$ and $k$, it is a sufficient condition for a maximum. The Hessian of the Hamiltonian is

(we can ignore the discount term)

$${\rm He}_H = \left [ \begin{matrix} u''(c) & 0\\ 0 & \lambda f''(k)\\ \end{matrix} \right]$$

In the standard case with $u''(c) <0, \; f''(k) <0$ this is a negative definite matrix and so the Hamiltonian is jointly strictly concave in $c$ and $k$.

When $f''(k) =0$, checking that the matrix is negative-semidefinite is straightforward using the definition. Consider a vector $\mathbf z = (z_1, z_2)^T \in \mathbb R^2$ and the product

$$\mathbf z^T{\rm He}_H\mathbf z = z_1^2u''(c) \leq 0$$

this weak inequality holds $\forall \mathbf z \in \mathbb R^2$, and so the Hessian is jointly concave in $c$ and $k$.

So in the $AK$ model of endogenous growth, the solution is indeed a maximum (subject to the parameter constraints needed for the problem to be well-defined of course).