# Dynamic Optimization: What if the second order condition does not hold?

Consider the following dynamic optimization problem \begin{align} &\max_u \int^T_0{F(x,u)dt}\\ \text{s.t.}~& \dot{x} = f(x,u) \end{align}

## FOCs

The Hamiltonian is given by \begin{align} H(x,u,\lambda) = F(x,u) + \lambda f(x,u) \end{align} The necessary condtions for optimality are given by the maximum principle \begin{align} \frac{\partial H}{\partial u} &= 0\\[2mm] \frac{\partial H}{\partial x} &= -\dot{\lambda} \end{align}

Suppose $u^*=\arg\max_u H(x,u,\lambda)$ is a maximizer, i.e. $H_{uu} < 0$.

## SOC

The Arrow Sufficient Theorem states, that the necessary condtions are sufficient if the maximized Hamiltonian \begin{align} H^0(x,\lambda) = \max_u H(x,u,\lambda) \end{align} is concave in $x$, i.e. if $H_{xx} < 0$.

## Problem

Suppose the FOCs hold, but the SOC fails to hold.

• What can be said about the optimality of the solution?
• Convexity is not the absence of concavity. – Michael Greinecker Jun 10 '15 at 9:39
• I removed the wrong part, I hope you don't mind. The answer is: not much, try something else (e.g. another sufficiency condition or, if you think it is convex show that it is convex). – The Almighty Bob Jun 10 '15 at 12:30

There is not a single answer, it will depend on the particulars of each problem. Let's look at a standard example.

Consider the benchmark intertemporal optimization problem for the Ramsey model

\begin{align} &\max_u \int^{\infty}_0{e^{-\rho t}u(c)dt}\\ \\ & \text{s.t.}\;\; \dot{k} = i-\delta k\\ & \text{s.t.}\;\; y = f(k)=c+i \end{align}

The current value Hamiltonian is

$$\tilde H = u(c) +\lambda [f(k)-c-\delta k]$$

Maximizing over $c$ alone we have

$$\frac {\partial \tilde H}{\partial c} = u'(c) - \lambda =0 \implies u'(c^*) = \lambda \implies c^* = (u')^{-1}(\lambda)$$

and the 2nd-order condition will hold if the utility function is concave, $$\frac {\partial^2 H}{\partial c^2} = u''(c^*) < 0$$

Moreover, from the first-order condition with respect to consumption, $\lambda >0$ if local non-satiation holds. Assume that we do have such "usual" preferences.

The maximized over consumption Hamiltonian is

$$\tilde H^0 = u[(u')^{-1}(\lambda)]+\lambda [f(k)-(u')^{-1}(\lambda)-\delta k]$$

The partial derivatives with respect to the state variable, $k$ are

$$\frac {\partial \tilde H^0}{\partial k} = \lambda[f'(k) - \delta], \;\;\;\; \frac {\partial^2 \tilde H^0}{\partial k^2} = \lambda f''(k)$$

So here, the Arrow-Kurz sufficiency condition boils down to whether the marginal product of capital is decreasing, constant, or increasing (which will depend on the sign of the second derivative of the production function). In the standard case $f''(k) < 0$ and we have the sufficient condition.

In the most famous case of deviation, Romer's $AK$ model that initiated the Endogenous Growth literature, $f''(k) =0$, and the marginal product of capital is a positive constant.

So what can we say in this case?

Here, Seierstad, A., & Sydsaeter, K. (1977). Sufficient conditions in optimal control theory. International Economic Review, 367-391. provide various results that can help us.

In particular, they prove that if the Hamiltonian is jointly concave in $c$ and $k$, it is a sufficient condition for a maximum. The Hessian of the Hamiltonian is

(we can ignore the discount term)

$${\rm He}_H = \left [ \begin{matrix} u''(c) & 0\\ 0 & \lambda f''(k)\\ \end{matrix} \right]$$

In the standard case with $u''(c) <0, \; f''(k) <0$ this is a negative definite matrix and so the Hamiltonian is jointly strictly concave in $c$ and $k$.

When $f''(k) =0$, checking that the matrix is negative-semidefinite is straightforward using the definition. Consider a vector $\mathbf z = (z_1, z_2)^T \in \mathbb R^2$ and the product

$$\mathbf z^T{\rm He}_H\mathbf z = z_1^2u''(c) \leq 0$$

this weak inequality holds $\forall \mathbf z \in \mathbb R^2$, and so the Hessian is jointly concave in $c$ and $k$.

So in the $AK$ model of endogenous growth, the solution is indeed a maximum (subject to the parameter constraints needed for the problem to be well-defined of course).

• Thanks. However, I think I should clarify my motives. I know that the Hamiltonian is neither strict concave in $x$, nor jointly concacve in $(x,u)$. Here $x$ drives the shape of the Hamiltonian since $u$ is bounded. It is a strict convex function for small $x$ and any $u$ and a strict concave function for large $x$ and any $u$. I was wondering if we can make a genereal statement about optimality in such a case. – clueless Jun 11 '15 at 5:18
• @clueless This is a different (and interesting) question, so it would be better to ask it in a separate post. – Alecos Papadopoulos Jun 11 '15 at 13:36