The question is related to this thread. I'd like to derive a unique steady state for an optimal control problem.

Consider the following programm \begin{align} &V(x_0) := \max_u \int^\infty_0 e^{-\rho t}F(x(t),u(t))dt\\ s.t.~&\dot x(t)=f(x(t),u(t))\\ &x(0) = x_0 \end{align} where $\rho > 0$ denotes time preference, $V(\cdot)$ is the value and $F(\cdot)$ an objective function. $x\in X$ is the state variable and $u\in U=[0,1]$ the control. The state is governed by $f(\cdot)$. The Hamilton-Jacobi-Bellman equation reads \begin{align} \rho V(x)=\max_u [F(x,u) + V'(x)f(x,u)],\quad \forall t\in[0,\infty) \end{align}

Now presume that the feedback control is given by \begin{align} u(x) = \frac{1}{1+V'(x)} = \arg\max_u [F(x,u) + V'(x)f(x,u)],\quad \forall x\in X. \end{align}

Suppose there is a fixed point at $x = \tilde x$ and we can derive an alternative representation for the optimal control in the fixed point with \begin{align} u(\tilde x) = \frac{\rho + u'(\tilde x)}{\rho + u'(\tilde x) + 1}. \end{align}

Suppose additionaly the HJB in the fixed point is given by \begin{align} \rho V(x)=\ln\left(\frac{1}{1+V'(x)}\right) + 1 - \frac{1}{1+V'(x)}. \end{align}

If $\rho\to 0$ approches zero we must have $V'(\tilde x) = 0 \Rightarrow u(\tilde x) = 1 \Rightarrow u'(\tilde x) = \infty$. On the other hand if $\rho\to\infty$ approches infinity we must have $V(\tilde x) = 0$ by the definition of the value function and thus again $V'(\tilde x) = 0 \Rightarrow u(\tilde x) = 1$. Summarizing we have the following properties in equilibrium \begin{align} \lim_{\rho\to 0} u(\tilde x)=\lim_{\rho\to\infty} u(\tilde x) = 1. \end{align}

Well, that's at odds with \begin{align} \frac{\partial u(\tilde x)}{\partial \rho}=\frac{1}{(\rho + u'(\tilde x) + 1)^2} > 0 \end{align}

being a strictly monoton increasing function, contradicting our previous result. Note, however, that we may resolve the issue by observing \begin{align} \lim_{u'(\tilde x)\to\infty}\frac{\partial u(\tilde x)}{\partial \rho}= 0 \end{align}

So, can we finally conjecture that we must have in the fixed point $u'(\tilde x) = \infty \Rightarrow u(\tilde x) = 1$ such that \begin{align} \rho V(\tilde x)=\ln(1) + 1 - 1 \Leftrightarrow V(\tilde x) = 0. \end{align}

  • $\begingroup$ I am not sure how you arrive at the expression for $\frac{\partial u(\tilde x)}{\partial \rho}$ $\endgroup$ Jul 29, 2015 at 9:34
  • $\begingroup$ You're right. I think it should read: $\partial u(\tilde x)/\partial \rho = (1+\partial u'(\tilde x)/\partial \rho)/(\rho + u'(\tilde x) + 1)^2$ where the sign is indeterminated and thus, my argumentation is flawed. $\endgroup$
    – clueless
    Jul 29, 2015 at 10:34
  • $\begingroup$ How are we going to proceed? Should I post this as a "negative-result" answer so that the post does not remain in the unanswered queue, or will you modify somehow the question? $\endgroup$ Jul 29, 2015 at 14:15
  • $\begingroup$ Probably negative result, cause I have no idea how to resolve the issue at hand. I think, I've actually found a different way to show that $u(\tilde x) = 1$ is a stable equlibrium associated with the highest value, since by the one shot deviation principle there is no incentive to deviate from that particular strategy. However, my orginial setting deals with two agents, being different from that one presented here. $\endgroup$
    – clueless
    Jul 29, 2015 at 22:27

1 Answer 1


Upgrading an exchange of comments, a critical point in the question is the expression \begin{align} \frac{\partial u(\tilde x)}{\partial \rho}=\frac{1}{(\rho + u'(\tilde x) + 1)^2} \end{align}

which is mistaken , because from

\begin{align} u(\tilde x) = \frac{\rho + u'(\tilde x)}{\rho + u'(\tilde x) + 1}. \end{align}

we obtain

\begin{align} \frac{\partial u(\tilde x)}{\partial \rho}=\frac{1+\partial u'(\tilde x)/\partial \rho}{(\rho + u'(\tilde x) + 1)^2} \end{align}

The OP notes that the sign of $\partial u'(\tilde x)/\partial \rho$ is indeterinate, and so $\partial u(\tilde x)/\partial \rho$ cannot be said to be monotonically increasing, which was what drove the pinning-down of the steady state.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.