# Autonomous or non-autonomous optimal control system?

I have a following system with endogeneous discounting. $c,k$ and $h(k)$ are consumption, capital and endogeneous discount function based on physical capital. (the properties of this function are not relevant with the question, so I don't write them in order to take less space in this post. The production function is increasing and concave as usual.)

$$\underset{\left\{ c_{t}\right\} }{max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right)\right]e^{-\triangle}dt$$

My state variables are

\begin{align} \dot{k_{t}}=f\left(k_{t}\right)-c_{t}\\ \dot{\triangle}_{t}=\rho + h\left(k_{t}\right) \end{align}

I note $\triangle=\int_{0}^{t} (\rho + h\left(k_{t}\right))dt$

I write the present-value Hamiltonian as

$$\mathcal{H}^{present}=u\left(c\right)e^{-\triangle}+\tilde{\lambda_{1}}\left[f\left(k\right)-c\right]+\tilde{\lambda_{2}}\left[\rho + h\left(k\right)\right]$$

In this model, my doubt is that if the system is autonomous or not because when I integrate $\dot{\triangle}$, I have $\triangle= \rho t+ \int_{0}^{t}h\left(k_{t}\right)dt$, which depends explicitly on time $t$. So, evidently, this differential equation is non-autonomous.

Is this system really non-autonomous ?

What I try to do, in order to have an autonomous system is to define variables. Let's say $\lambda_{1}=\tilde{\lambda_{1}}e^{\triangle}$ and $\lambda_{2}=\tilde{\lambda_{2}}e^{\triangle}$. In this way, I write the current-value Hamiltonian ;

$$\mathcal{H}^{current}=u\left(c\right) +\lambda_{1}\left[f\left(k\right)-c\right]+\lambda_{2}\left[\rho + h\left(k\right)\right]$$

I am really stuck at this point. For the moment, the system seems to be autonomous as the Hamiltonian does not depend explicitly on time here but I can not really be sure.

Hints or suggestions are all welcome.

• On the side, It would be better to change the dummy variable of integration from $t$ to something like $s$, to avoid confusion, since $t$ is your actual running index. – Alecos Papadopoulos Sep 29 '15 at 1:25
• "Evidently, this differential equation is non-autonomous". That's a wrong statement. The differential equation itself is autonomous (except if $h()$ contains $t$ explicitly), the fact that the time-variable appears after integrating is a different issue. – Alecos Papadopoulos Sep 29 '15 at 2:16

The OP's answer is correct in its conclusion, but he applies a strange argument at the end to arrive there.

Applying brute-force differentiation, the present value Hamiltonian is

$$\mathcal{H}=e^{-\triangle} U\left( c\right) +\lambda _{1}^{}\left[ f(k)-c\right] +\lambda _{2}\left[ \rho +h(k)\right]$$

and so

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ e^{-\triangle} U'(c)\dot c + \dot \lambda_1\left[ f(k)-c\right] + \lambda_1\left[ f'(k)\dot k-\dot c\right] + \dot \lambda _{2}\left[ \rho +h(k)\right]+\lambda _{2}h'(k) \dot k$$

Re-arranging, and using $\rho +h(k) = \dot \triangle$ we get

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ [e^{-\triangle}U'(c)-\lambda_1]\dot c + [\lambda_1f'(k)+\lambda _{2}h'(k)]\dot k + \dot \lambda _{2}\dot \triangle + \dot \lambda_1\left[ f(k)-c\right]$$

Along the optimal path, we have $e^{-\triangle}U'(c)=\lambda_1$ so the second term above vanishes. Also optimally we have $\dot{\lambda}_{1}=- \frac{\partial \mathcal{H}}{\partial k}$ and observe that $\frac{\partial \mathcal{H}}{\partial k} = [\lambda_1f'(k)+\lambda _{2}h'(k)]$. Substituting we get

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ \frac{\partial \mathcal{H}}{\partial k}\dot k + \dot \lambda _{2}\dot \triangle - \frac{\partial \mathcal{H}}{\partial k}\left[ f(k)-c\right]$$

$$=\frac{\partial \mathcal{H}}{\partial k}[\dot k - f(k)+c] + \dot \triangle \cdot\big[\dot \lambda _{2} - e^{-\triangle}U\left( c\right)\big]$$

The first term is zero, from the law of motion of capital. Moreover, another necessary condition for the optimal path, given how the problem has been formulated, is $\dot{\lambda}_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$. so we arrive at

$$\frac {d\mathcal{H}}{dt} =-\dot \triangle \cdot\left[ \frac{\partial \mathcal{H}}{\partial \triangle} + e^{-\triangle}U\left( c\right)\right]$$

Now $$\frac{\partial \mathcal{H}}{\partial \triangle} = -e^{-\triangle} U\left( c\right)$$ so we obtain

$$\frac {d\mathcal{H}}{dt} =0$$

This directly proves that the problem is autonomous.

More generally, irrespective of whether the problem is autonomous or not, we have that along the optimal path

$$\frac {d\mathcal{H}}{dt} =\frac {\partial \mathcal{H}}{\partial t}$$

So if it is autonomous, then we get the zero-derivative result.

• Thanks for your detailed response. What I have used was the direct transversality condition, as clueless mentionned in the comment. But your response is more clear, thanks again. – optimal control Oct 4 '15 at 11:22

Following Caputo, an Optimal Control problem is autonomous when none of the functions appearing in the description of the problem depends explicitly on the time-variable. But this means that the standard setup with an exogenous utility-discount factor is not autonomous, but it can be made so by redefining the multiplier(s) and then using the current-value Hamiltonian.

@ramazan approach starts correctly, by breaking the discount factor and including the naughty part in the instantaneous objective function. Nothing forbids the state variable to appear in the instantaneous objective function, on the contrary, the general theoretical treatment always includes it. Denoting $y_t = \int_{0}^{t}h\left(k_{s}\right)ds$, your problem now becomes

$$\underset{\left\{ c_{t}\right\} }{\max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right)e^{-y_t}\right]e^{-\rho t}dt$$

$$\dot{k_{t}}=f\left(k_{t}\right)-c_{t}$$

with current value Hamiltonian

$$\mathcal{H}^{current}=u\left(c_t\right)e^{-y_t} +\lambda_t\left[f\left(k_t\right)-c_t\right]$$

While taking the derivative with respect to consumption presents no issues, the derivative of the Hamiltonian with respect to capital needs some care (and this is why I kept the $t$-index in the Hamiltonian). If $G$ is the antiderivative of $h$, then $y_t = G(k_t) - G(k_0)$. I guess the rest are easy for you, although of course the first-order conditions will be more complicated.

• thanks for your time. So, when I look the hamiltonian you write, the system seems to be autonomous. So, you say that this is an autonomous problem ? Also, I did not understand why you did not put the second state variable $\dot{y}$ as ramazan did in his calculation ? Because, in a standard optimal control problem, integral could not be present on a hamiltonian, – optimal control Sep 29 '15 at 11:59
• Autonomous with the qualification I mentioned in the answer. I don't see any "second state variable", since $y_t$ is just a function of the capital state variable. Have you calculated the usual first-order conditions to see what they give you? – Alecos Papadopoulos Sep 29 '15 at 12:18

What about rewriting the problem in the following way?

$$\underset{\left\{ c_{t}\right\} }{max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right) e^{y_t}\right]e^{-\rho t}dt$$

with the new state variables defined as

\begin{align} \dot{k_{t}}=f\left(k_{t}\right)-c_{t}\\ \dot{y}_{t}=-h\left(k_{t}\right) \end{align}

given the initial conditions $(k(0),y(0))=(k_0,0)$.

• Thanks ramazan. So, I think in this way the problem reduces to a autonomous problem. Am I right ? – optimal control Sep 29 '15 at 12:00
• I am not knowledgeable from this point onwards, I have checked for the definition of an autonomous optimal control problem, but seems that I could not find a clear one. – ramazan Sep 29 '15 at 20:45

I think I proved in a rigorous way that the system is autonomous for the model that I have written 3-4 days and I think it is useful for the community, especially for those who are working on macroeconomics of growth.

Let's write the present-value Hamiltonian ;

$$\mathcal{H}=e^{-\triangle}\left[ U\left( c\right)\right] +\lambda _{2}^{}\left[ f(k)-c\right] +\lambda _{2}\left[ \rho +h(k)\right]$$

We know that ;

$$\dot{\lambda}_{1}=- \frac{\partial \mathcal{H}}{\partial k}$$

$$\dot{\lambda}_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$$

When I differenciate Hamiltonian according to time $t$ ;

$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t} + \mathcal{H}_k \dot{k} + \mathcal{H}_{\lambda_1} \dot{\lambda}_{1} + \mathcal{H}_\triangle \dot{\triangle} + \mathcal{H}_{\lambda_2} \dot{\lambda}_{2}$$

It is straightforward to see that this term reduces to the following one ;

$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t}$$

From page 299, theorem 9.6.1 of Léonard and Dockner (Optimal Control Theory and Static Optimization in Economics) about the transversality condition which shows that

$$\underset{t\rightarrow\infty}{lim}\mathcal{H}\left(t\right)=0$$

when $\rho > 0$

In this case, it is obvious that along the optimal path, the Hamiltonian takes the value 0, which ensures a autonomous dynamical system.

• I don't follow the last argument you use. The equality of partial and total derivatives of the Hamiltonian with respect to time is a general result, irrespective of whether the problem is autonomous or not. I have posted a second answer proving directly the autonomous nature of the problem. – Alecos Papadopoulos Oct 3 '15 at 19:53
• If the system is in steady state at $t=T$ we must have $H(T) = 0$ and $H(T+\Delta t) = 0$ by the transversality condition. This finding implies $dH/dt=0$ for $t \geq T$. But this is only true in steady state and not necessarily on the saddle path for $t<T$. – clueless Oct 3 '15 at 22:21