Autonomous or non-autonomous optimal control system?

Question

I have a following system with endogeneous discounting. $c,k$ and $h(k)$ are consumption, capital and endogeneous discount function based on physical capital. (the properties of this function are not relevant with the question, so I don't write them in order to take less space in this post. The production function is increasing and concave as usual.)

$$\underset{\left\{ c_{t}\right\} }{max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right)\right]e^{-\triangle}dt$$

My state variables are

$$\begin{align} \dot{k_{t}}=f\left(k_{t}\right)-c_{t}\\ \dot{\triangle}_{t}=\rho + h\left(k_{t}\right) \end{align}$$

I note $\triangle=\int_{0}^{t} (\rho + h\left(k_{t}\right))dt$

I write the present-value Hamiltonian as

$$\mathcal{H}^{present}=u\left(c\right)e^{-\triangle}+\tilde{\lambda_{1}}\left[f\left(k\right)-c\right]+\tilde{\lambda_{2}}\left[\rho + h\left(k\right)\right]$$

In this model, my doubt is that if the system is autonomous or not because when I integrate $\dot{\triangle}$, I have $\triangle= \rho t+ \int_{0}^{t}h\left(k_{t}\right)dt$, which depends explicitly on time $t$. So, evidently, this differential equation is non-autonomous.

Is this system really non-autonomous ?

What I try to do, in order to have an autonomous system is to define variables. Let's say $\lambda_{1}=\tilde{\lambda_{1}}e^{\triangle}$ and $\lambda_{2}=\tilde{\lambda_{2}}e^{\triangle}$. In this way, I write the current-value Hamiltonian ;

$$\mathcal{H}^{current}=u\left(c\right) +\lambda_{1}\left[f\left(k\right)-c\right]+\lambda_{2}\left[\rho + h\left(k\right)\right]$$

I am really stuck at this point. For the moment, the system seems to be autonomous as the Hamiltonian does not depend explicitly on time here but I can not really be sure.

Hints or suggestions are all welcome.

Answer 1

The OP's answer is correct in its conclusion, but he applies a strange argument at the end to arrive there.

Applying brute-force differentiation, the present value Hamiltonian is

$$\mathcal{H}=e^{-\triangle} U\left( c\right) +\lambda _{1}^{}\left[ f(k)-c\right] +\lambda _{2}\left[ \rho +h(k)\right] $$

and so

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ e^{-\triangle} U'(c)\dot c + \dot \lambda_1\left[ f(k)-c\right] + \lambda_1\left[ f'(k)\dot k-\dot c\right] + \dot \lambda _{2}\left[ \rho +h(k)\right]+\lambda _{2}h'(k) \dot k$$

Re-arranging, and using $\rho +h(k) = \dot \triangle$ we get

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ [e^{-\triangle}U'(c)-\lambda_1]\dot c + [\lambda_1f'(k)+\lambda _{2}h'(k)]\dot k + \dot \lambda _{2}\dot \triangle + \dot \lambda_1\left[ f(k)-c\right] $$

Along the optimal path, we have $e^{-\triangle}U'(c)=\lambda_1$ so the second term above vanishes. Also optimally we have $\dot{\lambda}_{1}=- \frac{\partial \mathcal{H}}{\partial k}$ and observe that $\frac{\partial \mathcal{H}}{\partial k} = [\lambda_1f'(k)+\lambda _{2}h'(k)]$. Substituting we get

$$\frac {d\mathcal{H}}{dt} = -\dot \triangle e^{-\triangle}U\left( c\right)+ \frac{\partial \mathcal{H}}{\partial k}\dot k + \dot \lambda _{2}\dot \triangle - \frac{\partial \mathcal{H}}{\partial k}\left[ f(k)-c\right] $$

$$=\frac{\partial \mathcal{H}}{\partial k}[\dot k - f(k)+c] + \dot \triangle \cdot\big[\dot \lambda _{2} - e^{-\triangle}U\left( c\right)\big] $$

The first term is zero, from the law of motion of capital. Moreover, another necessary condition for the optimal path, given how the problem has been formulated, is $\dot{\lambda}_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$. so we arrive at

$$\frac {d\mathcal{H}}{dt} =-\dot \triangle \cdot\left[ \frac{\partial \mathcal{H}}{\partial \triangle} + e^{-\triangle}U\left( c\right)\right]$$

Now $$\frac{\partial \mathcal{H}}{\partial \triangle} = -e^{-\triangle} U\left( c\right)$$ so we obtain

$$\frac {d\mathcal{H}}{dt} =0$$

This directly proves that the problem is autonomous.

More generally, irrespective of whether the problem is autonomous or not, we have that along the optimal path

$$\frac {d\mathcal{H}}{dt} =\frac {\partial \mathcal{H}}{\partial t}$$

So if it is autonomous, then we get the zero-derivative result.

Answer 2

Following Caputo, an Optimal Control problem is autonomous when none of the functions appearing in the description of the problem depends explicitly on the time-variable. But this means that the standard setup with an exogenous utility-discount factor is not autonomous, but it can be made so by redefining the multiplier(s) and then using the current-value Hamiltonian.

@ramazan approach starts correctly, by breaking the discount factor and including the naughty part in the instantaneous objective function. Nothing forbids the state variable to appear in the instantaneous objective function, on the contrary, the general theoretical treatment always includes it. Denoting $y_t = \int_{0}^{t}h\left(k_{s}\right)ds$, your problem now becomes

$$\underset{\left\{ c_{t}\right\} }{\max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right)e^{-y_t}\right]e^{-\rho t}dt$$

$$\dot{k_{t}}=f\left(k_{t}\right)-c_{t}$$

with current value Hamiltonian

$$\mathcal{H}^{current}=u\left(c_t\right)e^{-y_t} +\lambda_t\left[f\left(k_t\right)-c_t\right]$$

While taking the derivative with respect to consumption presents no issues, the derivative of the Hamiltonian with respect to capital needs some care (and this is why I kept the $t$-index in the Hamiltonian). If $G$ is the antiderivative of $h$, then $y_t = G(k_t) - G(k_0)$. I guess the rest are easy for you, although of course the first-order conditions will be more complicated.

Answer 3

What about rewriting the problem in the following way?

$$\underset{\left\{ c_{t}\right\} }{max}\int_{t=0}^{\infty}\left[u\left(c_{t}\right) e^{y_t}\right]e^{-\rho t}dt$$

with the new state variables defined as

$$\begin{align} \dot{k_{t}}=f\left(k_{t}\right)-c_{t}\\ \dot{y}_{t}=-h\left(k_{t}\right) \end{align}$$

given the initial conditions $(k(0),y(0))=(k_0,0)$.

Answer 4

I think I proved in a rigorous way that the system is autonomous for the model that I have written 3-4 days and I think it is useful for the community, especially for those who are working on macroeconomics of growth.

Let's write the present-value Hamiltonian ;

$$\mathcal{H}=e^{-\triangle}\left[ U\left( c\right)\right] +\lambda _{2}^{}\left[ f(k)-c\right] +\lambda _{2}\left[ \rho +h(k)\right] $$

We know that ;

$$\dot{\lambda}_{1}=- \frac{\partial \mathcal{H}}{\partial k}$$

$$\dot{\lambda}_{2}=- \frac{\partial \mathcal{H}}{\partial \triangle}$$

When I differenciate Hamiltonian according to time $t$ ;

$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t} + \mathcal{H}_k \dot{k} + \mathcal{H}_{\lambda_1} \dot{\lambda}_{1} + \mathcal{H}_\triangle \dot{\triangle} + \mathcal{H}_{\lambda_2} \dot{\lambda}_{2} $$

It is straightforward to see that this term reduces to the following one ;

$$\frac{d \mathcal{H}}{dt}=\frac{\partial \mathcal{H}}{\partial t}$$

From page 299, theorem 9.6.1 of Léonard and Dockner (Optimal Control Theory and Static Optimization in Economics) about the transversality condition which shows that

$$\underset{t\rightarrow\infty}{lim}\mathcal{H}\left(t\right)=0$$

when $\rho > 0$

In this case, it is obvious that along the optimal path, the Hamiltonian takes the value 0, which ensures a autonomous dynamical system.