Consider the simplest problem of optimal control \begin{align} &\max_u\int^T_0{F(y,u)dt}\\ \text{s.t.} \quad&\dot y = f(y,u)\\ & y(0) = y_0\\ & y(T)~~\text{free} \end{align} where $y$ is the state and $u$ the control. I'd like to use a static-like approach to derive the necessary conditions of the maximum principle. Build Lagrangian: \begin{align} L^1(y,u,\lambda) = \int^T_0{[F(y,u) + \lambda(f(y,u)-\dot y)]dt} \end{align}

Necessary conditionds for optimum are given by \begin{align} L^1_u &= \int^T_0{[F_u + \lambda f_u] dt} = 0\\ L^1_\lambda &= \int^T_0{[f(y,u)-\dot y ]dt} = 0 \end{align}

Define Hamiltonian \begin{align} H(y,u,\lambda) := F(y,u) + \lambda f(y,u) \end{align}

such that the FOCs can be written as \begin{align} H_u &= 0~\forall t\\ H_\lambda &= \dot y~\forall t \end{align}

For last FOC we write Lagrangian as (integrating by parts) \begin{align} L^2(y,u,\lambda) = \int^T_0{[H(y,u,\lambda) + \dot\lambda y]dt} + y_0\lambda(0) - y(t)|_{t=T}\lambda(T) \end{align}

Note that $L^1=L^2$. Last FOC is given by differentiating over $y$ ($y_0$ drops, cause it's fixed) \begin{align} L^2_y &= \int^T_0{[H_y + \dot \lambda] dt} - \lambda(T) = 0\\ \end{align}

and finally \begin{align} H_y &= -\dot \lambda ~\forall t\\ \lambda(T) &= 0 \end{align}

So we are done here.

  • How can I get those conditions without varying the Lagrangian, i.e. stick to either $L^1$ or $L^2$?
  • $\begingroup$ For all three focs and the terminal condition, it is pretty straightforward in discrete time -do you have a special reason to do it in continuous time? $\endgroup$ Oct 15 '15 at 18:01
  • $\begingroup$ There is no special reason. It's out of curiosity and since the orginial maximum principle is formulated in continuous time. I was just wondering why to use a variational approach, if the conditions can be presented that neatly. Actually, if you come from utility maximization of a consumer, the presented way seems to be more 'natural' than, say, this approach. $\endgroup$
    – clueless
    Oct 17 '15 at 8:35

After a comment exchange, let's provide the answer under disrete-time formulation. The problem is now written

\begin{align} &\max_{\{u\}_1^T, \{y\}_1^T}\sum^T_{t=1}{F(y_t,u_t)}\\ \text{s.t.} \quad & y_{t+1}-y_t = f(y_t,u_t)\\ & y_1 = \text{given}\\ & y_{T+1}~~\text{free} \end{align}

where $y_{t+1}$ is the value of the stock variable at the begining of period $t+1$/end of period $t$. The Lagrangean is \begin{align} \Lambda = \sum^T_{t=1}{\big(F(y_t,u_t) + \lambda_t[f(y_t,u_t)-y_{t+1}+y_t]\big)} \end{align}

Note that $\lambda_t$ is associated with $y_{t+1}$.

The FOCS are (to hold $\forall t$ in the horizon)

$$\frac {\partial \Lambda}{\partial u_t} = F_u(y_t,u_t) + \lambda_tf_u(y_t,u_t) = 0 \tag{1}$$

$$\frac {\partial \Lambda}{\partial y_{t+1}} = -\lambda_{t} + F_y(y_{t+1},u_{t+1}) + \lambda_{t+1}[f_u(y_{t+1},u_{t+1}) +1] = 0 $$

$$\implies F_y(y_{t+1},u_{t+1}) + \lambda_{t+1}f_y(y_{t+1},u_{t+1}) = -(\lambda_{t+1}-\lambda_{t}) \tag{2}$$

which are the conditions we would have obtain if we have used the Hamiltonian formulation. To obtain the terminal value for $\lambda_t$, we set $t=T$ in $(2)$, (since the last variable in the $y$-sequence is $y_{T+1}$), and we obtain:

$$F_y(y_{T+1},0) + \lambda_{T+1}f_y(y_{T+1},0) = -(\lambda_{T+1}-\lambda_{T})$$

$u_{T+1}=0$ because there is no decision to be made in period $T+1$. Moreover $\lambda_{T+1}$ should also be set to zero, since it is associated with $y_{T+2}$ (end of period $T+1$) and we are out of the horizon. So we are left with

$$F_y(y_{T+1},0) = \lambda_{T}$$

Whether now we will get $\lambda_T=0$ or not depends on the formulation of the objective function $F(y_{t},u_t)$: does it indeed include the state variable? If yes what is the value of its derivative with respect to the state variable when the decision variable is set to zero? Etc

  • $\begingroup$ Thanks for the effort, however, that post does not present an answer to the question at hand. Still, how come you changed to discrete time? $\endgroup$
    – clueless
    Oct 18 '15 at 0:50
  • $\begingroup$ @clueless Because as I wrote in a previous comment, obtaining the "Hamiltonian" FOCS using a static Lagrange formulation is straightforward in discrete time, and it will be useful content here. $\endgroup$ Oct 18 '15 at 1:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.