Simple Derivation of Maximum Principle

Question

Consider the simplest problem of optimal control \begin{align} &\max_u\int^T_0{F(y,u)dt}\\ \text{s.t.} \quad&\dot y = f(y,u)\\ & y(0) = y_0\\ & y(T)~~\text{free} \end{align} where $y$ is the state and $u$ the control. I'd like to use a static-like approach to derive the necessary conditions of the maximum principle. Build Lagrangian: \begin{align} L^1(y,u,\lambda) = \int^T_0{[F(y,u) + \lambda(f(y,u)-\dot y)]dt} \end{align}

Necessary conditionds for optimum are given by \begin{align} L^1_u &= \int^T_0{[F_u + \lambda f_u] dt} = 0\\ L^1_\lambda &= \int^T_0{[f(y,u)-\dot y ]dt} = 0 \end{align}

Define Hamiltonian \begin{align} H(y,u,\lambda) := F(y,u) + \lambda f(y,u) \end{align}

such that the FOCs can be written as \begin{align} H_u &= 0~\forall t\\ H_\lambda &= \dot y~\forall t \end{align}

For last FOC we write Lagrangian as (integrating by parts) \begin{align} L^2(y,u,\lambda) = \int^T_0{[H(y,u,\lambda) + \dot\lambda y]dt} + y_0\lambda(0) - y(t)|_{t=T}\lambda(T) \end{align}

Note that $L^1=L^2$. Last FOC is given by differentiating over $y$ ($y_0$ drops, cause it's fixed) \begin{align} L^2_y &= \int^T_0{[H_y + \dot \lambda] dt} - \lambda(T) = 0\\ \end{align}

and finally \begin{align} H_y &= -\dot \lambda ~\forall t\\ \lambda(T) &= 0 \end{align}

So we are done here.

• How can I get those conditions without varying the Lagrangian, i.e. stick to either $L^1$ or $L^2$?

Answer 1

After a comment exchange, let's provide the answer under disrete-time formulation. The problem is now written

\begin{align} &\max_{\{u\}_1^T, \{y\}_1^T}\sum^T_{t=1}{F(y_t,u_t)}\\ \text{s.t.} \quad & y_{t+1}-y_t = f(y_t,u_t)\\ & y_1 = \text{given}\\ & y_{T+1}~~\text{free} \end{align}

where $y_{t+1}$ is the value of the stock variable at the begining of period $t+1$/end of period $t$. The Lagrangean is \begin{align} \Lambda = \sum^T_{t=1}{\big(F(y_t,u_t) + \lambda_t[f(y_t,u_t)-y_{t+1}+y_t]\big)} \end{align}

Note that $\lambda_t$ is associated with $y_{t+1}$.

The FOCS are (to hold $\forall t$ in the horizon)

$$\frac {\partial \Lambda}{\partial u_t} = F_u(y_t,u_t) + \lambda_tf_u(y_t,u_t) = 0 \tag{1}$$

$$\frac {\partial \Lambda}{\partial y_{t+1}} = -\lambda_{t} + F_y(y_{t+1},u_{t+1}) + \lambda_{t+1}[f_u(y_{t+1},u_{t+1}) +1] = 0 $$

$$\implies F_y(y_{t+1},u_{t+1}) + \lambda_{t+1}f_y(y_{t+1},u_{t+1}) = -(\lambda_{t+1}-\lambda_{t}) \tag{2}$$

which are the conditions we would have obtain if we have used the Hamiltonian formulation. To obtain the terminal value for $\lambda_t$, we set $t=T$ in $(2)$, (since the last variable in the $y$-sequence is $y_{T+1}$), and we obtain:

$$F_y(y_{T+1},0) + \lambda_{T+1}f_y(y_{T+1},0) = -(\lambda_{T+1}-\lambda_{T})$$

$u_{T+1}=0$ because there is no decision to be made in period $T+1$. Moreover $\lambda_{T+1}$ should also be set to zero, since it is associated with $y_{T+2}$ (end of period $T+1$) and we are out of the horizon. So we are left with

$$F_y(y_{T+1},0) = \lambda_{T}$$

Whether now we will get $\lambda_T=0$ or not depends on the formulation of the objective function $F(y_{t},u_t)$: does it indeed include the state variable? If yes what is the value of its derivative with respect to the state variable when the decision variable is set to zero? Etc