# Dynamic optimisation

Consider a simple dynamic consumption-saving problem. A solution can be characterised using a Lagrangian approach that generates a set of first order conditions and some boundary conditions.

An alternative approach is to set use a Bellman equation. What I don't understand is how come a Bellman equation approach does not seem to use information regarding the boundary conditions in the solution methodology? That is, a Bellman equation does have a tradeoff between time period t and t+1 but not some overall budget constraint does not seem to be represented.

• Are you using a transversality condition? Something that tells you what happens as $\lim\limits_{t \to \infty}$? – Giskard Nov 18 '16 at 7:37
• Same question as above. In an infinite period model, transversality conditions are desireable if you want non trivial solutions. If you (the asker) assumes there are those conditions, then I'm not sure what the issue is. – Kitsune Cavalry Nov 18 '16 at 8:07

The Dynamic Programming ("Bellman' Equation") formulation incorporates the terminal boundary condition ("transversality conditions") needed in case we use the Lagrangian/Euler equation formulation. The initial conditions are still needed in both approaches.

To see why, consider the problem

$$\text{max}_{\{x_t\}} \sum_{t=0}^{\infty} \beta^t U(c_t)$$

$$s.t.\;\;\; a_{t+1}= (1-r)a_t + w - c_t, \;\;\; a_0 \; \text{given}$$

Solving the budget constraint for $c_t$ and inserting into the objective function, we have compactly

$$\text{max}_{\{a_{t}\}} \sum_{t=0}^{\infty} \beta^t U(a_t, a_{t+1}),\;\;\;\; a_0 \; \text{given}$$

where we treat the stock variable next period as our decision variable. Now for the optimal sequence, $a^*_{t+1}$ must solve the following problem:

$$\max_y \big[U(a^*_t, y) + \beta U(y, a^*_{t+2})\big],\;\;\; y\;\; \text{feasible given}\;\; a^*_t$$

In the words of Stokey and Lucas "a feasible variation on the sequence $\{a^*_t\}$ cannot lead to an improvement on an optimal policy".

Then we get the first order conditions (differentiating with respect to $y$ essentially)

$$U_2(a^*_t, a^*_{t+1}) + \beta U_1(a^*_{t+1}, a^*_{t+2}) =0$$

where the subscripts $1,2$ denote partial derivatives. This is a second-order difference equation, and so it requires two boundary conditions. We have already one of them, the given initial value of wealth $a_0$. We need one more, hence the terminal, "transversality" condition.

In contrast, using the Dynamic Programming approach we arrive at a first-order difference equation, and we do not need an additional boundary condition.

For the full exposition, see Stokey & Lucas p. 97-100.

I'm unsure exactly where your struggle is, in general. To try and address your issues, what kind of boundary conditions might we desire in dynamic problems?

Consider the two period consumption-savings problem where we have

$$c_1 + s_1 = y_1$$ $$c_2 = y_2 + (1+R)s_1$$

$y_1, y_2$ being endowments in each period, $s_1$ as savings, $R$ for interest, and $c_1, c_2$ for consumption. Given some $U(c_1, c_2)$, we want to solve for optimal consumption in each period. We can consolidate the budget as:

$$c_2 = y_2 + (1 + R)(y_1 - c_1)$$ $$\implies c_1 + \frac{c_2}{1 + R} = y_1 + \frac{y_2}{1 + R}$$

and express the Lagrangian problem:

$$\mathcal{L} = U(c_1, c_2) - \lambda(c_1 - \frac{c_2}{1 + R} - y_1 - \frac{y_2}{1 + R})$$

Solving for FOCs gets us:

$$U_{c_1} - \lambda = 0$$ $$U_{c_2} - \frac{\lambda}{1 + R} = 0$$ $$\implies \frac{U_{c_1}}{U_{c_2}} = 1 + R$$

But you can also express this problem as a Bellman, and still keep that original consolidated budget constraint.

$$\max_{c_1, c_2, y_1, y_2} U(c_1, c_2) \quad \text{s.t.} \ c_1 + \frac{c_2}{1 + R} = y_1 + \frac{y_2}{1 + R}$$

where you have a value function for $y_1$ (yes, $y_1$ is still given)

$$V(y_1) = \max_{c_1, c_2, y_2} \ U(c_1, c_2) \quad \text{s.t.} \ y_1 = c_1 + \frac{c_2}{1 + R} - \frac{y_2}{1 + R}$$

To make this problem less trivial for a Bellman, you can have a constraint where $c$ determines $y$ somehow instead. But otherwise you can still incorporate the budget constraint into your problem here, albeit it might look pretty weird.