# Solving Stochastic Dynamic Optimization Problems: A difficulty with Lagrange Multipliers

In Wickens' Macroeconomics book, in page 552, the author states the following:

«The stochastic problem can be solved using the method of Lagrange multipliers, but there is a problem with this solution.»

However, if I'm not mistaken, it's very usual to see other books using lagrange multipliers to solve this type of problems. Are they wrong? Am I wrong? Is there a way to go around this apparent limitation?

Any help would be appreciated.

Here's a the image:

Following a comment exchange below another answer, the critical detail of a link offered by the OP as an example of by-passing/ignoring Wicken's comment/argument, and one different from Wickens' formulation, is that in the link's eq. $(4)$,

$$\lambda_t = \beta E_t[\lambda_{t+1}(1+r_t)]$$

the multiplier $\lambda_{t+1}$ appears together with the interest rate of period $t$, which is assumed part of the information set at $t$ and so not a random variable in the specific equation. So the authors can proceed in the 2nd line after eq. $(6)$ to write the marginal rate of substitution as they do, without the need to assume uncorrelatedness between the multiplier/marginal utility of consumption, and the interest rate.

This goes back to how they formulate the income resource constraint (page 2 middle), where it is essentially assumed that the household has, at the beginning of period $t$, available assets or debt $(1+r_{t-1})B_t$. The authors explicitly discuss this "timing convention" immediately after eq. $(1)$

"Note a timing convention -$r_{t-1}$ is the interest you have to pay today on existing debt. $r_t$ is what you will have to pay tomorrow, but you choose how much debt to take into tomorrow today. Hence, we assume that households observe $r_t$ in time $t$. Hence we can treat $r_t$ as known from the perspective of time $t$."

• Alecos, do you have an opinion on this book by Michael Wickens? Oct 2 '16 at 20:45
• I haven't checked it thoroughly. But my first impression when I browsed its pages (I have the first edition), is that it is good for the basic tools (so graduate beginners level). Sometimes such books are very useful because the keep the core very visible. Oct 2 '16 at 20:56

5 years later I came across this question myself and I think that Alecos's answer should not the accepted one: First of all his answer does not really adress the question, I gave an example below. Secondly, even his answer makes the general problem to simple: In general we really want to stick to the case in which $$r_{t+1}$$ is unknown to the information at time $$t$$, i.e. \begin{align*} \lambda_t = \mathbb{E}_t[\lambda_{t+1} (1+r_{t+1})] \neq \mathbb{E}_t[\lambda_{t+1} ](1+r_{t+1}). \end{align*} But as pointed out in the comments, assuming short term interest rates such that indeed the interest rate $$r_{t+1}$$ is measurable w.r.t. the information at time $$t$$, i.e. \begin{align*} \lambda_t = \mathbb{E}_t[\lambda_{t+1} (1+r_{t+1})] = \mathbb{E}_t[\lambda_{t+1} ](1+r_{t+1}), \end{align*} isn't the real issue:

What the "usual" economist is doing (wrongly), and what you will see in a lot of lectures, and to what Wickens is referring to, is the following:

Consider an agent facing the following optimization problem \begin{align*} \max_{(C_t)_{t \in \mathbb{N}}} \mathbb{E}_0 \left[ \sum_{t=0}^\infty e^{-\delta t} u(C_t, C_{t-1})\right] \end{align*} where $$u(C_t, C_{t-1}) = \frac{1}{1-\gamma} (\alpha C_t + (1- \alpha)C_{t-1})^{1-\gamma}$$ for some constant $$\alpha \in (0,1]$$. The agent's budget constraint is given by \begin{align*} X_{t+1} = (1+r_t)X_t + Y_t - C_t \end{align*} where $$Y_t > 0$$ is a stochastic income process.

Then, we usually (as in the deterministic case) want to consider the Lagrangian, i.e. the Lagrangian is given by \begin{align*} &\mathcal{L}(\{C_t\}_{t=1}^\infty, \{X_{t+1}\}_{t=0}^\infty, \{\lambda_t\}_{t=0}^\infty)\\ &\hspace{4em}= \mathbb{E}_0 \left[\sum_{t=0}^\infty e^{-\delta t} u(C_t, C_{t-1}) - \lambda_t (X_{t+1} - X_t(1+r_t) - Y_t + C_t)\right]. \end{align*} Denote by $$u_1$$ and $$u_2$$ the derivative of $$u$$ w.r.t. the first and second variable, respectively. Then, you will read something like "now taking the FOC at period $$t$$" (i.e. w.r.t $$C_t$$, and $$X_{t+1}$$) and "time shift to $$t+1$$" (w.r.t $$C_{t+1}$$), we obtain the following \begin{align} &\text{w.r.t. } C_t \hspace{3em} e^{-\delta t} u_1 (C_t, C_{t-1}) + e^{- \delta (t+1)}\mathbb{E}_t [u_2 (C_{t+1}, C_t)] = \lambda_t\tag{1}\label{eqA}\\ &\text{w.r.t. } C_{t+1} \qquad\mathbb{E}_{t+1}\left[e^{-\delta(t+1)} u_1 (C_{t+1}, C_t) + e^{-\delta (t+2)} u_2 (C_{t+2}, C_{t+1})\right] = \lambda_{t+1}\tag{2}\label{eqB}\\ &\text{w.r.t. } X_{t+1} \qquad\lambda_t = \mathbb{E}_t[\lambda_{t+1} (1+r_{t+1})]\tag{3}\label{eqC}. \end{align} Then, as in the corresponding deterministic problem, we insert \eqref{eqA} and \eqref{eqB} into \eqref{eqC}, use the tower property $$\mathbb{E}_{t}[\mathbb{E}_{t+1}[\cdot]] = \mathbb{E}_t[\cdot]$$, and obtain the following: $$$$\begin{split} &u_1 (C_t, C_{t-1}) + e^{- \delta}\mathbb{E}_t [u_2 (C_{t+1}, C_t)] \\ &\hspace{4em}= \mathbb{E}_t\left[\left(e^{-\delta} u_1 (C_{t+1}, C_t) + e^{-2\delta } u_2 (C_{t+2}, C_{t+1})\right) (1+r_{t+1})\right]. \end{split} \label{eqWant} \tag{4}$$$$ Since the conditional expectation w.r.t the information at time $$t$$ is itself a measurable random variable w.r.t. the information at time $$t$$, we can reformulate above equation to the Euler equation: $$$$1= \mathbb{E}_t\left[\frac{e^{-\delta} u_1 (C_{t+1}, C_t) + e^{-2\delta } u_2 (C_{t+2}, C_{t+1})}{u_1 (C_t, C_{t-1}) + e^{- \delta}\mathbb{E}_t [u_2 (C_{t+1}, C_t)]} (1+r_{t+1})\right]. \label{eqReform} \tag{5}$$$$

But the actual FOC which we deduce from the Lagrangian are (w.r.t $$C_t$$, $$C_{t+1}$$, and $$X_{t+1}$$ at $$t=0,1,2, \dots$$): \begin{align} &\text{w.r.t. } C_t \hspace{3em} \mathbb{E}_0 \left[ e^{-\delta t} u_1 (C_t, C_{t-1}) + e^{- \delta (t+1)}u_2 (C_{t+1}, C_t)\right] = \mathbb{E}_0[ \lambda_t]\tag{6}\label{eq1}\\ &\text{w.r.t. } C_{t+1} \qquad\mathbb{E}_0\left[e^{-\delta(t+1)} u_1 (C_{t+1}, C_t) + e^{-\delta (t+2)} u_2 (C_{t+2}, C_{t+1})\right] = \mathbb{E}_0 [\lambda_{t+1}]\tag{7}\label{eq2}\\ &\text{w.r.t. } X_{t+1} \qquad \mathbb{E}_0[\lambda_t]= \mathbb{E}_0[\lambda_{t+1} (1+r_{t+1})]\tag{8}\label{eq3}. \end{align}

In principle, we want (as above) insert \eqref{eq1} and \eqref{eq2} into \eqref{eq3}, to obtain \eqref{eqReform}.

But this cannot be deduced in general with the Lagrangian approach, unless we assume that the multiplier $$\lambda_{t+1}$$ and the interest rate $$r_{t+1}$$, as well as interest rate $$r_{t+1}$$ and the marginal utility $$e^{-\delta(t+1)} u_1 (C_{t+1}, C_t) + e^{-\delta (t+2)} u_2 (C_{t+2}, C_{t+1})$$ are conditionally uncorrelated (w.r.t. to the information at time $$t=0$$), or e.g. the interest rate is constant. But still then, we have the problem that our expectation is conditioned on the information set at time $$t=0$$.

Using the Lagrangian approach essentially neglects the fact that we want to have a sequential condition for the optimal consumption flow. As a side remark, the first FOC I gave, would correspond to a different problem: \begin{align*} \max_{(C_t)_{t \in \mathbb{N}}} \sum_{t=0}^\infty\mathbb{E}_t \left[ e^{-\delta t} u(C_t, C_{t-1})\right] \end{align*}

How to solve the issue of the first problem? This issue can be solved by the use of the "Stochastic Dynamic Optimization" technique (see Section 15.6 of Wickens), i.e. going to the Bellman equation. In fact, in the case of the first example I gave, you are able to deduce the Euler equation rigorously. For this, consider equation (15.33) in Wickens.

Be aware that the "wrong" Lagrangian approach (i.e. just "inserting" equations \eqref{eqA} and \eqref{eqB} for example) does not lead to the same solution as in the dynamic programming approach in general.

Example that the analogous stochastic optimisation problem does not lead to a "naive stochastic version" of the deterministic solution: Consider Example 15.2 in Wickens' Macroeconomic book:

Maximize \begin{align*} V = \sum_{t=0}^T \beta^t \ln{c_t} \end{align*} subject to \begin{align*} s_{t+1} - s_t = \alpha (s_t - c_t), \qquad s_{T+1} = 0, \qquad 0<\alpha <1 \end{align*} The Lagrangian is given by \begin{align*} \mathcal{L} = \sum_{t=0}^T \{\beta^t \ln{c_t} + \lambda_t[(1+\alpha)s_t - \alpha c_t - s_{t+1}]\}. \end{align*} You can easily deduce that (FOC w.r.t. $$c_t$$, $$c_{t+1}$$ and $$s_{t+1}$$) \begin{align*} c_{t+1} = \beta (1+\alpha)c_t. \end{align*} If we now consider the stochastic version of this problem (see Example 15.7) \begin{align*} V = \mathbb{E}_0\left[\sum_{t=0}^T \beta^t \ln{c_t}\right] \end{align*} One might be tempted to deduce that \begin{align*} \mathbb{E}_t[c_{t+1}] = \beta (1+\alpha)c_t. \end{align*} But we are only able to deduce that \begin{align*} \frac{1}{c_t} = \beta (1+\alpha)\mathbb{E}_t\left[\frac{1}{c_{t+1}}\right], \end{align*} since in general $$\mathbb{E}_t[\frac{1}{c_{t+1}}] \neq \frac{1}{\mathbb{E}_t[c_{t+1}]}$$.

• Thanks for the answer. Is there a reason that you chose a durable consumption model of utility? I don't immediately see why it might be important to your broader point. Nov 17 at 21:08
• I'm trying to see where your conclusions differ from that in the notes linked by @Alecos's answer (www3.nd.edu/~esims1/rbc_notes_2016.pdf). Following the procedure from those notes applied to your problem, you would derive equation (4) by shifting the time subscripts on equation (1) before substituting. I'm internally debating what I think about that step. Nov 17 at 21:35
• It seems innocuous, but I think it would amount to assuming that we derive the FOCs for the agent one step in the future and using this assumed optimal future behavior to determine optimal behavior today. I assume that's what dynamic programming is doing more explicitly. Am I missing something? Nov 17 at 21:36
• @jmbejara There is no reason behind the choice of the model. I already had written down this model and wanted to save some time. Nov 17 at 21:41
• @jmbejara Your second question: In Alecos's notes you want to replace $r_{t-1}$ by $r_t$ such that you cannot "put out" the interest rate out of the conditional expectations. In Aleco's notes it holds that $\mathbb{E}_t[1+r_t]=1+r_t$ but general you want to consider $\mathbb{E}_t[1+r_{t+1}]\neq1+r_{t+1}$. This makes the problem harder to solve and you often run into the problem of equation (1)-(3) Nov 17 at 21:44