I am trying to better understand the process of deriving Euler Equations using the first order condition of the problem on the right hand side of a Bellman equation and the Benveniste-Scheinkman formula. In particular, there is a line in Ljungqvist and Sargent's (LS) Recursive Macroeconomic Theory that I don't quite understand in practice. It is this:

When the state and controls can be defined in such a way that only $u$ appears in the transition equation, i.e., $x' = g(u)$: the derivative of the value function becomes,... $$V'(x)=r_1(x,h(x)).$$

I see why the result follows, granted that states and controls can be defined in the way described. But lets look at a simple optimal growth example.

Let the sequential problem be

$$\max\sum_{t=0}^\infty\beta^t\log(c_t) \\ \text{s.t.} \\ k_{t+1} + c_t = Ak_t^\alpha.$$

The associated Bellman equation is $$V(k)=\max\log(c) + \beta V(k') \\ \text{s.t.} \\ k' + c = Ak^\alpha.$$

Now, in my current understanding $k'$ and $c$ are both controls and $k$ is the state. But, (SL) say "let the state be $k$ and the control be $k'$ where $k'$ denotes next period's value of $k$." However, this does not give me the desired form, i.e. writing $k'=Ak^\alpha-c$ has both a control and a state on the right hand side. How does this fit into the form $x'=g(u)$?


1 Answer 1


The change from $c$ as a decision variable towards $k'$ as a decision variable is by a simple change of variables.

In the original setup, $k$ is the state and $c$ is the control (decision) variable. The Bellman equation is the following. $$ v(k) = \max_{c \in [0, Ak^\alpha]} u(c) + \beta v(Ak^\alpha - c). $$ (here the constraint $k' = Ak^\alpha - c$ is already substituted into the Bellman equation).

Now, you can define $k' = Ak^\alpha - c$. As $c \in [0, Ak^\alpha]$, we have that $k' \in [0, Ak^\alpha]$. Notice that there is a one-to-one correspondence between the values of $c$ and the values of $k'$. Substituting $c$ by $Ak^\alpha - k'$ in the Bellman equation gives: $$ v(k) = \max_{k' \in [0, Ak^\alpha]} u(Ak^\alpha - k') + \beta v(k'). $$ In this problem $k$ is the state and $k'$ is the control (and also the state in the next period.

The Envelope condition gives: $$ v'(k) = A \alpha k^{\alpha - 1} u'(Ak^\alpha - k') $$ The first order condition gives: $$ -u'(Ak^\alpha - k') + \beta v'(k') = 0. $$ These two can be combined to give the desired Euler equation.

General setting

In general, the Bellman equation takes the following form: $$ v(x) = \max_{u \in \Gamma(x)} f(x,u) + \beta v(r(x,u)). $$ where $x$ is the state variable and $u$ is the control. The law-of-motion function $x' = r(x,u)$ gives the state tomorrow $(x')$ as a function of the state today $(x)$ and the control today $(u)$.

If $r$ is invertible in $u$ (as a function of $u$) we can sometimes rewrite this as: $$ v(x) = \max_{x' \in \Delta(x)} g(x,x') + \beta v(x'). $$ where $g(x,x') = f(x, r^{-1}(x,x'))$ and $$ \Delta(x) = \{x'| \exists u \in \Gamma(x), x' = r(x,u)\}. $$

This new Bellman equation has as state $(x)$ and control $(x')$ which happen to coincide with the state tomorrow.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.