Regarding the arbitrariness of states and controls

Question

I am trying to better understand the process of deriving Euler Equations using the first order condition of the problem on the right hand side of a Bellman equation and the Benveniste-Scheinkman formula. In particular, there is a line in Ljungqvist and Sargent's (LS) Recursive Macroeconomic Theory that I don't quite understand in practice. It is this:

When the state and controls can be defined in such a way that only $u$ appears in the transition equation, i.e., $x' = g(u)$: the derivative of the value function becomes,... $$V'(x)=r_1(x,h(x)).$$

I see why the result follows, granted that states and controls can be defined in the way described. But lets look at a simple optimal growth example.

Let the sequential problem be

$$\max\sum_{t=0}^\infty\beta^t\log(c_t) \\ \text{s.t.} \\ k_{t+1} + c_t = Ak_t^\alpha.$$

The associated Bellman equation is $$V(k)=\max\log(c) + \beta V(k') \\ \text{s.t.} \\ k' + c = Ak^\alpha.$$

Now, in my current understanding $k'$ and $c$ are both controls and $k$ is the state. But, (SL) say "let the state be $k$ and the control be $k'$ where $k'$ denotes next period's value of $k$." However, this does not give me the desired form, i.e. writing $k'=Ak^\alpha-c$ has both a control and a state on the right hand side. How does this fit into the form $x'=g(u)$?

Answer 1

The change from $c$ as a decision variable towards $k'$ as a decision variable is by a simple change of variables.

In the original setup, $k$ is the state and $c$ is the control (decision) variable. The Bellman equation is the following. $$ v(k) = \max_{c \in [0, Ak^\alpha]} u(c) + \beta v(Ak^\alpha - c). $$ (here the constraint $k' = Ak^\alpha - c$ is already substituted into the Bellman equation).

Now, you can define $k' = Ak^\alpha - c$. As $c \in [0, Ak^\alpha]$, we have that $k' \in [0, Ak^\alpha]$. Notice that there is a one-to-one correspondence between the values of $c$ and the values of $k'$. Substituting $c$ by $Ak^\alpha - k'$ in the Bellman equation gives: $$ v(k) = \max_{k' \in [0, Ak^\alpha]} u(Ak^\alpha - k') + \beta v(k'). $$ In this problem $k$ is the state and $k'$ is the control (and also the state in the next period.

The Envelope condition gives: $$ v'(k) = A \alpha k^{\alpha - 1} u'(Ak^\alpha - k') $$ The first order condition gives: $$ -u'(Ak^\alpha - k') + \beta v'(k') = 0. $$ These two can be combined to give the desired Euler equation.

General setting

In general, the Bellman equation takes the following form: $$ v(x) = \max_{u \in \Gamma(x)} f(x,u) + \beta v(r(x,u)). $$ where $x$ is the state variable and $u$ is the control. The law-of-motion function $x' = r(x,u)$ gives the state tomorrow $(x')$ as a function of the state today $(x)$ and the control today $(u)$.

If $r$ is invertible in $u$ (as a function of $u$) we can sometimes rewrite this as: $$ v(x) = \max_{x' \in \Delta(x)} g(x,x') + \beta v(x'). $$ where $g(x,x') = f(x, r^{-1}(x,x'))$ and $$ \Delta(x) = \{x'| \exists u \in \Gamma(x), x' = r(x,u)\}. $$

This new Bellman equation has as state $(x)$ and control $(x')$ which happen to coincide with the state tomorrow.