# Dynamic programming, optimal consumption-savings (finite horizon) problem

Let $$w_t$$ denote a consumer's wealth at time $$t$$ and $$c_t$$, the amount she chooses to consume, so her savings exiting this time period are $$w_t-c_t$$. Given this savings decision, her savings $$w_{t+1}$$ at time $$t+1$$ are determined by a random process in which $$w_{t+1}=\alpha w_t$$ or $$w_{t+1}=\beta w_t$$, where $$\alpha,\beta$$ are postitive constants, each with probability $$\dfrac{1}{2}$$ and independent of past rates of return on her savings. At time $$t$$, when she is choosing $$c_t$$, she knows $$w_t$$ but she doesn’t know anything about future returns except the probabilistic law just given; she does know her past consumption decisions, of course, and past rates of return. Considering a log utility, i.e. $$u(t)=ln(c_t)$$ and assuming that the consumer is impatient, so that he discounts future utility by a factor $$b$$ each period, where $$0 < b < 1$$, then the consumer's decision problem can be written as follows: $$v(c_t)=\max_{c_t \geqslant 0}\sum_{t=0}^{T}b^{t}u(c_t)$$

If the consumer can not borrow, then what is the optimal consumption level for the consumer?

$$\underline{Note:}$$ The problem is based on David M. Kreps' microeconomic theory book, but it is adjusted to be a finite horizon problem. Kreps, in his book, solves this problem in a fuzzy way for $$T=3$$, that is not obvious to me. However, due to the fact that I know little of dynamic programming, since I am in the begining of examining this topic, I would appreciate it, if someone could provide a solution in the finite horizon. If the problem is not set in a rigorous way, I would be glad too to see somene making any appropriate change. I beleive that it is a classic problem in the field of economics.

• Could you elaborate how $w_{t+1}$ evolves? Is it $\alpha w_t$ or $\alpha (w_t - c_t)$ (i.e. pre or post savings)? – Walrasian Auctioneer Nov 16 '19 at 5:14
• @Walrasian Auctioneer you are right! It is $\alpha(w_t-c_t)$, post savings! – Nav89 Nov 16 '19 at 8:04
• Ok! Since nobody answered the question, I would like to know if there is some kind of cookbook which provides all the techniques for the dynamic programming (from deterministic to stochastic maths)? – Nav89 Nov 19 '19 at 23:44

Your value function is as follows: $$V_t[w] = \max_{c_t \in[0,w]} \left\{u(c_t) + \frac{1}{2}V_{t+1}[\alpha(w_t - c_t)] + \frac{1}{2}V_{t+1}[\beta(w_t-c_t)] \right\}$$ with the terminal condition $$V_{T}[w_T] = \max_{c_T \in [0,w_T]} u(c_T)$$

So, we can solve this via backward induction. Clearly, at the final period $$T$$, since $$u$$ is monotonic, we consume everything, so $$V_T[w_T] = u(w_T) = \ln(w_T)$$.

Let us now consider one period previous, so period $$T-1$$. The value function is: $$V_{T-1}[w] = \max_{c_{T-1} \in [0,w_{T-1}]} \left\{u(c_{T-1}) + \frac{1}{2}V_{T}[\alpha(w_{T-1} - c_{T-1})] + \frac{1}{2}V_{T}[\beta(w_{T-1}-c_{T-1})] \right\}$$ We already know what $$V_T[\cdot]$$ is, so substituting, $$V_{T-1}[w] = \max_{c_{T-1} \in [0,w_{T-1}]} \left\{u(c_{T-1}) + \frac{1}{2}u(\alpha(w_{T-1} - c_{T-1})) + \frac{1}{2}u(\beta(w_{T-1}-c_{T-1})) \right\}$$ considering the case $$u(\cdot) = \ln(\cdot)$$, $$V_{T-1}[w] = \max_{c_{T-1} \in [0,w_{T-1}]} \left\{\ln(c_{T-1}) + \frac{1}{2}\ln(\alpha(w_{T-1} - c_{T-1})) + \frac{1}{2}\ln(\beta(w_{T-1}-c_{T-1})) \right\}$$ Take first order conditions, and we can get optimal $$c_{T-1}^*(w_{T-1})$$.

Then we have solved what $$V_{T-1}[\cdot]$$ is! We can then do the same procedure by considering $$V_{T-2}$$. Repeat until we get to $$V_0$$.

EDIT to reflect comment. You now know that $$c^*_{T−1}(w_{𝑇−1})=\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1}$$. Plugging back into $$V_{T-1}$$, we have solved for the value of $$V_{T-1}$$, \begin{align} V_{T-1}[w] &= \ln(\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1}) + \frac{1}{2}\ln(\alpha(w_{T-1} - \frac{\alpha\beta}{1 + \alpha \beta}w_{T-1})) + \frac{1}{2}\ln(\beta(w_{T-1}-\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1})) \\ &=\ln(\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1}) + \frac{1}{2}\ln(\alpha(\frac{w_{T-1}}{1 + \alpha \beta})) + \frac{1}{2}\ln(\beta(\frac{w_{T-1}}{1 + \alpha \beta})) \end{align}

Finally, lets now go to $$T-2$$. The value function is $$V_{T-2}[w] = \max_{c_{T-2} \in[0,w_{T-2}]} \left\{\ln(c_{T-2}) + \frac{1}{2}V_{T-1}[\alpha(w_{T-2} - c_{T-2})] + \frac{1}{2}V_{T-1}[\beta(w_{T-2}-c_{T-2})] \right\}$$ We just solved what $$V_{T-1}$$ is! Plug it in, and repeat.

This expression will probably blow up quite quickly, so it will be a pain to solve analytically.

• Walrasian Auctioneer thank you very much! Does any book have these techiques gathered all together so as not to disturb anyone in the future :) for such a problem? – Nav89 Nov 21 '19 at 10:26
• Well, if we solve thr FOC for $V_{T-1}$, we can take that $c^{*}_{T-1}(w_{T-1})=\dfrac{\alpha\beta}{1+\alpha\beta}w_{T-1}$. But, I am a little confused, about how to apply this result to find $V_{T-2}$ and so on until I find $V_0$ – Nav89 Nov 21 '19 at 23:12
• @Nav89 Hopefully the edited answer will be useful. As for techniques, Recursive Macroeconomic Theory Ljungqvist and Sargent is a standard graduate level textbook, but I think googling for lecture notes will be more useful. – Walrasian Auctioneer Nov 22 '19 at 19:18
• Walrasian Auctioneer Thank you for everything! I need to get back to the work! – Nav89 Nov 22 '19 at 23:05