# Dynamic programming, optimal consumption-savings (finite horizon) problem

Let $$w_t$$ denote a consumer's wealth at time $$t$$ and $$c_t$$, the amount she chooses to consume, so her savings exiting this time period are $$w_t-c_t$$. Given this savings decision, her savings $$w_{t+1}$$ at time $$t+1$$ are determined by a random process in which $$w_{t+1}=\alpha w_t$$ or $$w_{t+1}=\beta w_t$$, where $$\alpha,\beta$$ are postitive constants, each with probability $$\dfrac{1}{2}$$ and independent of past rates of return on her savings. At time $$t$$, when she is choosing $$c_t$$, she knows $$w_t$$ but she doesn’t know anything about future returns except the probabilistic law just given; she does know her past consumption decisions, of course, and past rates of return. Considering a log utility, i.e. $$u(t)=ln(c_t)$$ and assuming that the consumer is impatient, so that he discounts future utility by a factor $$b$$ each period, where $$0 < b < 1$$, then the consumer's decision problem can be written as follows: $$v(c_t)=\max_{c_t \geqslant 0}\sum_{t=0}^{T}b^{t}u(c_t)$$

If the consumer can not borrow, then what is the optimal consumption level for the consumer?

$$\underline{Note:}$$ The problem is based on David M. Kreps' microeconomic theory book, but it is adjusted to be a finite horizon problem. Kreps, in his book, solves this problem in a fuzzy way for $$T=3$$, that is not obvious to me. However, due to the fact that I know little of dynamic programming, since I am in the begining of examining this topic, I would appreciate it, if someone could provide a solution in the finite horizon. If the problem is not set in a rigorous way, I would be glad too to see somene making any appropriate change. I beleive that it is a classic problem in the field of economics.

• Could you elaborate how $w_{t+1}$ evolves? Is it $\alpha w_t$ or $\alpha (w_t - c_t)$ (i.e. pre or post savings)? Nov 16, 2019 at 5:14
• @Walrasian Auctioneer you are right! It is $\alpha(w_t-c_t)$, post savings! Nov 16, 2019 at 8:04
• Ok! Since nobody answered the question, I would like to know if there is some kind of cookbook which provides all the techniques for the dynamic programming (from deterministic to stochastic maths)? Nov 19, 2019 at 23:44

Your value function is as follows: $$V_t[w] = \max_{c_t \in[0,w]} \left\{u(c_t) + \frac{1}{2}V_{t+1}[\alpha(w_t - c_t)] + \frac{1}{2}V_{t+1}[\beta(w_t-c_t)] \right\}$$ with the terminal condition $$V_{T}[w_T] = \max_{c_T \in [0,w_T]} u(c_T)$$

So, we can solve this via backward induction. Clearly, at the final period $$T$$, since $$u$$ is monotonic, we consume everything, so $$V_T[w_T] = u(w_T) = \ln(w_T)$$.

Let us now consider one period previous, so period $$T-1$$. The value function is: $$V_{T-1}[w] = \max_{c_{T-1} \in [0,w_{T-1}]} \left\{u(c_{T-1}) + \frac{1}{2}V_{T}[\alpha(w_{T-1} - c_{T-1})] + \frac{1}{2}V_{T}[\beta(w_{T-1}-c_{T-1})] \right\}$$ We already know what $$V_T[\cdot]$$ is, so substituting, $$V_{T-1}[w] = \max_{c_{T-1} \in [0,w_{T-1}]} \left\{u(c_{T-1}) + \frac{1}{2}u(\alpha(w_{T-1} - c_{T-1})) + \frac{1}{2}u(\beta(w_{T-1}-c_{T-1})) \right\}$$ considering the case $$u(\cdot) = \ln(\cdot)$$, $$V_{T-1}[w] = \max_{c_{T-1} \in [0,w_{T-1}]} \left\{\ln(c_{T-1}) + \frac{1}{2}\ln(\alpha(w_{T-1} - c_{T-1})) + \frac{1}{2}\ln(\beta(w_{T-1}-c_{T-1})) \right\}$$ Take first order conditions, and we can get optimal $$c_{T-1}^*(w_{T-1})$$.

Then we have solved what $$V_{T-1}[\cdot]$$ is! We can then do the same procedure by considering $$V_{T-2}$$. Repeat until we get to $$V_0$$.

EDIT to reflect comment. You now know that $$c^*_{T−1}(w_{𝑇−1})=\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1}$$. Plugging back into $$V_{T-1}$$, we have solved for the value of $$V_{T-1}$$, \begin{align} V_{T-1}[w] &= \ln(\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1}) + \frac{1}{2}\ln(\alpha(w_{T-1} - \frac{\alpha\beta}{1 + \alpha \beta}w_{T-1})) + \frac{1}{2}\ln(\beta(w_{T-1}-\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1})) \\ &=\ln(\frac{\alpha\beta}{1 + \alpha \beta}w_{T-1}) + \frac{1}{2}\ln(\alpha(\frac{w_{T-1}}{1 + \alpha \beta})) + \frac{1}{2}\ln(\beta(\frac{w_{T-1}}{1 + \alpha \beta})) \end{align}

Finally, lets now go to $$T-2$$. The value function is $$V_{T-2}[w] = \max_{c_{T-2} \in[0,w_{T-2}]} \left\{\ln(c_{T-2}) + \frac{1}{2}V_{T-1}[\alpha(w_{T-2} - c_{T-2})] + \frac{1}{2}V_{T-1}[\beta(w_{T-2}-c_{T-2})] \right\}$$ We just solved what $$V_{T-1}$$ is! Plug it in, and repeat.

This expression will probably blow up quite quickly, so it will be a pain to solve analytically.

• Walrasian Auctioneer thank you very much! Does any book have these techiques gathered all together so as not to disturb anyone in the future :) for such a problem? Nov 21, 2019 at 10:26
• Well, if we solve thr FOC for $V_{T-1}$, we can take that $c^{*}_{T-1}(w_{T-1})=\dfrac{\alpha\beta}{1+\alpha\beta}w_{T-1}$. But, I am a little confused, about how to apply this result to find $V_{T-2}$ and so on until I find $V_0$ Nov 21, 2019 at 23:12
• @Nav89 Hopefully the edited answer will be useful. As for techniques, Recursive Macroeconomic Theory Ljungqvist and Sargent is a standard graduate level textbook, but I think googling for lecture notes will be more useful. Nov 22, 2019 at 19:18
• Walrasian Auctioneer Thank you for everything! I need to get back to the work! Nov 22, 2019 at 23:05
• Dear Walrasian Auctioneer user can you please my question which is about recursive competitive equilibrium model economics.stackexchange.com/questions/52248/… Aug 1, 2022 at 18:25

I still can't comment, so this is not another answer it is adding some math to the given answer Since

$$Log X*Y = log X + log Y$$

$$Log(X ^ i) = i * log X$$

So substituting

$$Ln Z +1/2 Ln X + 1/2 Ln Y = Ln (z * Srqt(X*Y))$$

will simplify the expression a lot, but I have to reduce it in handwritten form because I'm writing from a phone now

You will reach

$$max [ w*(w-c)]$$

which clearly happens at

$$c=w/2$$

(try any values)

2- Also, are you sure of ur use of the term "dynamic programming"???

3- And I don't understand from where came the term

$$[ Alpha*Beta / (Alpha*Beta +1)]$$ , that's why I substituted twice (in the original formula before this reduction too)

4- Finally, I don't see why didn't u substitute $$V(t) = u(t)$$ from the beginning to simplify things? I simplified ur formulas mathematically, but I don't get how u reached the EQ (I admit I'm not specialized in ur field I came for an interdisciplinary problems). I get that

$$Saving(t) = Return_of_Saving(t-1) + w(t)- c(t)$$ Then u say her expected wealth "w" at time t+1 is an average of two values. So
$$W(t+1)= (alpha + beta) /2 * W(t) + Income(t+1) - C(t+1)$$ - that u have to add a factor for the decrease of money value & the increase of prices with time, that's ur "b" in the maximization formula right?

Anyways, if u don't care to explain just be informed of these logarithmic relations & use them to simplify • Welcome! Please make an effort to make this answer useful. Apr 18, 2020 at 9:48
• What should I do?
– ShAr
Apr 18, 2020 at 18:28
• Please read this economics.stackexchange.com/help/formatting and also use LaTeX commands, e.g. alpha should be written $\alpha$ to get $\alpha$. Also write down in the post all the results in the image. Apr 18, 2020 at 19:17
• This could be a side note about the write format yes, but I was expecting comments on the topic & points discussed.
– ShAr
Apr 19, 2020 at 3:46
• Hi ShAr, can you please look at my question which is about recursive model economics.stackexchange.com/questions/52248/… Aug 1, 2022 at 18:26