Bellman equation:

$V(x) = max \{F(x,y)+ \beta V(y)\}$

$V(x) = max \{F(x,y), \beta V(y)\}$

When to use the plus and when to use the comma?

Would you mind give me an example to explain such difference?

Thank you very much !

  • $\begingroup$ The two expressions have totally different meanings. Where did you see the "comma" expression? $\endgroup$ – Alecos Papadopoulos Jun 15 '16 at 18:43
  • $\begingroup$ A lecture note. Here is the link goo.gl/aTDLpH . At the top of page 4. @AlecosPapadopoulos $\endgroup$ – XJ.C Jun 16 '16 at 12:28

You should be careful about the way you wrote your two equations because the $\max$ operators are different. In the first equation, you maximize an objective function on a set, meaning you look for the best value of $y$ given that $y$ belongs to a known set. You did not define this set. In the second case, you choose the maximum among two values. Contrary to the first equation, $y$ is predefined and is not a parameter to find given your notations.

If I abstract from the notation mistakes, I think you want an example to understand the difference between a classical cake-eating problem (your first equation) and a stopping-rule problem (probably your second equation).

Consider the following framework. Time is discrete. There is one good in the economy. We are interested in the problem of a consumer. At each period $t$, the consumer has access to a stock of good $x_t$. She faces a standard inter-temporal saving problem; she has to decide how much to consume, $c_t$, and how much to save for the next periods, $x_{t+1}$. The consumer enjoys a flow utility $u(c_t)$ and discounts future consumption with a factor $\beta$. To close the model, a saving technology must be defined. We can consider two different cases:

(i) There is an interest rate $r$. The consumer can freely withdraw any amount from the saving account. (ii) There is an interest rate $r$. The consumer, however, cannot withdraw a quantity of good without closing the saving account.

Let's write the optimization problem in the two cases:

(i) $$V(x_t)=\underset{0\leq x_{t+1}\leq (1+r)x_t}{\max} \{u((1+r)x_t-x_{t+1})+\beta V(x_{t+1})\}$$

(ii) $$V(x_t)=\underset{x_{t+1}= (1+r)x_t \text{ or } 0}{\max} \{u((1+r)x_t-x_{t+1})+\beta V(x_{t+1})\}$$ Consumption $c_t$ is equal to $(1+r)x_t-x_{t+1}$. We can make an analogy with what you wrote. My $x_t$ and $x_{t+1}$ are your $x$ and $y$, and $F(x,y)=u((1+r)x-y)$. In the first case, the consumer has to choose the amount to save $x_{t+1}$ in the interval $(0,(1+r)x_t)$. In the second case, she can only choose whether to keep on saving, $x_{t+1}=(1+r)x_t$, or to close the account, $x_{t+1}=0$. By normalizing $u(0)=0$, the second case can write:

(ii) $$V(x_t)=\max \{u((1+r)x_t),\beta V((1+r)x_t)\}$$

In the first problem, the consumer eats a share of the cake (which is expanding at a rate $r$) and keeps the remaining for the next periods to maximize her inter-temporal utility. In the second problem, the consumer has to decide when to eat entirely the (growing) cake, which defines the stopping rule.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.