What is the difference between comma and plus in the bellman equation?

Question

Bellman equation:

$V(x) = max \{F(x,y)+ \beta V(y)\}$

$V(x) = max \{F(x,y), \beta V(y)\}$

When to use the plus and when to use the comma?

Would you mind give me an example to explain such difference?

Thank you very much !

Answer 1

You should be careful about the way you wrote your two equations because the $\max$ operators are different. In the first equation, you maximize an objective function on a set, meaning you look for the best value of $y$ given that $y$ belongs to a known set. You did not define this set. In the second case, you choose the maximum among two values. Contrary to the first equation, $y$ is predefined and is not a parameter to find given your notations.

If I abstract from the notation mistakes, I think you want an example to understand the difference between a classical cake-eating problem (your first equation) and a stopping-rule problem (probably your second equation).

Consider the following framework. Time is discrete. There is one good in the economy. We are interested in the problem of a consumer. At each period $t$, the consumer has access to a stock of good $x_t$. She faces a standard inter-temporal saving problem; she has to decide how much to consume, $c_t$, and how much to save for the next periods, $x_{t+1}$. The consumer enjoys a flow utility $u(c_t)$ and discounts future consumption with a factor $\beta$. To close the model, a saving technology must be defined. We can consider two different cases:

(i) There is an interest rate $r$. The consumer can freely withdraw any amount from the saving account. (ii) There is an interest rate $r$. The consumer, however, cannot withdraw a quantity of good without closing the saving account.

Let's write the optimization problem in the two cases:

(i) $$V(x_t)=\underset{0\leq x_{t+1}\leq (1+r)x_t}{\max} \{u((1+r)x_t-x_{t+1})+\beta V(x_{t+1})\}$$

(ii) $$V(x_t)=\underset{x_{t+1}= (1+r)x_t \text{ or } 0}{\max} \{u((1+r)x_t-x_{t+1})+\beta V(x_{t+1})\}$$ Consumption $c_t$ is equal to $(1+r)x_t-x_{t+1}$. We can make an analogy with what you wrote. My $x_t$ and $x_{t+1}$ are your $x$ and $y$, and $F(x,y)=u((1+r)x-y)$. In the first case, the consumer has to choose the amount to save $x_{t+1}$ in the interval $(0,(1+r)x_t)$. In the second case, she can only choose whether to keep on saving, $x_{t+1}=(1+r)x_t$, or to close the account, $x_{t+1}=0$. By normalizing $u(0)=0$, the second case can write:

(ii) $$V(x_t)=\max \{u((1+r)x_t),\beta V((1+r)x_t)\}$$

In the first problem, the consumer eats a share of the cake (which is expanding at a rate $r$) and keeps the remaining for the next periods to maximize her inter-temporal utility. In the second problem, the consumer has to decide when to eat entirely the (growing) cake, which defines the stopping rule.