The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [bellman-equations]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
3answers
79 views

What is unknown in Bellman Equation?

\begin{align} V(W)=\max\limits_{W'\in[0,W]}\qquad& u(W-W')+\beta V(W')\qquad\forall W \end{align} $\textbf{My Question}$: Why is the unknown in the Bellman equation $V(W)$ itself? Isn't the ...
1
vote
0answers
212 views

Explanation of Dynamic Programming “Guess and Verify” Technique

According to my textbook, the analytical technique for solving a Bellman's Equation is as follows: Guess a form for $V_0(x)$ Solve the maximization problem with respect to the control and obtain a ...
4
votes
2answers
157 views

Optimisation using value function

I have the following optimisation problem: max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$ ...
3
votes
1answer
592 views

Bellman equation for this dynamic programming problem

For the following problem \begin{equation}\max_{(\tilde{c}_t,\tilde{a}_{t+1+s})}\sum_{s=0}^{\infty}\beta ^su(\tilde{c}_{t+s})\end{equation} s.t. the following restrictions $\begin{equation}...
3
votes
1answer
1k views

Solution to the Bellman equation is a fixed point

I have recently started studying dynamic optimization. I cannot quite wrap my head around the fact that the value function of the Bellman equation is a fixed point of a contraction mapping. As far my ...
2
votes
1answer
86 views

What is the difference between comma and plus in the bellman equation?

Bellman equation: $V(x) = max \{F(x,y)+ \beta V(y)\}$ $V(x) = max \{F(x,y), \beta V(y)\}$ When to use the plus and when to use the comma? Would you mind give me an example to explain such ...
1
vote
1answer
372 views

Update of value function in continuous time - HJB

When solving (numerically, by value function iteration) a dynamic programming problem in discrete time, such as $$V_1(a) = \max_{c} \ u(c) + \dfrac{1}{1+\rho}V_0(a')$$ we maximize with respect to ...
3
votes
1answer
2k views

Benveniste-Scheinkman condition gives derivative that still depends on the value function

What I mean by the title is often, if we have a value function like $$V(K,I) = \max_{K',I'} F(K') +\beta V(K',I')$$ the First order conditions will give us something that depends on the derivative of ...
2
votes
1answer
70 views

Optimal Stopping

When we solve Bellman equations, I normally would think of the Blanchard Kahn technique. But in the case that I have an optimal stopping problem, or where the decision that the agent has to take is to ...
1
vote
0answers
44 views

Forward-Looking HJB: Rewrite as PDV

We live in continuous time. Let there be some discount rate $D(t)$, which consists of a discount rate, and some death probability. $V(t)$ contains the flow value of, say, being alive. If you are alive,...
3
votes
0answers
95 views

Solving this system of ODE

I have the following system of equations $$ \rho V(u, \epsilon^i) = F(u, \epsilon^i) + V_u(u, \epsilon^i)g(u, V(u, \epsilon^i) + \lambda^i \left(V(u, \epsilon^{-i}) - V(u, \epsilon^i)\right)$$ with $...
4
votes
1answer
165 views

Markov decision processes, contractions and value iteration

I am reviewing Markov decision processes (MDP) and there is something I am missing with respect to the contraction argument. I am pretty sure it is a silly mistake somewhere (maybe computational), but ...