Questions tagged [bellman-equations]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
61 views

How does one find values of constants in a value function?

Problem: suppose I have the following maximization problem: \begin{align} &\max_{(c_t)_{t \geq 0}}E_0\left[\sum^{\infty}_{t=0}\beta^{t}\ln c_t\right]\\[2mm] \text{s.t.} \quad & k_{t+1}=A^{1-\...
3
votes
1answer
81 views

More than one Bellman Equation

I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation. How do you realize that? I mean how do you know ...
1
vote
1answer
47 views

What is the result of the Bellman Equation

I'm just starting with dynamic optimization and although I understant the proof's of the theorem I'm not able to fully understand whether the bellman equation is a function , a function valuated at ...
4
votes
0answers
145 views

Eating a Cake with Uncertain Preferences

I've been playing around with a lot of cake eating problems and have been messing with how uncertainty could enter the model. One thing that I'm thinking about is whether we can solve a cake eating ...
2
votes
2answers
57 views

Why do game theorists use a discounted payoff of this form?

Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form $$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$ This differs from the discounted ...
2
votes
1answer
89 views

Do policy functions exist for Finite Horizon Dynamic programming problems?

I've been looking at the cake eating problem over a finite horizion and have been trying to figure out if we can derive a policy function for such a problem. My work is written below. sequence form ...
1
vote
0answers
41 views

Can the Bellman Equation be used for Finite time problems?

Its known that the Bellman equation in recursive macroeconomics is used for the following problems: $$\sum_{t=1}^\infty\beta^tU(c_t)$$ $$s.t. c_t+k_{t+1}=f(k_t)$$ $$k_0>0$$ Im wondering if we can ...
0
votes
3answers
114 views

What is unknown in Bellman Equation?

\begin{align} V(W)=\max\limits_{W'\in[0,W]}\qquad& u(W-W')+\beta V(W')\qquad\forall W \end{align} $\textbf{My Question}$: Why is the unknown in the Bellman equation $V(W)$ itself? Isn't the ...
1
vote
0answers
265 views

Explanation of Dynamic Programming “Guess and Verify” Technique

According to my textbook, the analytical technique for solving a Bellman's Equation is as follows: Guess a form for $V_0(x)$ Solve the maximization problem with respect to the control and obtain a ...
4
votes
2answers
157 views

Optimisation using value function

I have the following optimisation problem: max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$ ...
3
votes
1answer
667 views

Bellman equation for this dynamic programming problem

For the following problem \begin{equation}\max_{(\tilde{c}_t,\tilde{a}_{t+1+s})}\sum_{s=0}^{\infty}\beta ^su(\tilde{c}_{t+s})\end{equation} s.t. the following restrictions $\begin{equation}...
4
votes
1answer
2k views

Solution to the Bellman equation is a fixed point

I have recently started studying dynamic optimization. I cannot quite wrap my head around the fact that the value function of the Bellman equation is a fixed point of a contraction mapping. As far my ...
2
votes
1answer
87 views

What is the difference between comma and plus in the bellman equation?

Bellman equation: $V(x) = max \{F(x,y)+ \beta V(y)\}$ $V(x) = max \{F(x,y), \beta V(y)\}$ When to use the plus and when to use the comma? Would you mind give me an example to explain such ...
1
vote
1answer
454 views

Update of value function in continuous time - HJB

When solving (numerically, by value function iteration) a dynamic programming problem in discrete time, such as $$V_1(a) = \max_{c} \ u(c) + \dfrac{1}{1+\rho}V_0(a')$$ we maximize with respect to ...
3
votes
1answer
2k views

Benveniste-Scheinkman condition gives derivative that still depends on the value function

What I mean by the title is often, if we have a value function like $$V(K,I) = \max_{K',I'} F(K') +\beta V(K',I')$$ the First order conditions will give us something that depends on the derivative of ...
2
votes
1answer
76 views

Optimal Stopping

When we solve Bellman equations, I normally would think of the Blanchard Kahn technique. But in the case that I have an optimal stopping problem, or where the decision that the agent has to take is to ...
1
vote
0answers
44 views

Forward-Looking HJB: Rewrite as PDV

We live in continuous time. Let there be some discount rate $D(t)$, which consists of a discount rate, and some death probability. $V(t)$ contains the flow value of, say, being alive. If you are alive,...
3
votes
0answers
105 views

Solving this system of ODE

I have the following system of equations $$ \rho V(u, \epsilon^i) = F(u, \epsilon^i) + V_u(u, \epsilon^i)g(u, V(u, \epsilon^i) + \lambda^i \left(V(u, \epsilon^{-i}) - V(u, \epsilon^i)\right)$$ with $...
4
votes
1answer
182 views

Markov decision processes, contractions and value iteration

I am reviewing Markov decision processes (MDP) and there is something I am missing with respect to the contraction argument. I am pretty sure it is a silly mistake somewhere (maybe computational), but ...