Questions tagged [bellman-equations]
The bellman-equations tag has no usage guidance.
32
questions
1
vote
1
answer
59
views
two questions regarding a bellman optimality proof
Hi: I'm reading a proof of "The Bellman Optimality Principle"
First, if someone knows a clearer proof using any reasonably rigorous methodology, I'm open to reading that one instead. I've ...
- 583
3
votes
1
answer
85
views
When could value functions in Bellman equations be calculated explicitly?
Given the simplest form of a Lucas model, i.e., a Bellman equation given by
\begin{align}
J(x_t) & = \max_{c_t, x_{t+1}} \{ u(c_t) + \beta E_{\pi} [ J(x_{t+1})] \} \\
& \textrm{ s.t. } ...
- 31
4
votes
2
answers
2k
views
Bellman Equation & Envelope Theorem
I'm unsure where the envelope theorem comes into play when i differentiate the Bellman Equation with respect to $k_t$.
To me it looks like the regular chain rule and in fact the exact opposite of the ...
- 991
4
votes
1
answer
562
views
Differentiating Bellman equation
Assume that we have a Bellman equation that is
$$
V(k)=\max_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\right)
$$
The textbook says that if we differentiate with ...
- 321
3
votes
1
answer
287
views
Write down budget constraint
Assume an infinite horizon representative agent economy with the following consumer preferences $u(c_t)$
The production technology of this economy uses capital and land, which is fixed amount in ...
- 172
1
vote
1
answer
119
views
Wealth in the utility function
Suppose there is a representative household of unit mass who lives forever. Preferences are given as: $$\sum\beta^tu(c_t,k_{t-1})$$
Technology is given as: $$k_{t+1}=AF(K_t,L_t)+(1-\delta)K_t-c_t$$ ...
- 1,048
4
votes
1
answer
398
views
Dynamic programming in infinite horizon model
Using an infinite horizon model, a dynamic programming approach uses a fixed point to solve the model: $V = \Gamma(V)$.
How do I interpret the meaning of $V$? For example, when we decide a investment ...
- 195
3
votes
1
answer
224
views
Non-trivial steady state
Consider the growth model with inelastic labor supply, full depreciation, log utility and CRS technology with the Bellman equation be defined as follows:
$$V(k)=\max(log(k^\alpha-k')+\beta V(k'))$$
st ...
- 1,048
4
votes
1
answer
158
views
Bellman Equation with Two Discount Factors
For the following social planner's problem
$$
\max \mathbb{E_{0}}\sum_{s=0}^{\infty}\beta_{1}^{s}(\alpha U(C_{s}^{1}))+\beta_{2}^{s}((1-\alpha)U(C_{s}^{2}))
$$
$$
s.t.\ \text{some constraints ...
- 65
1
vote
1
answer
261
views
Bellman equation corresponding to stochastic EZW recursive utility
A bit of a basic question maybe, but I am confused how to write down the Bellman of this recursive utility function based on Epstein-Zin-Weil preferences and some stochastic constraints.
$$
U_t = \...
- 1,898
0
votes
0
answers
80
views
Dynamic programming and Difference equations applications
I'm asked by my teacher to prepare a presentation with economic applications of Dynamic Programing (Bellman Equation) and Difference equations. I'm not sure what this things are used for in economics ...
4
votes
0
answers
201
views
What are the boundary value conditions for generic HJBs in economics?
Consider a routine continuous time optimization problem:
$
V(t,a_{t}) :=
\max \int_{\tau=t}^{\tau = T}
e^{-\rho (\tau -t)} u(c_{\tau})d\tau
$ $\text{ s.t. }$
$\dot{a}_{t} = y + ra_{t} - c_{t}$,
$a_{...
- 340
0
votes
1
answer
47
views
What is the probability of an unemployed worker receiving no job offer during a time period?
we are currently covering one sided search models and I had a question for you all. I kind of understand the raw calculus behind finding the probability of a job offer over a time interval h, but what ...
- 87
0
votes
0
answers
67
views
How does one find values of constants in a value function?
Problem:
suppose I have the following maximization problem:
\begin{align}
&\max_{(c_t)_{t \geq 0}}E_0\left[\sum^{\infty}_{t=0}\beta^{t}\ln c_t\right]\\[2mm]
\text{s.t.} \quad & k_{t+1}=A^{1-\...
- 87
4
votes
1
answer
618
views
More than one Bellman Equation
I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation.
How do you realize that? I mean how do you know ...
- 103
2
votes
1
answer
165
views
What is the result of the Bellman Equation
I'm just starting with dynamic optimization and although I understant the proof's of the theorem I'm not able to fully understand whether the bellman equation is a function , a function valuated at ...
- 103
4
votes
0
answers
227
views
Eating a Cake with Uncertain Preferences
I've been playing around with a lot of cake eating problems and have been messing with how uncertainty could enter the model. One thing that I'm thinking about is whether we can solve a cake eating ...
- 8,487
2
votes
2
answers
112
views
Why do game theorists use a discounted payoff of this form?
Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form
$$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$
This differs from the discounted ...
- 265
2
votes
1
answer
720
views
Do policy functions exist for Finite Horizon Dynamic programming problems?
I've been looking at the cake eating problem over a finite horizion and have been trying to figure out if we can derive a policy function for such a problem. My work is written below.
sequence form ...
- 8,487
1
vote
0
answers
244
views
Can the Bellman Equation be used for Finite time problems?
Its known that the Bellman equation in recursive macroeconomics is used for the following problems:
$$\sum_{t=1}^\infty\beta^tU(c_t)$$
$$s.t. c_t+k_{t+1}=f(k_t)$$
$$k_0>0$$
Im wondering if we can ...
- 8,487
0
votes
3
answers
303
views
What is unknown in Bellman Equation?
\begin{align}
V(W)=\max\limits_{W'\in[0,W]}\qquad& u(W-W')+\beta V(W')\qquad\forall W
\end{align}
$\textbf{My Question}$: Why is the unknown in the Bellman equation $V(W)$ itself? Isn't the ...
- 1,231
1
vote
0
answers
534
views
Explanation of Dynamic Programming "Guess and Verify" Technique
According to my textbook, the analytical technique for solving a Bellman's Equation is as follows:
Guess a form for $V_0(x)$
Solve the maximization problem with respect to the control and obtain a ...
4
votes
2
answers
338
views
Optimisation using value function
I have the following optimisation problem:
max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$
...
- 661
3
votes
1
answer
925
views
Bellman equation for this dynamic programming problem
For the following problem
\begin{equation}\max_{(\tilde{c}_t,\tilde{a}_{t+1+s})}\sum_{s=0}^{\infty}\beta ^su(\tilde{c}_{t+s})\end{equation}
s.t. the following restrictions
$\begin{equation}...
- 61
4
votes
1
answer
3k
views
Solution to the Bellman equation is a fixed point
I have recently started studying dynamic optimization. I cannot quite wrap my head around the fact that the value function of the Bellman equation is a fixed point of a contraction mapping.
As far my ...
- 43
2
votes
1
answer
169
views
What is the difference between comma and plus in the bellman equation?
Bellman equation:
$V(x) = max \{F(x,y)+ \beta V(y)\}$
$V(x) = max \{F(x,y), \beta V(y)\}$
When to use the plus and when to use the comma?
Would you mind give me an example to explain such ...
- 283
4
votes
1
answer
958
views
Update of value function in continuous time - HJB
When solving (numerically, by value function iteration) a dynamic programming problem in discrete time, such as
$$V_1(a) = \max_{c} \ u(c) + \dfrac{1}{1+\rho}V_0(a')$$
we maximize with respect to ...
- 43
3
votes
1
answer
2k
views
Benveniste-Scheinkman condition gives derivative that still depends on the value function
What I mean by the title is often, if we have a value function like
$$V(K,I) = \max_{K',I'} F(K') +\beta V(K',I')$$
the First order conditions will give us something that depends on the derivative of ...
- 260
2
votes
1
answer
110
views
Optimal Stopping
When we solve Bellman equations, I normally would think of the Blanchard Kahn technique. But in the case that I have an optimal stopping problem, or where the decision that the agent has to take is to ...
- 1,633
1
vote
0
answers
54
views
Forward-Looking HJB: Rewrite as PDV
We live in continuous time. Let there be some discount rate $D(t)$, which consists of a discount rate, and some death probability. $V(t)$ contains the flow value of, say, being alive. If you are alive,...
- 10.7k
3
votes
0
answers
121
views
Solving this system of ODE
I have the following system of equations
$$ \rho V(u, \epsilon^i) = F(u, \epsilon^i) + V_u(u, \epsilon^i)g(u, V(u, \epsilon^i) + \lambda^i \left(V(u, \epsilon^{-i}) - V(u, \epsilon^i)\right)$$
with $...
- 10.7k
4
votes
1
answer
285
views
Markov decision processes, contractions and value iteration
I am reviewing Markov decision processes (MDP) and there is something I am missing with respect to the contraction argument. I am pretty sure it is a silly mistake somewhere (maybe computational), but ...
- 5,237