5
votes
Accepted
Differentiating Bellman equation
In a sense, the Bellman equation is a definition of $V(k)$, which might be more obvious if it's written
$$
V(k):=\max_{0\leq k'\leq f\left(k\right)}u\left(f\left(k\right)-k'\right)+\beta V\left(k'\...
- 1,621
5
votes
Accepted
Dynamic programming in infinite horizon model
There are two interrelated maximisation problems. The first is the infinite horizon maximisation problem:
$$
\begin{align*}
v(k) = &\max_{a_1, a_2, \ldots} \sum_{t = 0}^\infty \delta^t F(k_t, c_t),...
- 10.3k
4
votes
Accepted
What is unknown in Bellman Equation?
The original problem was probably of the form $$\max_{\{W_t\}_{t=1}^\infty}\sum_{t=0}^\infty \beta^t u(W_t-W_{t+1}),$$
$$\mbox{s.t. } W_{t+1}\in[0,W_t] \ \forall \ t, ~~ W_0 \mbox{ given}$$
When ...
- 4,188
4
votes
Accepted
What is the difference between comma and plus in the bellman equation?
You should be careful about the way you wrote your two equations because the $\max$ operators are different. In the first equation, you maximize an objective function on a set, meaning you look for ...
- 887
4
votes
Accepted
Bellman Equation with Two Discount Factors
No. Consider the following problem: Each period, one total unit of consumption falls from the sky that can be distributed between the two agents in any way. Their per-period utility is simply how much ...
- 11.7k
4
votes
Accepted
Bellman equation corresponding to stochastic EZW recursive utility
What you have there are the preferences under an arbitrary policy -- what some call the prevalue function. The only thing missing is the max operator. Written with maximization (and making the state ...
3
votes
Accepted
What is the result of the Bellman Equation
...the result of applying sup operator is a NUMBER...
Read it carefully. The equation is
$$
v(x_0) = \sup_{ \{x_t \}_{t \geq 1}} \cdots \quad (1)
$$
This defines a function $v$, called the value ...
- 2,619
3
votes
Accepted
Optimisation using value function
Starting with your original equation:
$max_{c_t, m_t, b_t} E_0\sum_{t=0}^\infty U(c_t, m_t)$
s.t.
(1) $y+\frac{m_{t-1}}{1+\pi_t}+\frac{1+i_{t-1}}{1+\pi_t}b_{t-1}=c_t+m_t+b_t+\tau_t$
Here: $R_{t-1} ...
- 98
3
votes
Bellman equation for this dynamic programming problem
The "second" constraint appears redundant and it confuses matters. Re-arrange the first one to obtain
$$\tilde{a}_{t+1} = \big[\tilde a_t+(1-\delta )Y_t-\tilde{c}_t\big]R_t$$
This tells us that ...
- 33.4k
3
votes
Accepted
Bellman Equation & Envelope Theorem
The key thing to note here is that in the optimum, $c_t$ will depend on $k_t$. Thus, the value function is
\begin{align}
V(k_t, t) &= \max\{u(c_t) + \beta V(f(k_t) - c_t, t + 1)\}\\
&= u(c_t(...
2
votes
When could value functions in Bellman equations be calculated explicitly?
As far as I know, the only method that works is to guess and verify: You guess the functional form of the value function $J(x)$ and verify that it indeed satisfies the Bellman equation.
Of course, ...
- 10.3k
2
votes
Accepted
Wealth in the utility function
Two points:
As Richard Hardy points out in the comment, there is not really any depiction of wealth in the question.
I am very curious which model this is. I can not think of any models that include ...
- 721
2
votes
Accepted
Update of value function in continuous time - HJB
You iterate towards a fixed point, so you want to reach a situation where plugging in your current iterated value produces itself. Now using your notation, we are told that we should calculate
$$V_{n+...
- 33.4k
2
votes
Accepted
Optimal Stopping
I recommend the manuscript here by Lawrence Evans. The related example is described as 'Rocket Railroad Car'. The first instance is on pages 9-12 where a geometric solution is provided. The choice set ...
- 967
2
votes
Accepted
Do policy functions exist for Finite Horizon Dynamic programming problems?
No, they cannot by definition.
To derive a time-invariant policy function, you need to have an infinite horizon problem. This is because the structure of the solution remains the same, no matter when ...
- 1,631
2
votes
Optimisation using value function
Thanks to @Boaten I was able to find the solution. For those interested here are the steps for deriving the Fisher relation:
Combining the FOC $[b_{t}]$ and the envelope for $[b_{t-1}]$ we get
$U_{...
- 651
2
votes
What is the probability of an unemployed worker receiving no job offer during a time period?
Unless I misunderstand the question these seem to be complementing events with probabilities $p$ and $1-p$.
- 28.7k
2
votes
Accepted
Non-trivial steady state
Let's guess that the value function is of the form $a + b \ln(k)$.
Then substituting for $V(k) = a + b \ln(k)$ in the Bellman equation gives:
$$
a + b \ln(k) = \max_{k'}\left(\ln(k^\alpha - k') + \...
- 10.3k
2
votes
Accepted
Why do game theorists use a discounted payoff of this form?
In my experience, it's mainly just for cleanliness for results.
Consider an infinite horizon repeated game, with discounted payoff representation (where I use $\delta = (1-\lambda)$ in your notation)...
- 1,869
1
vote
More than one Bellman Equation
(The second equation for the value function of the unemployed should be
$$
v(w,U)= \max \{v(w,E); \,u[c,1]+\beta\int v(w', U) dF(w')\}. \quad (*)
$$
)
...how do you know when your problem solution ...
- 2,619
1
vote
What is unknown in Bellman Equation?
What you want to maximize is the discounted stream of utility over time. To that end, you can set control variables over time (subject to some constraints, like technology, budget...). We are ...
1
vote
Accepted
Solution to the Bellman equation is a fixed point
I am by no means an expert on this, but maybe this helps. Here is a simple example for a bellman equation
$V(y) = \max_x u(x,y) + \beta V(y')$
$s.t. \, y' = f(x,y)$
This is a functional equation in ...
- 970
Only top scored, non community-wiki answers of a minimum length are eligible