Questions tagged [dynamic-programming]
The dynamic-programming tag has no usage guidance.
59
questions
2
votes
1answer
67 views
Exercise 4.7 in SLP (dynamic programming)
Exercise 4.7 (b) : Show that under Assumptions 4.10 and 4.11, $T:H(X) \to H(X)$.
$H(X)$ is the set of continuous and homogeneous of degree one functions and $Tf(x) = \sup_{y \in \Gamma(x)} \{F(x,y) + ...
0
votes
0answers
53 views
Cost-optimal p2p-trade in a community of households
I’m trying to solve the following problem and I’ve been working on it for a long time already:
I want to optimize electricity-costs in a smart grid. There’s producer and consumer households in the ...
2
votes
1answer
76 views
Proving that the utility is concave
Consider a household which solves the following problem:
$$v(k,r,w)=\underset{c,l\in B{(k,r,ω)}}{\ {max}} \{u(c,l)\}$$
where $u : R_+^2 \rightarrow R$ is a strictly concave, twice continuously ...
2
votes
0answers
45 views
Cake eating problem
Consider an infinitely lived agent born in time zero, endowed with a cake of size $x_0$. The cake is storable (without depreciation) and infinitely divisible. The agent derives contemporary utility ...
0
votes
0answers
34 views
Dynamic programming and Difference equations applications
I'm asked by my teacher to prepare a presentation with economic applications of Dynamic Programing (Bellman Equation) and Difference equations. I'm not sure what this things are used for in economics ...
3
votes
1answer
141 views
A Cake Eating Problem in Continuous Time: Hamiltonian or HJB?
Your standard continuous time cake eating problem is defined as follows:
$$\max_{c(t)}\int_0^\infty e^{-rt} \ln (c(t)) dt$$
subject to
$$f(k(t))=k(t)$$
$$\dot{k}(t)=-c(t)$$
Approaching this problem by ...
2
votes
1answer
237 views
The Principle of Optimality and the Bellman Equation
Given (1) and (2), is it possible to show the existence of a Bellman equation (3), using Bellman's Principle of Optimality?
$$\ max \Sigma\beta^s U(C_t)$$
Subject to the following resource ...
3
votes
0answers
110 views
What are the boundary value conditions for generic HJBs in economics?
Consider a routine continuous time optimization problem:
$
V(t,a_{t}) :=
\max \int_{\tau=t}^{\tau = T}
e^{-\rho (\tau -t)} u(c_{\tau})d\tau
$ $\text{ s.t. }$
$\dot{a}_{t} = y + ra_{t} - c_{t}$,
$a_{...
1
vote
2answers
99 views
What is state space representation for DSGE modeling
I'm beginning with DSGE modeling, and a mathematical representation (perhaps trivial for most of the people that are more with this topic) is the space-state state representation of a dynamical model, ...
5
votes
0answers
54 views
Optimization in discrete time
I have made optimizations in continuous time that belong to the control theory, for example one case:
$\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$
constraint to: $\dot x=g(t,x(t),u(t))$
Where:
$x(t)$: ...
2
votes
1answer
52 views
Can anyone help me understand the Motrtensen-Pissarides model?
I am seeking help with the Mortensen-Pissarides model in discrete time? Basically, I was given the following in class example (which we didn’t complete do to time constraints associated with the end ...
0
votes
0answers
24 views
Numerical Backward Induction Optimal portfolio choice
I am currently considering a simple life-cycle problem. We consider a market with equity risk only, which follows a geometric Brownian motion. We seek to maximize the terminal wealth of a CRRA utility ...
4
votes
2answers
181 views
Solving a HJB with a probability to transit to a new state
I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time.
The firm maximizes $V(k)=\int_{t=0}^{\infty}e^{-rt} \pi_t dt$ with $\pi_t=f(k_t)-i_t$ and $\dot{k}...
3
votes
1answer
126 views
More than one Bellman Equation
I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation.
How do you realize that? I mean how do you know ...
1
vote
0answers
35 views
Coase conjecture and Stokey model
The durable goods monopolist can charge different prices by every periods. Critical type consumers are indifferent between buying today and buying tomorrow. So I construct a Bellman equation like ...
1
vote
1answer
76 views
What is the result of the Bellman Equation
I'm just starting with dynamic optimization and although I understant the proof's of the theorem I'm not able to fully understand whether the bellman equation is a function , a function valuated at ...
2
votes
2answers
69 views
Why do game theorists use a discounted payoff of this form?
Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form
$$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$
This differs from the discounted ...
2
votes
0answers
78 views
Linearization of the dynamic system (I did it, but I have a mistake that I cannot catch. Help me please)
I have the following dynamic system in discrete time
For p is price, d is demand and s is supply.
$$p_{t+1}-p_t= a(d_t-s_t)$$
$$s_{t+1}-s_t=bp_ts_t-ws_t$$
$$d_t= k-gp_t$$
I have to linearize this ...
2
votes
1answer
330 views
The Cake Eating Problem with Depreciation (Modelling difficulties)
How does one go about modelling the cake eating problem with depreciation? (i.e The cake goes bad over time)
The problem I have is the following.
Lets define a cake eating problem sequentially as:
...
2
votes
2answers
242 views
Dynamic programming, optimal consumption-savings (finite horizon) problem
Let $w_t$ denote a consumer's wealth at time $t$ and $c_t$,
the amount she chooses to consume, so her savings exiting this time period are $w_t-c_t$. Given this savings decision, her savings $w_{t+1}$ ...
1
vote
1answer
156 views
A profit maximization problem (whole problem has been solved, I just have question about interpretation)
I would like to discuss with you about the following production function.
$$y=f(t_m, t_l)=\rho t_m^m(n+t_l)$$
where $0<m<1 $ and $n>0$ are fixed parameters.
$t_m$ is manager time.
$t_l$ ...
0
votes
1answer
73 views
Sequential Price Competition for Perfect Complements
There are two goods, $1$ and $2$ produced by two firms at zero marginal costs. The goods are perfect complements. The demand for each goods is: $Q_1=Q_2=a-(p_1+p_2)$. The prices are set sequentially, ...
0
votes
3answers
151 views
What is unknown in Bellman Equation?
\begin{align}
V(W)=\max\limits_{W'\in[0,W]}\qquad& u(W-W')+\beta V(W')\qquad\forall W
\end{align}
$\textbf{My Question}$: Why is the unknown in the Bellman equation $V(W)$ itself? Isn't the ...
2
votes
0answers
46 views
Why is there a Lagrangian Multiplier in the Dynamic Programming Problem of the RBC model?
Suppose that the household faces the following problem:
$\underset{ c_t , k_{t+1}, n_t } \max \mathbb{E}_0 \sum_{t=0}^{\infty} \beta^t \ln c_t + \ln (1 - n_t)$
subjected to
$ k_{t+1} = A_t k_t ^{\...
1
vote
2answers
274 views
In Blackwell's condition for T to be a contraction mapping, we require that satisfies discounting. What is the intuition of discounting?
The discounting condition is as follow:
There exists some $\beta \in (0, 1)$ such that $[T(f + a)](x) ≤ (T f)(x) + βa$, for all $f ∈ B(X), a ≥ 0, x ∈ X$.
While the monotonicity condition makes sense,...
1
vote
0answers
201 views
Natural borrowing/debt limit and other borrowing constraints
When confronted with the simple household consumption maximization problem under uncertainty (and with Arrow security sequential trading)
$$\max_{\{c_t(s^t),a_{t+1}(s^t,s_{t+1})\}_{t=0}^{\infty}}\...
1
vote
0answers
280 views
Explanation of Dynamic Programming “Guess and Verify” Technique
According to my textbook, the analytical technique for solving a Bellman's Equation is as follows:
Guess a form for $V_0(x)$
Solve the maximization problem with respect to the control and obtain a ...
0
votes
1answer
73 views
Capital accumulation
I have following dynamic optimization problem
$$V(k_0)=\max \sum_{t=0}^{\infty} b^t((1-a)ln c_t+a ln(l_t))$$
Subject to $l_t+e_t=h$ and $y=Ak_t^pe_t^{1-p}$ and $k_{t+1}=i_t+(1-x)k_t$
Where c is ...
3
votes
0answers
646 views
Market clearing condition with Walras law
I have a diamond overlapping model
The question is as follows
Let us consider an infinitely lived production economy populated at time t by $N_t$ identical and perfectly competitive adult ...
6
votes
1answer
191 views
How do I begin to approach this dynamic discrete choice model?
I'm working through an old problem set (that sadly I don't have solutions for) and I got stuck. It is a dynamic model of entrepreneurship and invention. I'm looking for guidance on this model as well ...
4
votes
0answers
193 views
Solution to Dynamic Programming (Bellman Equation) Problem
Could someone please provide pointers on how to solve the below? If any theoretical approximations are possible, that would be very helpful. If numerical solutions are the right approach, could you ...
1
vote
0answers
97 views
Stochastic Dynamic Programming: Deriving the Steady-State for a Lottery
I am working through the basic examples of the stochastic RBC models in the book by McCandless (2008): The ABCs of RBCs, pp. 71 - 75
A Standard Stochastic Dynamic Programming Problem
Here is a ...
1
vote
1answer
103 views
A reference for most used utility functions in macroeconomic problems of intertemporal optimization
I'm looking for a reference with the most used utility functions in macroeconomic problems of intertemporal optimization. The reference should preferably include a list of properties of those ...
4
votes
1answer
34 views
Question regarding Carlstrom and Feurst (1997)
I am reading through Carlstrom and Feurst's 1997 paper Agency Costs, Net Worth, and Business Fluctuations: A Computable General Equilibrium Analysis and had a question, although it could apply to ...
4
votes
2answers
163 views
Optimisation using value function
I have the following optimisation problem:
max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$
...
1
vote
1answer
158 views
Dynamic programming with housing consumption and labor
I try to solve the following maximization problem of a representative household with dynamic programming. However, my last result is not similar to the solution. Could any one help me?
$$\max\...
3
votes
1answer
710 views
Bellman equation for this dynamic programming problem
For the following problem
\begin{equation}\max_{(\tilde{c}_t,\tilde{a}_{t+1+s})}\sum_{s=0}^{\infty}\beta ^su(\tilde{c}_{t+s})\end{equation}
s.t. the following restrictions
$\begin{equation}...
4
votes
1answer
2k views
Solution to the Bellman equation is a fixed point
I have recently started studying dynamic optimization. I cannot quite wrap my head around the fact that the value function of the Bellman equation is a fixed point of a contraction mapping.
As far my ...
1
vote
1answer
138 views
How to deal with Prescott's formulation of time to build in his original RBC model?
So I was replicating the results obtained in section 4 of Prescott's original paper, which derives optimality conditions in steady state without shock. I hope to solve the social planner's ...
1
vote
0answers
144 views
How to solve a variation of Merton's optimal portfolio problem?
Does anyone know how to solve the following problem?
I have tried to solve this but I'm lost since I have never dealt with a Stochastic Dynamic Programming problem with many variables.
$max_{c_{t},\...
1
vote
0answers
74 views
Hamilton-Jacobi-Bellman with heterogeneous discount rates
Let $i=1,2$ denote the players, $x$ the state and $u_i$ the control of player $i$. The state equation reads $\dot x = f(x,u_1,u_2)$ and the objective function is given by $F_i(x,u_1,u_2)$. Now I'd ...
2
votes
1answer
128 views
Dynamic programming problem with dimension over 1000
I am working on a project which need to solve a dynamic programming problem with dimension over 1000. In past literature, there exist several methods like Smolyak algorithm and Sparse grid method that ...
7
votes
3answers
726 views
Solution Method for Infinite-Horizon Maximization Problem
Full disclosure: this problem was part of a final exam that none of our class could really solve definitively. Below the general form is a specific utility function we worked with that I'll try to ...
2
votes
1answer
519 views
Update of value function in continuous time - HJB
When solving (numerically, by value function iteration) a dynamic programming problem in discrete time, such as
$$V_1(a) = \max_{c} \ u(c) + \dfrac{1}{1+\rho}V_0(a')$$
we maximize with respect to ...
3
votes
1answer
68 views
Joint Dynamic Programming: Group Activity
Here we have two agents who can spend their time doing some group activity ($h$) or staying at home ($l$). Each agent $i$ is trying to maximize their respective dynamic programming problem:
\begin{...
3
votes
1answer
277 views
Prove the uniqueness of steady state
I have a difference equation
$$
p_t^{1-\alpha}=\alpha\sigma \left(y-p_t-\frac{(\sigma p_{t-1}^\alpha+b)p_t^{1-\alpha}}{\alpha\sigma} \right)
$$
where $\alpha \in [0,1]$ and everything else is $&...
1
vote
0answers
101 views
Dynamic programming: verification principle
Consider the Gale cake problem with $u(c) = log(c)$, so that the problem becomes:$$\max_{x}\sum_{t=0}^{\infty} \beta^{t}log(x_t-x_{t+1})\: sub\: 0<x_{t+1}\leq x_t,\: x_0>0\ given$$
It is ...
3
votes
1answer
2k views
Benveniste-Scheinkman condition gives derivative that still depends on the value function
What I mean by the title is often, if we have a value function like
$$V(K,I) = \max_{K',I'} F(K') +\beta V(K',I')$$
the First order conditions will give us something that depends on the derivative of ...
2
votes
0answers
98 views
Taking limit of a sequence
I am given a following dynamic programming problem;
$$
\sup_{k_{t+1}}\sum_{t=0}^{t=\infty}\delta^t(ak_t-\frac{b}{2}k_t^2-\frac{c}{2}(k_{t+1}-k_t)^2)
$$
where $f(k_t) = ak_t-\frac{b}{2}k_t^2$ is the ...
4
votes
1answer
81 views
Conjecture Steady State from limit properties
The question is related to this thread. I'd like to derive a unique steady state for an optimal control problem.
Consider the following programm
\begin{align}
&V(x_0) := \max_u \int^\infty_0 e^{-\...
