Questions tagged [dynamic-programming]
The dynamic-programming tag has no usage guidance.
81
questions
0
votes
0
answers
13
views
One shot-deviation property for games of imperfect information
The equivalence between subgame perfection and one-deviation property is typically stated for games of perfect information (where information sets are singletons). Does the Blackwell-style argument ...
2
votes
0
answers
43
views
Mathematical Prerequisites for Recursive Macroeconomic Theory (Thomas J Sargent, Lars Ljungqvist)
I'm a math grad who is interested in learning more about economics for fun.
Reading through RMT, I saw some interesting math (in chapter 2) around using "invariant functions" to determine ...
- 23
1
vote
0
answers
34
views
Are there any ``sophisticated'' mathematical modelling where they solve for the utility function?
Are there any references in literature of any ``sophisticated'' mathematical modelling where they solve for the utility function under specific conditions using differential equations theory?
In such ...
- 143
0
votes
0
answers
19
views
Difference between complete market and incomplete market in terms of welfare loss
What do you think about the relative welfare of an incomplete market and complete market in the recursive competitive economy with both physical and human capital accumulation
—
I think that in the ...
- 96
2
votes
0
answers
44
views
When do Economists use the Linear Quadratic Regulator in simulating DSGE models
I was watching some excellent videos on DSGE models by Klaus Pretner. The author was able to solve some simple model such as the Ramsey-Cass-Koopman's model, and a New Keynesian model with frictions, ...
- 385
1
vote
0
answers
37
views
Advantages of using Bellman equations
Suppose we wish to solve a simple infinite horizon cake eating problem,
such that:
$$
\max_{\left\{ c_{t}\right\} _{t=0}^{\infty}}\sum_{t}^{\infty}\beta^{t}u\left(c_{t}\right)
$$
subject to:
$$
c_{t}+...
- 157
0
votes
0
answers
40
views
catching up with Johnes utility in the recursive equilibrium model
Consider an infinite horizon representative agent economy in which the agents enjoy consumption as usual but also cares about its relative level of consumption compared to last period’s average level ...
- 96
3
votes
0
answers
71
views
Current best methods for solving dynamic optimization problems in high dimensional state spaces
I was wondering what the current best methods are for solving dynamic optimization problems in high dimensional state spaces. Let me lay out the common cases where I would do something like this in ...
- 385
5
votes
1
answer
107
views
When does it make sense to use variational methods, versus dynamic programming, versus nonlinear control methods so solve DSGE models
I come from a statistics and applied math background, but have been looking at some problems related to macroeconomic DSGE models lately. So I am still trying to understand the ideas and economics ...
- 385
0
votes
0
answers
22
views
Policy function iteration method in continuous time (with shocks)
Is there any reference available on the algorithm of policy function iteration method in continuous time, when we have uncertainty in the model?
Currently, my conclusion is that the combination of ...
- 1
4
votes
0
answers
41
views
What assumptions can be made to ensure convexity in this optimization problem?
This question is a continuation of the question I asked at:
How can I show convexity of this value function?
Where I came to the conclusion that more assumptions are required to show that the ...
- 183
4
votes
1
answer
238
views
How can I show convexity of this value function?
I have set up an optmization problem as follows:
$$V(A)=\max_{l, C} \quad u(C,l)$$
Where the only constraint is as follows:
$$C=f(l,A)$$
Here $u$ is the utility function which captures social welfare. ...
- 183
6
votes
1
answer
200
views
Optimal stopping (reference request)
I am interested in the following optimal stopping problem:
On each day, a number $a_i$ is drawn from a (possibly fixed) distribution.
I can either stop now, getting a payoff of $a_i$, or wait for a ...
- 852
3
votes
2
answers
183
views
How can I formulate the following optimization problem?
I want to set up an optmiization problem for global warming in which a planner determines how much carbon dioxide gas is emitted. Let's say we reduce this problem down to two periods, then I ...
user40867
1
vote
1
answer
177
views
simplification of FOC
This is the first-order condition of a dynamic programming problem where I am trying to get the Euler equation from a sequential problem.
(1) $$\frac{\partial V(d_2)}{\partial d_3} = \frac{-1}{d_2-d_3}...
2
votes
0
answers
40
views
More equations than endogenous variables in RBC model
I recently came across my late father's bachelor's thesis, in which a RBC model is described. It appears to be a variant of Hansen's 1985 model with indivisible labor. I wrote the model into Dynare to ...
- 21
1
vote
0
answers
34
views
How to define a dynamic programming problem in an incomplete information game?
How can I define a problem of dynamic programming, to use the Hamilton-Jacobi-Bellman equation in order to solve the utility maximization problem of the generic agent of a dynamic game with incomplete ...
- 448
2
votes
0
answers
59
views
Closed form solution to consumption-saving model in discrete time
Consider the simplest consumption-savings model of the following form:
$$
\max_{\{c_t,a_{t+1}\}_t}\mathbb{E_0}\sum_{t\geq 0} \beta^tu(c_t) \\
a_{t+1} + c_t = (1+r)a_t + y_t \\
y_t \mid y_{t-1} \sim F
$...
- 59
4
votes
1
answer
183
views
Dynamic programming in infinite horizon model
Using an infinite horizon model, a dynamic programming approach uses a fixed point to solve the model: $V = \Gamma(V)$.
How do I interpret the meaning of $V$? For example, when we decide a investment ...
- 195
2
votes
0
answers
65
views
Easy introduction to Markov processes and dynamic programming [closed]
I am taking advanced macro course in Fall. Could you please advise me a simple introduction to Markov processes and dynamic programming? I mean easy. Thanks!
- 113
3
votes
2
answers
172
views
Resources to derive economic forecasts
What publicly available resources are there, with sample code, that can be used to build my own macroeconomic model? A search on Github shows that some people have posted code, but I think we can do ...
- 3,761
4
votes
0
answers
41
views
Value function iteration with stochastic productivity's standard deviation
Hello I would like to know how would you discretize the AR(1) process of technology in a standart RBC model when there is stochastic productivity's standard deviation. Namely I have:
Technology $Z_t$ ...
- 437
4
votes
0
answers
53
views
Value function iteration with habit
I would like to know how I could write a value function when there are habits in preferences. I have the following equations:
$$
u\left(C, t, H_{t}, L_{t}\right)=\frac{\left(C_{t} / H_{t}^{\kappa}\...
- 437
2
votes
1
answer
100
views
Exercise 4.7 in SLP (dynamic programming)
Exercise 4.7 (b) : Show that under Assumptions 4.10 and 4.11, $T:H(X) \to H(X)$.
$H(X)$ is the set of continuous and homogeneous of degree one functions and $Tf(x) = \sup_{y \in \Gamma(x)} \{F(x,y) + ...
- 393
0
votes
0
answers
53
views
Cost-optimal p2p-trade in a community of households
I’m trying to solve the following problem and I’ve been working on it for a long time already:
I want to optimize electricity-costs in a smart grid. There’s producer and consumer households in the ...
- 1
2
votes
1
answer
150
views
Proving that the utility is concave
Consider a household which solves the following problem:
$$v(k,r,w)=\underset{c,l\in B{(k,r,ω)}}{\ {max}} \{u(c,l)\}$$
where $u : R_+^2 \rightarrow R$ is a strictly concave, twice continuously ...
- 1,038
2
votes
0
answers
135
views
Cake eating problem
Consider an infinitely lived agent born in time zero, endowed with a cake of size $x_0$. The cake is storable (without depreciation) and infinitely divisible. The agent derives contemporary utility ...
- 1,038
0
votes
0
answers
58
views
Dynamic programming and Difference equations applications
I'm asked by my teacher to prepare a presentation with economic applications of Dynamic Programing (Bellman Equation) and Difference equations. I'm not sure what this things are used for in economics ...
3
votes
1
answer
461
views
A Cake Eating Problem in Continuous Time: Hamiltonian or HJB?
Your standard continuous time cake eating problem is defined as follows:
$$\max_{c(t)}\int_0^\infty e^{-rt} \ln (c(t)) dt$$
subject to
$$f(k(t))=k(t)$$
$$\dot{k}(t)=-c(t)$$
Approaching this problem by ...
- 7,910
2
votes
1
answer
352
views
The Principle of Optimality and the Bellman Equation
Given (1) and (2), is it possible to show the existence of a Bellman equation (3), using Bellman's Principle of Optimality?
$$\ max \Sigma\beta^s U(C_t)$$
Subject to the following resource ...
- 564
4
votes
0
answers
159
views
What are the boundary value conditions for generic HJBs in economics?
Consider a routine continuous time optimization problem:
$
V(t,a_{t}) :=
\max \int_{\tau=t}^{\tau = T}
e^{-\rho (\tau -t)} u(c_{\tau})d\tau
$ $\text{ s.t. }$
$\dot{a}_{t} = y + ra_{t} - c_{t}$,
$a_{...
- 340
3
votes
2
answers
535
views
What is state space representation for DSGE modeling
I'm beginning with DSGE modeling, and a mathematical representation (perhaps trivial for most of the people that are more with this topic) is the space-state state representation of a dynamical model, ...
- 727
5
votes
0
answers
128
views
Optimization in discrete time
I have made optimizations in continuous time that belong to the control theory, for example one case:
$\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$
constraint to: $\dot x=g(t,x(t),u(t))$
Where:
$x(t)$: ...
- 727
2
votes
1
answer
65
views
Can anyone help me understand the Motrtensen-Pissarides model?
I am seeking help with the Mortensen-Pissarides model in discrete time? Basically, I was given the following in class example (which we didn’t complete do to time constraints associated with the end ...
- 87
3
votes
2
answers
202
views
Solving a HJB with a probability to transit to a new state
I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time.
The firm maximizes $V(k)=\int_{t=0}^{\infty}e^{-rt} \pi_t dt$ with $\pi_t=f(k_t)-i_t$ and $\dot{k}...
- 41
3
votes
1
answer
294
views
More than one Bellman Equation
I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation.
How do you realize that? I mean how do you know ...
- 93
1
vote
0
answers
52
views
Coase conjecture and Stokey model
The durable goods monopolist can charge different prices by every periods. Critical type consumers are indifferent between buying today and buying tomorrow. So I construct a Bellman equation like ...
- 23
2
votes
1
answer
134
views
What is the result of the Bellman Equation
I'm just starting with dynamic optimization and although I understant the proof's of the theorem I'm not able to fully understand whether the bellman equation is a function , a function valuated at ...
- 93
2
votes
2
answers
93
views
Why do game theorists use a discounted payoff of this form?
Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form
$$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$
This differs from the discounted ...
- 265
2
votes
0
answers
84
views
Linearization of the dynamic system (I did it, but I have a mistake that I cannot catch. Help me please)
I have the following dynamic system in discrete time
For p is price, d is demand and s is supply.
$$p_{t+1}-p_t= a(d_t-s_t)$$
$$s_{t+1}-s_t=bp_ts_t-ws_t$$
$$d_t= k-gp_t$$
I have to linearize this ...
- 96
2
votes
1
answer
530
views
The Cake Eating Problem with Depreciation (Modelling difficulties)
How does one go about modelling the cake eating problem with depreciation? (i.e The cake goes bad over time)
The problem I have is the following.
Lets define a cake eating problem sequentially as:
...
- 7,910
3
votes
2
answers
433
views
Dynamic programming, optimal consumption-savings (finite horizon) problem
Let $w_t$ denote a consumer's wealth at time $t$ and $c_t$,
the amount she chooses to consume, so her savings exiting this time period are $w_t-c_t$. Given this savings decision, her savings $w_{t+1}$ ...
- 448
1
vote
1
answer
165
views
A profit maximization problem (whole problem has been solved, I just have question about interpretation)
I would like to discuss with you about the following production function.
$$y=f(t_m, t_l)=\rho t_m^m(n+t_l)$$
where $0<m<1 $ and $n>0$ are fixed parameters.
$t_m$ is manager time.
$t_l$ ...
- 96
0
votes
1
answer
169
views
Sequential Price Competition for Perfect Complements
There are two goods, $1$ and $2$ produced by two firms at zero marginal costs. The goods are perfect complements. The demand for each goods is: $Q_1=Q_2=a-(p_1+p_2)$. The prices are set sequentially, ...
- 401
0
votes
3
answers
230
views
What is unknown in Bellman Equation?
\begin{align}
V(W)=\max\limits_{W'\in[0,W]}\qquad& u(W-W')+\beta V(W')\qquad\forall W
\end{align}
$\textbf{My Question}$: Why is the unknown in the Bellman equation $V(W)$ itself? Isn't the ...
- 1,167
2
votes
0
answers
130
views
Why is there a Lagrangian Multiplier in the Dynamic Programming Problem of the RBC model?
Suppose that the household faces the following problem:
$\underset{ c_t , k_{t+1}, n_t } \max \mathbb{E}_0 \sum_{t=0}^{\infty} \beta^t \ln c_t + \ln (1 - n_t)$
subjected to
$ k_{t+1} = A_t k_t ^{\...
- 39
3
votes
2
answers
581
views
In Blackwell's condition for T to be a contraction mapping, we require that satisfies discounting. What is the intuition of discounting?
The discounting condition is as follow:
There exists some $\beta \in (0, 1)$ such that $[T(f + a)](x) ≤ (T f)(x) + βa$, for all $f ∈ B(X), a ≥ 0, x ∈ X$.
While the monotonicity condition makes sense,...
- 119
1
vote
0
answers
296
views
Natural borrowing/debt limit and other borrowing constraints
When confronted with the simple household consumption maximization problem under uncertainty (and with Arrow security sequential trading)
$$\max_{\{c_t(s^t),a_{t+1}(s^t,s_{t+1})\}_{t=0}^{\infty}}\...
- 111
1
vote
0
answers
417
views
Explanation of Dynamic Programming "Guess and Verify" Technique
According to my textbook, the analytical technique for solving a Bellman's Equation is as follows:
Guess a form for $V_0(x)$
Solve the maximization problem with respect to the control and obtain a ...
0
votes
2
answers
121
views
Capital accumulation
I have following dynamic optimization problem
$$V(k_0)=\max \sum_{t=0}^{\infty} b^t((1-a)ln c_t+a ln(l_t))$$
Subject to $l_t+e_t=h$ and $y=Ak_t^pe_t^{1-p}$ and $k_{t+1}=i_t+(1-x)k_t$
Where c is ...
- 63