Skip to main content

Questions tagged [dynamic-programming]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
30 views

Cake eating problem with income over time

I'm trying to implement a solution to a dynamic programmig exercise (cake eating) in Python. Consider the following problem. An individual lives for 20 periods. In the first 15 periods he receives an ...
Calogero Sortino's user avatar
1 vote
1 answer
60 views

What exactly is an exogenous stationary distribution?

All jobs are identical except for their wages, and wages are given by an exogenous stationary distribution of $F (w )$ with finite (bounded) support $\mathbb W$. This is from page 6 of https://ocw....
Maadhav's user avatar
  • 113
1 vote
1 answer
107 views

How to solve dynamic problem with 2 production functions?

Suppose we have the following problem: $\max \int_0^\infty \exp(-\rho t) u(c(t))dt$ where $c(t)$ is consumption at time $t$. Subject to: $\dot{k}(t)= f(k(t))- c(t) - \delta k(t)$. where $k$ is capital,...
WilliamT's user avatar
  • 1,819
3 votes
1 answer
150 views

Has mathematical economics contributed to the mathematics of space exploration?

We see the work of Bellman showing up in both orbital trajectory planning/optimization and, obviously DSGE modeling. Also, in a recent example, the JWST uses ideas surrounding Pareto optimization and ...
whydaho99's user avatar
1 vote
1 answer
87 views

Does this contraction mapping map strictly concave functions into strictly concave functions?

Consider the following functional equation: $$TV(k)=\max[W(k),\beta V(f(k))]$$ where $\beta\in (0,1)$, $W(k)$ is continuous, increasing, bounded, and strictly concave function defined on $[0,\bar{k}]$,...
user avatar
0 votes
2 answers
141 views

How can I show that the policy function is non-decreasing?

Consider the following functional equation: $$V(x)=\max_{y\in [0,f(x)]}[u(f(x)-y)+\beta V(y)]$$ where $u$ is continuous, strictly increasing, and strictly concave; the function $f$ is continuous and ...
Ludwig Gershwin's user avatar
2 votes
1 answer
86 views

Understanding Duality between Individual and Collective Maximization in Macroeconomic Models

I'm currently studying macroeconomic models, specifically from the book "Recursive Macroeconomic Theory." In Chapter Seven, it is mentioned that some economic models involving firms and ...
jrudd's user avatar
  • 155
3 votes
1 answer
247 views

How can I show that the optimal savings are 0 for all time periods?

Consider an infinitely-lived agent’s consumption-saving problem. The agent receives $e > 0$ units of endowment every period, can save via an asset with constant return $R$. The agent is endowed ...
Ludwig Gershwin's user avatar
1 vote
1 answer
38 views

Showing that reward function is bounded (dynamic programing)

I have the following dynamic programming problem: $$\max_{\{x_t,y_{t+1}\}_{t=0}^\infty}\sum_{t=0}^\infty \beta^tu(F(x_t)-y_{t+1})\;\;\;\;\;\text{s.t}\;\;\;\;y_{t+1}\in\Gamma(x_t)$$ where $\Gamma(x)=\{...
manifold's user avatar
  • 843
3 votes
1 answer
99 views

Applying dynamic programming to constrained utility

I am trying to solve problem that looks like this; there is utility function that takes $x$ and $y$ as inputs, $x$ is produced by production function that depends on labor $l+y=1$. $x, y$ depend on $t$...
WilliamT's user avatar
  • 1,819
0 votes
0 answers
24 views

Estimation of learning model with finite horizon forward-looking individuals

I am stuck with the estimation of learning model where individuals are forward-looking in a finite horizon. Specially, a user is watching a TV program containing many episodes, and he doesn't know the ...
S.X's user avatar
  • 11
1 vote
1 answer
68 views

Regarding the arbitrariness of states and controls

I am trying to better understand the process of deriving Euler Equations using the first order condition of the problem on the right hand side of a Bellman equation and the Benveniste-Scheinkman ...
ConsistentlyCaffeinated's user avatar
2 votes
0 answers
126 views

Stochastic control of jumps of random size

Consider the problem of maximizing expected lifetime utility $$ V(a_t) \equiv \max_c\mathrm{E}_t \int_t^\infty e^{\rho (s - t)}u(c_t)\mathrm{d}t $$ subject to a state process $\mathrm{d}a_t$ which is ...
Wittgenstein's Poker's user avatar
0 votes
1 answer
38 views

Technical question about grid setting in dynamic programming models

I already know that expanding the grids of state variables around the steady states works, however in my toy model it's hard to get steady states analytically, so I cannot determine the boundary for ...
Barry's user avatar
  • 23
1 vote
1 answer
255 views

Habit forming Model & State Variables

Taking the habit formation model of consumption, as a standard dynamic programming problem. Bellman Value Function for Habit Model Max$\sum_{t=1}^Tβ^tu(c_t - γc_{t-1})$ $\qquad \qquad (1)$ s.t. $w_{t+...
CormJack's user avatar
  • 981
4 votes
2 answers
1k views

Bellman Equation & Envelope Theorem

I'm unsure where the envelope theorem comes into play when i differentiate the Bellman Equation with respect to $k_t$. To me it looks like the regular chain rule and in fact the exact opposite of the ...
CormJack's user avatar
  • 981
3 votes
1 answer
116 views

Simulation of a dynamic search and matching model

I want to simulate a search and matching model very close to the canonical model developed by Pissarides. I am interested in representing the trajectory of the unemployment rate when the unemployment ...
mayo's user avatar
  • 31
0 votes
0 answers
61 views

One shot-deviation property for games of imperfect information

The equivalence between subgame perfection and one-deviation property is typically stated for games of perfect information (where information sets are singletons). Does the Blackwell-style argument ...
raving-bandit's user avatar
3 votes
0 answers
129 views

Mathematical Prerequisites for Recursive Macroeconomic Theory (Thomas J Sargent, Lars Ljungqvist)

I'm a math grad who is interested in learning more about economics for fun. Reading through RMT, I saw some interesting math (in chapter 2) around using "invariant functions" to determine ...
jrudd's user avatar
  • 155
1 vote
0 answers
45 views

Are there any ``sophisticated'' mathematical modelling where they solve for the utility function?

Are there any references in literature of any ``sophisticated'' mathematical modelling where they solve for the utility function under specific conditions using differential equations theory? In such ...
Oliver Queen's user avatar
2 votes
0 answers
78 views

When do Economists use the Linear Quadratic Regulator in simulating DSGE models

I was watching some excellent videos on DSGE models by Klaus Pretner. The author was able to solve some simple model such as the Ramsey-Cass-Koopman's model, and a New Keynesian model with frictions, ...
krishnab's user avatar
  • 385
1 vote
0 answers
86 views

Advantages of using Bellman equations

Suppose we wish to solve a simple infinite horizon cake eating problem, such that: $$ \max_{\left\{ c_{t}\right\} _{t=0}^{\infty}}\sum_{t}^{\infty}\beta^{t}u\left(c_{t}\right) $$ subject to: $$ c_{t}+...
Kwame Brown's user avatar
3 votes
0 answers
94 views

Current best methods for solving dynamic optimization problems in high dimensional state spaces

I was wondering what the current best methods are for solving dynamic optimization problems in high dimensional state spaces. Let me lay out the common cases where I would do something like this in ...
krishnab's user avatar
  • 385
5 votes
1 answer
199 views

When does it make sense to use variational methods, versus dynamic programming, versus nonlinear control methods so solve DSGE models

I come from a statistics and applied math background, but have been looking at some problems related to macroeconomic DSGE models lately. So I am still trying to understand the ideas and economics ...
krishnab's user avatar
  • 385
4 votes
1 answer
59 views

What assumptions can be made to ensure convexity in this optimization problem?

This question is a continuation of the question I asked at: How can I show convexity of this value function? Where I came to the conclusion that more assumptions are required to show that the ...
L. Johnson's user avatar
4 votes
1 answer
325 views

How can I show convexity of this value function?

I have set up an optmization problem as follows: $$V(A)=\max_{l, C} \quad u(C,l)$$ Where the only constraint is as follows: $$C=f(l,A)$$ Here $u$ is the utility function which captures social welfare. ...
L. Johnson's user avatar
6 votes
1 answer
229 views

Optimal stopping (reference request)

I am interested in the following optimal stopping problem: On each day, a number $a_i$ is drawn from a (possibly fixed) distribution. I can either stop now, getting a payoff of $a_i$, or wait for a ...
afreelunch's user avatar
3 votes
2 answers
204 views

How can I formulate the following optimization problem?

I want to set up an optmiization problem for global warming in which a planner determines how much carbon dioxide gas is emitted. Let's say we reduce this problem down to two periods, then I ...
user avatar
1 vote
1 answer
215 views

simplification of FOC

This is the first-order condition of a dynamic programming problem where I am trying to get the Euler equation from a sequential problem. (1) $$\frac{\partial V(d_2)}{\partial d_3} = \frac{-1}{d_2-d_3}...
econ_confused's user avatar
2 votes
0 answers
64 views

More equations than endogenous variables in RBC model

I recently came across my late father's bachelor's thesis, in which a RBC model is described. It appears to be a variant of Hansen's 1985 model with indivisible labor. I wrote the model into Dynare to ...
DDV's user avatar
  • 21
1 vote
0 answers
43 views

How to define a dynamic programming problem in an incomplete information game?

How can I define a problem of dynamic programming, to use the Hamilton-Jacobi-Bellman equation in order to solve the utility maximization problem of the generic agent of a dynamic game with incomplete ...
Nav89's user avatar
  • 478
2 votes
0 answers
83 views

Closed form solution to consumption-saving model in discrete time

Consider the simplest consumption-savings model of the following form: $$ \max_{\{c_t,a_{t+1}\}_t}\mathbb{E_0}\sum_{t\geq 0} \beta^tu(c_t) \\ a_{t+1} + c_t = (1+r)a_t + y_t \\ y_t \mid y_{t-1} \sim F $...
Jsck's user avatar
  • 59
4 votes
1 answer
366 views

Dynamic programming in infinite horizon model

Using an infinite horizon model, a dynamic programming approach uses a fixed point to solve the model: $V = \Gamma(V)$. How do I interpret the meaning of $V$? For example, when we decide a investment ...
hrkshr's user avatar
  • 195
2 votes
0 answers
70 views

Easy introduction to Markov processes and dynamic programming [closed]

I am taking advanced macro course in Fall. Could you please advise me a simple introduction to Markov processes and dynamic programming? I mean easy. Thanks!
Mr. T's user avatar
  • 113
3 votes
2 answers
199 views

Resources to derive economic forecasts

What publicly available resources are there, with sample code, that can be used to build my own macroeconomic model? A search on Github shows that some people have posted code, but I think we can do ...
Jamzy's user avatar
  • 3,751
4 votes
0 answers
56 views

Value function iteration with stochastic productivity's standard deviation

Hello I would like to know how would you discretize the AR(1) process of technology in a standart RBC model when there is stochastic productivity's standard deviation. Namely I have: Technology $Z_t$ ...
BAL's user avatar
  • 457
4 votes
0 answers
78 views

Value function iteration with habit

I would like to know how I could write a value function when there are habits in preferences. I have the following equations: $$ u\left(C, t, H_{t}, L_{t}\right)=\frac{\left(C_{t} / H_{t}^{\kappa}\...
BAL's user avatar
  • 457
2 votes
1 answer
118 views

Exercise 4.7 in SLP (dynamic programming)

Exercise 4.7 (b) : Show that under Assumptions 4.10 and 4.11, $T:H(X) \to H(X)$. $H(X)$ is the set of continuous and homogeneous of degree one functions and $Tf(x) = \sup_{y \in \Gamma(x)} \{F(x,y) + ...
shk910's user avatar
  • 495
0 votes
0 answers
54 views

Cost-optimal p2p-trade in a community of households

I’m trying to solve the following problem and I’ve been working on it for a long time already: I want to optimize electricity-costs in a smart grid. There’s producer and consumer households in the ...
ElLl's user avatar
  • 1
2 votes
1 answer
240 views

Proving that the utility is concave

Consider a household which solves the following problem: $$v(k,r,w)=\underset{c,l\in B{(k,r,ω)}}{\ {max}} \{u(c,l)\}$$ where $u : R_+^2 \rightarrow R$ is a strictly concave, twice continuously ...
Maybeline Lee's user avatar
2 votes
0 answers
238 views

Cake eating problem

Consider an infinitely lived agent born in time zero, endowed with a cake of size $x_0$. The cake is storable (without depreciation) and infinitely divisible. The agent derives contemporary utility ...
Maybeline Lee's user avatar
0 votes
0 answers
76 views

Dynamic programming and Difference equations applications

I'm asked by my teacher to prepare a presentation with economic applications of Dynamic Programing (Bellman Equation) and Difference equations. I'm not sure what this things are used for in economics ...
Juan Sebastian's user avatar
3 votes
1 answer
682 views

A Cake Eating Problem in Continuous Time: Hamiltonian or HJB?

Your standard continuous time cake eating problem is defined as follows: $$\max_{c(t)}\int_0^\infty e^{-rt} \ln (c(t)) dt$$ subject to $$f(k(t))=k(t)$$ $$\dot{k}(t)=-c(t)$$ Approaching this problem by ...
EconJohn's user avatar
  • 8,405
2 votes
1 answer
430 views

The Principle of Optimality and the Bellman Equation

Given (1) and (2), is it possible to show the existence of a Bellman equation (3), using Bellman's Principle of Optimality? $$\ max \Sigma\beta^s U(C_t)$$ Subject to the following resource ...
EB3112's user avatar
  • 577
4 votes
0 answers
187 views

What are the boundary value conditions for generic HJBs in economics?

Consider a routine continuous time optimization problem: $ V(t,a_{t}) := \max \int_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)} u(c_{\tau})d\tau $ $\text{ s.t. }$ $\dot{a}_{t} = y + ra_{t} - c_{t}$, $a_{...
Albert Zevelev's user avatar
3 votes
2 answers
879 views

What is state space representation for DSGE modeling

I'm beginning with DSGE modeling, and a mathematical representation (perhaps trivial for most of the people that are more with this topic) is the space-state state representation of a dynamical model, ...
manifold's user avatar
  • 843
5 votes
0 answers
163 views

Optimization in discrete time

I have made optimizations in continuous time that belong to the control theory, for example one case: $\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$ constraint to: $\dot x=g(t,x(t),u(t))$ Where: $x(t)$: ...
manifold's user avatar
  • 843
2 votes
1 answer
98 views

Can anyone help me understand the Motrtensen-Pissarides model?

I am seeking help with the Mortensen-Pissarides model in discrete time? Basically, I was given the following in class example (which we didn’t complete do to time constraints associated with the end ...
Tony456's user avatar
  • 87
3 votes
2 answers
227 views

Solving a HJB with a probability to transit to a new state

I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time. The firm maximizes $V(k)=\int_{t=0}^{\infty}e^{-rt} \pi_t dt$ with $\pi_t=f(k_t)-i_t$ and $\dot{k}...
Mr. Fafa's user avatar
4 votes
1 answer
574 views

More than one Bellman Equation

I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation. How do you realize that? I mean how do you know ...
mmendina's user avatar
  • 103