Questions tagged [dynamic-programming]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

Policy function iteration method in continuous time (with shocks)

Is there any reference available on the algorithm of policy function iteration method in continuous time, when we have uncertainty in the model? Currently, my conclusion is that the combination of ...
user avatar
  • 1
4 votes
0 answers
39 views

What assumptions can be made to ensure convexity in this optimization problem?

This question is a continuation of the question I asked at: How can I show convexity of this value function? Where I came to the conclusion that more assumptions are required to show that the ...
user avatar
4 votes
1 answer
189 views

How can I show convexity of this value function?

I have set up an optmization problem as follows: $$V(A)=\max_{l, C} \quad u(C,l)$$ Where the only constraint is as follows: $$C=f(l,A)$$ Here $u$ is the utility function which captures social welfare. ...
user avatar
6 votes
1 answer
186 views

Optimal stopping (reference request)

I am interested in the following optimal stopping problem: On each day, a number $a_i$ is drawn from a (possibly fixed) distribution. I can either stop now, getting a payoff of $a_i$, or wait for a ...
user avatar
3 votes
2 answers
157 views

How can I formulate the following optimization problem?

I want to set up an optmiization problem for global warming in which a planner determines how much carbon dioxide gas is emitted. Let's say we reduce this problem down to two periods, then I ...
user avatar
1 vote
1 answer
140 views

simplification of FOC

This is the first-order condition of a dynamic programming problem where I am trying to get the Euler equation from a sequential problem. (1) $$\frac{\partial V(d_2)}{\partial d_3} = \frac{-1}{d_2-d_3}...
user avatar
0 votes
0 answers
24 views

How to determine state variables in Value Functions

I've always been slightly confused as to how to determine what is and isn't a state variable in dynamic programming. The mathematics of DP doesn't seem to offer any clues either in my experience. ...
user avatar
  • 199
2 votes
0 answers
34 views

More equations than endogenous variables in RBC model

I recently came across my late father's bachelor's thesis, in which a RBC model is described. It appears to be a variant of Hansen's 1985 model with indivisible labor. I wrote the model into Dynare to ...
user avatar
  • 21
1 vote
0 answers
31 views

How to define a dynamic programming problem in an incomplete information game?

How can I define a problem of dynamic programming, to use the Hamilton-Jacobi-Bellman equation in order to solve the utility maximization problem of the generic agent of a dynamic game with incomplete ...
user avatar
  • 448
0 votes
0 answers
29 views

Mechanism Design and game theory with stochastic caclulus

Does anybody know if there is a literature that combines mechanism design or market games with dynamic programming or stochastic calculus in general? I have heard about stochastic differential game ...
user avatar
2 votes
0 answers
41 views

Closed form solution to consumption-saving model in discrete time

Consider the simplest consumption-savings model of the following form: $$ \max_{\{c_t,a_{t+1}\}_t}\mathbb{E_0}\sum_{t\geq 0} \beta^tu(c_t) \\ a_{t+1} + c_t = (1+r)a_t + y_t \\ y_t \mid y_{t-1} \sim F $...
user avatar
  • 59
4 votes
1 answer
99 views

Dynamic programming in infinite horizon model

Using an infinite horizon model, a dynamic programming approach uses a fixed point to solve the model: $V = \Gamma(V)$. How do I interpret the meaning of $V$? For example, when we decide a investment ...
user avatar
  • 195
2 votes
0 answers
62 views

Easy introduction to Markov processes and dynamic programming [closed]

I am taking advanced macro course in Fall. Could you please advise me a simple introduction to Markov processes and dynamic programming? I mean easy. Thanks!
user avatar
  • 113
3 votes
2 answers
158 views

Resources to derive economic forecasts

What publicly available resources are there, with sample code, that can be used to build my own macroeconomic model? A search on Github shows that some people have posted code, but I think we can do ...
user avatar
  • 3,721
4 votes
0 answers
29 views

Value function iteration with stochastic productivity's standard deviation

Hello I would like to know how would you discretize the AR(1) process of technology in a standart RBC model when there is stochastic productivity's standard deviation. Namely I have: Technology $Z_t$ ...
user avatar
  • 437
4 votes
0 answers
46 views

Value function iteration with habit

I would like to know how I could write a value function when there are habits in preferences. I have the following equations: $$ u\left(C, t, H_{t}, L_{t}\right)=\frac{\left(C_{t} / H_{t}^{\kappa}\...
user avatar
  • 437
2 votes
1 answer
91 views

Exercise 4.7 in SLP (dynamic programming)

Exercise 4.7 (b) : Show that under Assumptions 4.10 and 4.11, $T:H(X) \to H(X)$. $H(X)$ is the set of continuous and homogeneous of degree one functions and $Tf(x) = \sup_{y \in \Gamma(x)} \{F(x,y) + ...
user avatar
  • 393
0 votes
0 answers
53 views

Cost-optimal p2p-trade in a community of households

I’m trying to solve the following problem and I’ve been working on it for a long time already: I want to optimize electricity-costs in a smart grid. There’s producer and consumer households in the ...
user avatar
  • 1
2 votes
1 answer
101 views

Proving that the utility is concave

Consider a household which solves the following problem: $$v(k,r,w)=\underset{c,l\in B{(k,r,ω)}}{\ {max}} \{u(c,l)\}$$ where $u : R_+^2 \rightarrow R$ is a strictly concave, twice continuously ...
user avatar
2 votes
0 answers
71 views

Cake eating problem

Consider an infinitely lived agent born in time zero, endowed with a cake of size $x_0$. The cake is storable (without depreciation) and infinitely divisible. The agent derives contemporary utility ...
user avatar
0 votes
0 answers
44 views

Dynamic programming and Difference equations applications

I'm asked by my teacher to prepare a presentation with economic applications of Dynamic Programing (Bellman Equation) and Difference equations. I'm not sure what this things are used for in economics ...
user avatar
3 votes
1 answer
329 views

A Cake Eating Problem in Continuous Time: Hamiltonian or HJB?

Your standard continuous time cake eating problem is defined as follows: $$\max_{c(t)}\int_0^\infty e^{-rt} \ln (c(t)) dt$$ subject to $$f(k(t))=k(t)$$ $$\dot{k}(t)=-c(t)$$ Approaching this problem by ...
user avatar
  • 7,685
2 votes
1 answer
311 views

The Principle of Optimality and the Bellman Equation

Given (1) and (2), is it possible to show the existence of a Bellman equation (3), using Bellman's Principle of Optimality? $$\ max \Sigma\beta^s U(C_t)$$ Subject to the following resource ...
user avatar
  • 565
3 votes
0 answers
144 views

What are the boundary value conditions for generic HJBs in economics?

Consider a routine continuous time optimization problem: $ V(t,a_{t}) := \max \int_{\tau=t}^{\tau = T} e^{-\rho (\tau -t)} u(c_{\tau})d\tau $ $\text{ s.t. }$ $\dot{a}_{t} = y + ra_{t} - c_{t}$, $a_{...
user avatar
2 votes
2 answers
362 views

What is state space representation for DSGE modeling

I'm beginning with DSGE modeling, and a mathematical representation (perhaps trivial for most of the people that are more with this topic) is the space-state state representation of a dynamical model, ...
user avatar
  • 717
5 votes
0 answers
94 views

Optimization in discrete time

I have made optimizations in continuous time that belong to the control theory, for example one case: $\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$ constraint to: $\dot x=g(t,x(t),u(t))$ Where: $x(t)$: ...
user avatar
  • 717
2 votes
1 answer
58 views

Can anyone help me understand the Motrtensen-Pissarides model?

I am seeking help with the Mortensen-Pissarides model in discrete time? Basically, I was given the following in class example (which we didn’t complete do to time constraints associated with the end ...
user avatar
  • 87
3 votes
2 answers
193 views

Solving a HJB with a probability to transit to a new state

I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time. The firm maximizes $V(k)=\int_{t=0}^{\infty}e^{-rt} \pi_t dt$ with $\pi_t=f(k_t)-i_t$ and $\dot{k}...
user avatar
3 votes
1 answer
201 views

More than one Bellman Equation

I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation. How do you realize that? I mean how do you know ...
user avatar
1 vote
0 answers
42 views

Coase conjecture and Stokey model

The durable goods monopolist can charge different prices by every periods. Critical type consumers are indifferent between buying today and buying tomorrow. So I construct a Bellman equation like ...
user avatar
  • 23
2 votes
1 answer
120 views

What is the result of the Bellman Equation

I'm just starting with dynamic optimization and although I understant the proof's of the theorem I'm not able to fully understand whether the bellman equation is a function , a function valuated at ...
user avatar
2 votes
2 answers
85 views

Why do game theorists use a discounted payoff of this form?

Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form $$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$ This differs from the discounted ...
user avatar
  • 265
2 votes
0 answers
82 views

Linearization of the dynamic system (I did it, but I have a mistake that I cannot catch. Help me please)

I have the following dynamic system in discrete time For p is price, d is demand and s is supply. $$p_{t+1}-p_t= a(d_t-s_t)$$ $$s_{t+1}-s_t=bp_ts_t-ws_t$$ $$d_t= k-gp_t$$ I have to linearize this ...
user avatar
  • 615
2 votes
1 answer
453 views

The Cake Eating Problem with Depreciation (Modelling difficulties)

How does one go about modelling the cake eating problem with depreciation? (i.e The cake goes bad over time) The problem I have is the following. Lets define a cake eating problem sequentially as: ...
user avatar
  • 7,685
3 votes
2 answers
327 views

Dynamic programming, optimal consumption-savings (finite horizon) problem

Let $w_t$ denote a consumer's wealth at time $t$ and $c_t$, the amount she chooses to consume, so her savings exiting this time period are $w_t-c_t$. Given this savings decision, her savings $w_{t+1}$ ...
user avatar
  • 448
1 vote
1 answer
159 views

A profit maximization problem (whole problem has been solved, I just have question about interpretation)

I would like to discuss with you about the following production function. $$y=f(t_m, t_l)=\rho t_m^m(n+t_l)$$ where $0<m<1 $ and $n>0$ are fixed parameters. $t_m$ is manager time. $t_l$ ...
user avatar
  • 615
0 votes
1 answer
133 views

Sequential Price Competition for Perfect Complements

There are two goods, $1$ and $2$ produced by two firms at zero marginal costs. The goods are perfect complements. The demand for each goods is: $Q_1=Q_2=a-(p_1+p_2)$. The prices are set sequentially, ...
user avatar
  • 401
0 votes
3 answers
199 views

What is unknown in Bellman Equation?

\begin{align} V(W)=\max\limits_{W'\in[0,W]}\qquad& u(W-W')+\beta V(W')\qquad\forall W \end{align} $\textbf{My Question}$: Why is the unknown in the Bellman equation $V(W)$ itself? Isn't the ...
user avatar
2 votes
0 answers
95 views

Why is there a Lagrangian Multiplier in the Dynamic Programming Problem of the RBC model?

Suppose that the household faces the following problem: $\underset{ c_t , k_{t+1}, n_t } \max \mathbb{E}_0 \sum_{t=0}^{\infty} \beta^t \ln c_t + \ln (1 - n_t)$ subjected to $ k_{t+1} = A_t k_t ^{\...
user avatar
2 votes
2 answers
441 views

In Blackwell's condition for T to be a contraction mapping, we require that satisfies discounting. What is the intuition of discounting?

The discounting condition is as follow: There exists some $\beta \in (0, 1)$ such that $[T(f + a)](x) ≤ (T f)(x) + βa$, for all $f ∈ B(X), a ≥ 0, x ∈ X$. While the monotonicity condition makes sense,...
user avatar
  • 109
1 vote
0 answers
262 views

Natural borrowing/debt limit and other borrowing constraints

When confronted with the simple household consumption maximization problem under uncertainty (and with Arrow security sequential trading) $$\max_{\{c_t(s^t),a_{t+1}(s^t,s_{t+1})\}_{t=0}^{\infty}}\...
user avatar
  • 111
1 vote
0 answers
312 views

Explanation of Dynamic Programming "Guess and Verify" Technique

According to my textbook, the analytical technique for solving a Bellman's Equation is as follows: Guess a form for $V_0(x)$ Solve the maximization problem with respect to the control and obtain a ...
user avatar
0 votes
2 answers
110 views

Capital accumulation

I have following dynamic optimization problem $$V(k_0)=\max \sum_{t=0}^{\infty} b^t((1-a)ln c_t+a ln(l_t))$$ Subject to $l_t+e_t=h$ and $y=Ak_t^pe_t^{1-p}$ and $k_{t+1}=i_t+(1-x)k_t$ Where c is ...
user avatar
  • 63
3 votes
0 answers
698 views

Market clearing condition with Walras law

I have a diamond overlapping model The question is as follows Let us consider an infinitely lived production economy populated at time t by $N_t$ identical and perfectly competitive adult ...
user avatar
  • 295
6 votes
1 answer
211 views

How do I begin to approach this dynamic discrete choice model?

I'm working through an old problem set (that sadly I don't have solutions for) and I got stuck. It is a dynamic model of entrepreneurship and invention. I'm looking for guidance on this model as well ...
user avatar
  • 161
5 votes
0 answers
211 views

Solution to Dynamic Programming (Bellman Equation) Problem

Could someone please provide pointers on how to solve the below? If any theoretical approximations are possible, that would be very helpful. If numerical solutions are the right approach, could you ...
user avatar
1 vote
0 answers
109 views

Stochastic Dynamic Programming: Deriving the Steady-State for a Lottery

I am working through the basic examples of the stochastic RBC models in the book by McCandless (2008): The ABCs of RBCs, pp. 71 - 75 A Standard Stochastic Dynamic Programming Problem Here is a ...
user avatar
1 vote
1 answer
104 views

A reference for most used utility functions in macroeconomic problems of intertemporal optimization

I'm looking for a reference with the most used utility functions in macroeconomic problems of intertemporal optimization. The reference should preferably include a list of properties of those ...
user avatar
4 votes
1 answer
35 views

Question regarding Carlstrom and Feurst (1997)

I am reading through Carlstrom and Feurst's 1997 paper Agency Costs, Net Worth, and Business Fluctuations: A Computable General Equilibrium Analysis and had a question, although it could apply to ...
user avatar
4 votes
2 answers
172 views

Optimisation using value function

I have the following optimisation problem: max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$ ...
user avatar
  • 641