Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [dynamic-programming]

The tag has no usage guidance.

1
vote
1answer
61 views

In Blackwell's condition for T to be a contraction mapping, we require that satisfies discounting. What is the intuition of discounting?

The discounting condition is as follow: There exists some $\beta \in (0, 1)$ such that $[T(f + a)](x) ≤ (T f)(x) + βa$, for all $f ∈ B(X), a ≥ 0, x ∈ X$. While the monotonicity condition makes sense,...
1
vote
0answers
56 views

Natural borrowing/debt limit and other borrowing constraints

When confronted with the simple household consumption maximization problem under uncertainty (and with Arrow security sequential trading) $$\max_{\{c_t(s^t),a_{t+1}(s^t,s_{t+1})\}_{t=0}^{\infty}}\...
1
vote
0answers
144 views

Explanation of Dynamic Programming “Guess and Verify” Technique

According to my textbook, the analytical technique for solving a Bellman's Equation is as follows: Guess a form for $V_0(x)$ Solve the maximization problem with respect to the control and obtain a ...
0
votes
1answer
57 views

Capital accumulation

I have following dynamic optimization problem $$V(k_0)=\max \sum_{t=0}^{\infty} b^t((1-a)ln c_t+a ln(l_t))$$ Subject to $l_t+e_t=h$ and $y=Ak_t^pe_t^{1-p}$ and $k_{t+1}=i_t+(1-x)k_t$ ...
3
votes
0answers
197 views

Market clearing condition with Walras law

I have a diamond overlapping model The question is as follows Let us consider an infinitely lived production economy populated at time t by $N_t$ identical and perfectly competitive adult ...
6
votes
1answer
140 views

How do I begin to approach this dynamic discrete choice model?

I'm working through an old problem set (that sadly I don't have solutions for) and I got stuck. It is a dynamic model of entrepreneurship and invention. I'm looking for guidance on this model as well ...
2
votes
0answers
111 views

Solution to Dynamic Programming (Bellman Equation) Problem

Could someone please provide pointers on how to solve the below? If any theoretical approximations are possible, that would be very helpful. If numerical solutions are the right approach, could you ...
1
vote
0answers
46 views

Stochastic Dynamic Programming: Deriving the Steady-State for a Lottery

I am working through the basic examples of the stochastic RBC models in the book by McCandless (2008): The ABCs of RBCs, pp. 71 - 75 A Standard Stochastic Dynamic Programming Problem Here is a ...
1
vote
1answer
61 views

A reference for most used utility functions in macroeconomic problems of intertemporal optimization

I'm looking for a reference with the most used utility functions in macroeconomic problems of intertemporal optimization. The reference should preferably include a list of properties of those ...
4
votes
1answer
30 views

Question regarding Carlstrom and Feurst (1997)

I am reading through Carlstrom and Feurst's 1997 paper Agency Costs, Net Worth, and Business Fluctuations: A Computable General Equilibrium Analysis and had a question, although it could apply to ...
4
votes
2answers
150 views

Optimisation using value function

I have the following optimisation problem: max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$ ...
1
vote
1answer
112 views

Dynamic programming with housing consumption and labor

I try to solve the following maximization problem of a representative household with dynamic programming. However, my last result is not similar to the solution. Could any one help me? $$\max\...
2
votes
1answer
527 views

Bellman equation for this dynamic programming problem

For the following problem \begin{equation}\max_{(\tilde{c}_t,\tilde{a}_{t+1+s})}\sum_{s=0}^{\infty}\beta ^su(\tilde{c}_{t+s})\end{equation} s.t. the following restrictions $\begin{equation}...
4
votes
1answer
1k views

Solution to the Bellman equation is a fixed point

I have recently started studying dynamic optimization. I cannot quite wrap my head around the fact that the value function of the Bellman equation is a fixed point of a contraction mapping. As far my ...
1
vote
1answer
87 views

How to deal with Prescott's formulation of time to build in his original RBC model?

So I was replicating the results obtained in section 4 of Prescott's original paper, which derives optimality conditions in steady state without shock. I hope to solve the social planner's ...
1
vote
0answers
112 views

How to solve a variation of Merton's optimal portfolio problem?

Does anyone know how to solve the following problem? I have tried to solve this but I'm lost since I have never dealt with a Stochastic Dynamic Programming problem with many variables. $max_{c_{t},\...
1
vote
0answers
47 views

Hamilton-Jacobi-Bellman with heterogeneous discount rates

Let $i=1,2$ denote the players, $x$ the state and $u_i$ the control of player $i$. The state equation reads $\dot x = f(x,u_1,u_2)$ and the objective function is given by $F_i(x,u_1,u_2)$. Now I'd ...
2
votes
1answer
111 views

Dynamic programming problem with dimension over 1000

I am working on a project which need to solve a dynamic programming problem with dimension over 1000. In past literature, there exist several methods like Smolyak algorithm and Sparse grid method that ...
5
votes
3answers
483 views

Solution Method for Infinite-Horizon Maximization Problem

Full disclosure: this problem was part of a final exam that none of our class could really solve definitively. Below the general form is a specific utility function we worked with that I'll try to ...
1
vote
1answer
301 views

Update of value function in continuous time - HJB

When solving (numerically, by value function iteration) a dynamic programming problem in discrete time, such as $$V_1(a) = \max_{c} \ u(c) + \dfrac{1}{1+\rho}V_0(a')$$ we maximize with respect to ...
4
votes
1answer
55 views

Joint Dynamic Programming: Group Activity

Here we have two agents who can spend their time doing some group activity ($h$) or staying at home ($l$). Each agent $i$ is trying to maximize their respective dynamic programming problem: \begin{...
2
votes
1answer
171 views

Prove the uniqueness of steady state

I have a difference equation $$ p_t^{1-\alpha}=\alpha\sigma \left(y-p_t-\frac{(\sigma p_{t-1}^\alpha+b)p_t^{1-\alpha}}{\alpha\sigma} \right) $$ where $\alpha \in [0,1]$ and everything else is $&...
1
vote
0answers
86 views

Dynamic programming: verification principle

Consider the Gale cake problem with $u(c) = log(c)$, so that the problem becomes:$$\max_{x}\sum_{t=0}^{\infty} \beta^{t}log(x_t-x_{t+1})\: sub\: 0<x_{t+1}\leq x_t,\: x_0>0\ given$$ It is ...
3
votes
1answer
1k views

Benveniste-Scheinkman condition gives derivative that still depends on the value function

What I mean by the title is often, if we have a value function like $$V(K,I) = \max_{K',I'} F(K') +\beta V(K',I')$$ the First order conditions will give us something that depends on the derivative of ...
2
votes
0answers
93 views

Taking limit of a sequence

I am given a following dynamic programming problem; $$ \sup_{k_{t+1}}\sum_{t=0}^{t=\infty}\delta^t(ak_t-\frac{b}{2}k_t^2-\frac{c}{2}(k_{t+1}-k_t)^2) $$ where $f(k_t) = ak_t-\frac{b}{2}k_t^2$ is the ...
3
votes
1answer
72 views

Conjecture Steady State from limit properties

The question is related to this thread. I'd like to derive a unique steady state for an optimal control problem. Consider the following programm \begin{align} &V(x_0) := \max_u \int^\infty_0 e^{-\...
8
votes
1answer
1k views

Guess and Verify

In dynamic programming, the method of undetermined coefficients is sometimes known as "guess and verify." I've periodically heard there are canonical guesses one might make. In particular, I've seen ...
13
votes
1answer
898 views

Solving the Hamilton-Jacobi-Bellman equation; necessary and sufficient for optimality?

Consider the following differential equation \begin{align} \dot x(t)=f(x(t),u(t)) \end{align} where $x$ is the state and $u$ the control variable. The solution is given by \begin{align} x(t)=x_0 + \...
5
votes
2answers
153 views

Multiple equilibria: which one to select?

There are two agents $i=1,2$. Consider the following programm \begin{align} &V_1(x_0) := \max_u \int^\infty_0 e^{-\rho t}F_1(x(t),u(t),v(t))dt\\ &V_2(x_0) := \max_v \int^\infty_0 e^{-\rho t}...
6
votes
1answer
99 views

How to verify Value Function in nonzero sum two player Differential Game?

There are two agents $i=1,2$. The state $k$ is governed by $\tau_i\in[0,1]$ where \begin{align} \dot{k} = f(k,\tau_1,\tau_2). \end{align} Define the value function of player $i$ by \begin{align} v_i(...
2
votes
2answers
281 views

Understanding subscripts in first order conditions of dynamic optimization problems

Suppose we have a simple maximization problem as described in Equation 1.1 here or here. This leads us to the Lagrangian Equation 1.3: $$\begin{align*}\mathcal{L} &= \sum_{t=1}^\infty \beta^{t-1}\...
5
votes
2answers
490 views

One-shot deviation principle for infinite repeated games and dynamic programming

In a context that future return is discounted by a constant parameter, one-shot deviation principle holds for both repeated games and dynamic programming. Because, in repeated games, a one-shot ...
10
votes
6answers
896 views

References to learn continuous-time dynamic programming

Does anyone know of good references to learn continuous-time dynamic programming? The references don't have to be books. They could be links to online resources as well. Links to clear, concise ...
15
votes
3answers
469 views

When Optimal Control fails (?)

In order to "ask my question", I have to solve a model first. I will omit some steps but still, this will unavoidably make this post very long -so this is also a test to see whether this community ...
6
votes
1answer
231 views

Time costs and the St. Petersburg paradox

In the St. Petersburg paradox, we end up with the problem that a rational agent should be willing to play the game for any wager, if we look at expected income or utility of expected income. The ...