Questions tagged [dynamic-programming]

The tag has no usage guidance.

61 questions
Filter by
Sorted by
Tagged with
692 views

When Optimal Control fails (?)

In order to "ask my question", I have to solve a model first. I will omit some steps but still, this will unavoidably make this post very long -so this is also a test to see whether this community ...
2k views

Solving the Hamilton-Jacobi-Bellman equation; necessary and sufficient for optimality?

Consider the following differential equation \begin{align} \dot x(t)=f(x(t),u(t)) \end{align} where $x$ is the state and $u$ the control variable. The solution is given by \begin{align} x(t)=x_0 + \...
1k views

References to learn continuous-time dynamic programming

Does anyone know of good references to learn continuous-time dynamic programming? The references don't have to be books. They could be links to online resources as well. Links to clear, concise ...
2k views

Guess and Verify

In dynamic programming, the method of undetermined coefficients is sometimes known as "guess and verify." I've periodically heard there are canonical guesses one might make. In particular, I've seen ...
752 views

Solution Method for Infinite-Horizon Maximization Problem

Full disclosure: this problem was part of a final exam that none of our class could really solve definitively. Below the general form is a specific utility function we worked with that I'll try to ...
113 views

How to verify Value Function in nonzero sum two player Differential Game?

There are two agents $i=1,2$. The state $k$ is governed by $\tau_i\in[0,1]$ where \begin{align} \dot{k} = f(k,\tau_1,\tau_2). \end{align} Define the value function of player $i$ by \begin{align} v_i(...
292 views

Time costs and the St. Petersburg paradox

In the St. Petersburg paradox, we end up with the problem that a rational agent should be willing to play the game for any wager, if we look at expected income or utility of expected income. The ...
199 views

Multiple equilibria: which one to select?

There are two agents $i=1,2$. Consider the following programm \begin{align} &V_1(x_0) := \max_u \int^\infty_0 e^{-\rho t}F_1(x(t),u(t),v(t))dt\\ &V_2(x_0) := \max_v \int^\infty_0 e^{-\rho t}...
195 views

How do I begin to approach this dynamic discrete choice model?

I'm working through an old problem set (that sadly I don't have solutions for) and I got stuck. It is a dynamic model of entrepreneurship and invention. I'm looking for guidance on this model as well ...
594 views

One-shot deviation principle for infinite repeated games and dynamic programming

In a context that future return is discounted by a constant parameter, one-shot deviation principle holds for both repeated games and dynamic programming. Because, in repeated games, a one-shot ...
63 views

Optimization in discrete time

I have made optimizations in continuous time that belong to the control theory, for example one case: $\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$ constraint to: $\dot x=g(t,x(t),u(t))$ Where: $x(t)$: ...
164 views

Optimisation using value function

I have the following optimisation problem: max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$ ...
183 views

145 views

More than one Bellman Equation

I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation. How do you realize that? I mean how do you know ...
68 views

Joint Dynamic Programming: Group Activity

Here we have two agents who can spend their time doing some group activity ($h$) or staying at home ($l$). Each agent $i$ is trying to maximize their respective dynamic programming problem: \begin{...
24 views

Value function iteration with stochastic productivity's standard deviation

Hello I would like to know how would you discretize the AR(1) process of technology in a standart RBC model when there is stochastic productivity's standard deviation. Namely I have: Technology $Z_t$ ...
123 views

70 views

Why do game theorists use a discounted payoff of this form?

Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form $$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$ This differs from the discounted ...
344 views

The Cake Eating Problem with Depreciation (Modelling difficulties)

How does one go about modelling the cake eating problem with depreciation? (i.e The cake goes bad over time) The problem I have is the following. Lets define a cake eating problem sequentially as: ...
79 views

Proving that the utility is concave

Consider a household which solves the following problem: $$v(k,r,w)=\underset{c,l\in B{(k,r,ω)}}{\ {max}} \{u(c,l)\}$$ where $u : R_+^2 \rightarrow R$ is a strictly concave, twice continuously ...
52 views

Can anyone help me understand the Motrtensen-Pissarides model?

I am seeking help with the Mortensen-Pissarides model in discrete time? Basically, I was given the following in class example (which we didn’t complete do to time constraints associated with the end ...
133 views

Dynamic programming problem with dimension over 1000

I am working on a project which need to solve a dynamic programming problem with dimension over 1000. In past literature, there exist several methods like Smolyak algorithm and Sparse grid method that ...
544 views

Update of value function in continuous time - HJB

When solving (numerically, by value function iteration) a dynamic programming problem in discrete time, such as $$V_1(a) = \max_{c} \ u(c) + \dfrac{1}{1+\rho}V_0(a')$$ we maximize with respect to ...
54 views

Cake eating problem

Consider an infinitely lived agent born in time zero, endowed with a cake of size $x_0$. The cake is storable (without depreciation) and infinitely divisible. The agent derives contemporary utility ...
79 views

Linearization of the dynamic system (I did it, but I have a mistake that I cannot catch. Help me please)

I have the following dynamic system in discrete time For p is price, d is demand and s is supply. $$p_{t+1}-p_t= a(d_t-s_t)$$ $$s_{t+1}-s_t=bp_ts_t-ws_t$$ $$d_t= k-gp_t$$ I have to linearize this ...
47 views