# Questions tagged [dynamic-programming]

The tag has no usage guidance.

61 questions
Filter by
Sorted by
Tagged with
692 views

### When Optimal Control fails (?)

In order to "ask my question", I have to solve a model first. I will omit some steps but still, this will unavoidably make this post very long -so this is also a test to see whether this community ...
2k views

### Solving the Hamilton-Jacobi-Bellman equation; necessary and sufficient for optimality?

Consider the following differential equation \begin{align} \dot x(t)=f(x(t),u(t)) \end{align} where $x$ is the state and $u$ the control variable. The solution is given by \begin{align} x(t)=x_0 + \...
1k views

### References to learn continuous-time dynamic programming

Does anyone know of good references to learn continuous-time dynamic programming? The references don't have to be books. They could be links to online resources as well. Links to clear, concise ...
2k views

### Guess and Verify

In dynamic programming, the method of undetermined coefficients is sometimes known as "guess and verify." I've periodically heard there are canonical guesses one might make. In particular, I've seen ...
752 views

### Solution Method for Infinite-Horizon Maximization Problem

Full disclosure: this problem was part of a final exam that none of our class could really solve definitively. Below the general form is a specific utility function we worked with that I'll try to ...
113 views

### How to verify Value Function in nonzero sum two player Differential Game?

There are two agents $i=1,2$. The state $k$ is governed by $\tau_i\in[0,1]$ where \begin{align} \dot{k} = f(k,\tau_1,\tau_2). \end{align} Define the value function of player $i$ by \begin{align} v_i(...
292 views

### Time costs and the St. Petersburg paradox

In the St. Petersburg paradox, we end up with the problem that a rational agent should be willing to play the game for any wager, if we look at expected income or utility of expected income. The ...
199 views

### Multiple equilibria: which one to select?

There are two agents $i=1,2$. Consider the following programm \begin{align} &V_1(x_0) := \max_u \int^\infty_0 e^{-\rho t}F_1(x(t),u(t),v(t))dt\\ &V_2(x_0) := \max_v \int^\infty_0 e^{-\rho t}...
195 views

### How do I begin to approach this dynamic discrete choice model?

I'm working through an old problem set (that sadly I don't have solutions for) and I got stuck. It is a dynamic model of entrepreneurship and invention. I'm looking for guidance on this model as well ...
594 views

### One-shot deviation principle for infinite repeated games and dynamic programming

In a context that future return is discounted by a constant parameter, one-shot deviation principle holds for both repeated games and dynamic programming. Because, in repeated games, a one-shot ...
63 views

### Optimization in discrete time

I have made optimizations in continuous time that belong to the control theory, for example one case: $\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$ constraint to: $\dot x=g(t,x(t),u(t))$ Where: $x(t)$: ...
164 views

### Optimisation using value function

I have the following optimisation problem: max $E_{0}\sum_{t=0}^{\infty}[log(c_{t}) + log(m_{t})]$ subject to $y + \frac{M_{t-1}}{p_{t}} + R_{t-1}\frac{B_{t-1}}{p_{t}} = c_{t} + m_{t}+b_{t}+\tau_{t}$ ...
183 views

145 views

### More than one Bellman Equation

I'm attending to my first dynamic optimization course, and what I don't fully graps yet is that sometimes we have to use more than one bellman equation. How do you realize that? I mean how do you know ...
68 views

### Joint Dynamic Programming: Group Activity

Here we have two agents who can spend their time doing some group activity ($h$) or staying at home ($l$). Each agent $i$ is trying to maximize their respective dynamic programming problem: \begin{...
24 views

### Value function iteration with stochastic productivity's standard deviation

Hello I would like to know how would you discretize the AR(1) process of technology in a standart RBC model when there is stochastic productivity's standard deviation. Namely I have: Technology $Z_t$ ...
123 views

70 views

### Why do game theorists use a discounted payoff of this form?

Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form $$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$ This differs from the discounted ...
344 views

### The Cake Eating Problem with Depreciation (Modelling difficulties)

How does one go about modelling the cake eating problem with depreciation? (i.e The cake goes bad over time) The problem I have is the following. Lets define a cake eating problem sequentially as: ...
79 views

### Proving that the utility is concave

Consider a household which solves the following problem: $$v(k,r,w)=\underset{c,l\in B{(k,r,ω)}}{\ {max}} \{u(c,l)\}$$ where $u : R_+^2 \rightarrow R$ is a strictly concave, twice continuously ...
52 views

### Can anyone help me understand the Motrtensen-Pissarides model?

I am seeking help with the Mortensen-Pissarides model in discrete time? Basically, I was given the following in class example (which we didn’t complete do to time constraints associated with the end ...
133 views

### Dynamic programming problem with dimension over 1000

I am working on a project which need to solve a dynamic programming problem with dimension over 1000. In past literature, there exist several methods like Smolyak algorithm and Sparse grid method that ...
544 views

### Update of value function in continuous time - HJB

When solving (numerically, by value function iteration) a dynamic programming problem in discrete time, such as $$V_1(a) = \max_{c} \ u(c) + \dfrac{1}{1+\rho}V_0(a')$$ we maximize with respect to ...
54 views

### Cake eating problem

Consider an infinitely lived agent born in time zero, endowed with a cake of size $x_0$. The cake is storable (without depreciation) and infinitely divisible. The agent derives contemporary utility ...
79 views

### Linearization of the dynamic system (I did it, but I have a mistake that I cannot catch. Help me please)

I have the following dynamic system in discrete time For p is price, d is demand and s is supply. $$p_{t+1}-p_t= a(d_t-s_t)$$ $$s_{t+1}-s_t=bp_ts_t-ws_t$$ $$d_t= k-gp_t$$ I have to linearize this ...
47 views