12
$\begingroup$

Does anyone know of good references to learn continuous-time dynamic programming? The references don't have to be books. They could be links to online resources as well. Links to clear, concise discussions of even just the basics would be helpful.

Improve this question
$\endgroup$
3
  • $\begingroup$ deterministic, stochastic, or both? depends quite a bit on which. $\endgroup$
    – nominally rigid
    Jan 5, 2015 at 23:08
  • 1
    $\begingroup$ Sorry. Both would be helpful, but I actually do have stochastic in mind. Thanks! $\endgroup$
    – jmbejara
    Jan 5, 2015 at 23:09
  • $\begingroup$ Just thought I'd mention that if we are going to do these types of questions, we should keep with the decided convention of providing 1 recommendation per answer. $\endgroup$
    – cc7768
    Jan 9, 2015 at 18:41

6 Answers 6

Reset to default
9
$\begingroup$

For continuous-time stochastic dynamic programming, the small, nontechnical Art of Smooth Pasting by Dixit is a wonderful option. It does a very effective job of conveying the basic intuition.

Stokey's more recent The Economics of Inaction is also decent, but for a practical-minded person it probably underperforms Dixit - its much greater length and somewhat heavier notation do not yield commensurate rewards.

If the underlying stochastic processes are not Itō diffusions, then I'm not sure what the best reference is. The most common such case I've seen (and that I use myself) is the case of discretely many exogenous states, where if we are currently in state $s$ there is some constant hazard rate $\lambda_{s,s'}$ of a switch to state $s'$. Fortunately, this is a pretty simple case in practice: one can just modify the HJB equation to account for the flow probability of switching from $V(\cdot,s)$ to $V(\cdot,s')$. (You can see this, for instance, in equations (1)-(5) in this Acemoglu and Akcigit paper. Conceptually it's no different from setting up the HJB equation when we have an Itō diffusion as the driving process, except that it's simpler because we just get a system of linear equations and we don't need to think about Itō's lemma, etc.)

Of course, maybe there is a good textbook reference for this too - but unlike the potentially much more complicated cases involving stochastic calculus, this is straightforward enough that a text has never seemed necessary to me.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The Economics of Inaction looks great. I looked a little at it. Given your description of The Art of Smooth Pasting, I'd really like to check it out, but it seems a little hard to get a hold of. Thanks for the recommendations! $\endgroup$
    – jmbejara
    Jan 7, 2015 at 5:27
6
$\begingroup$

Dynamic Programming & Optimal Control by Bertsekas

Introduction to Modern Economic Growth by Acemoglu

The Acemoglu book, even though it specializes in growth theory, does a very good job presenting continuous time dynamic programming.

Improve this answer
$\endgroup$
5
$\begingroup$

I think Kamien and Schwartz's Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management is pretty well known.

Improve this answer
$\endgroup$
1
  • $\begingroup$ It is a very good but note that the book is not really an introduction book for dynamic optimization. $\endgroup$
    – optimal control
    Sep 24, 2015 at 9:09
4
$\begingroup$

Controlled Markov Processes and Viscosity Solutions by Fleming and Soner includes a number of applications to Finance and Differential Games.

Improve this answer
$\endgroup$
3
$\begingroup$

A really nice methodology for approximating the HJB is the upwind scheme, which I learnt quite quickly using Ben Moll et al's notes and codes

The examples are continuous time versions of familiar heterogenous agents economies models such as Hugget and Aiyagari.

Improve this answer
$\endgroup$
2
$\begingroup$

Applied Intertemporal Optimization by Klaus Wälde is a very very nice book, even for those who are not really familiar with mathematics.

The book treats deterministic and stochastic models, both in discrete and continuous time.

I would really say for this book "Dynamic Optimization for dummies". I was not familiar at all with dynamic optimization but this book let me to get through.

Improve this answer
$\endgroup$
2
  • $\begingroup$ This book looks fantastic. I love the organization. The tables at the beginning really illustrate how thorough it is. The book covers most cases of interest and provides many examples. (Apologies for the 4 year delay in responding... :) ) $\endgroup$
    – jmbejara
    Mar 29, 2019 at 16:57
  • 1
    $\begingroup$ Happy that it helps! I learned so much from this book :) $\endgroup$
    – optimal control
    Mar 29, 2019 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.