14 votes
Accepted

When Optimal Control fails (?)

I believe the problem is that the steady state may not exist, and the system instead exhibits steady growth (depending on parameters). The reason is because the model is equivalent to the standard ...
user avatar
  • 1,036
9 votes

References to learn continuous-time dynamic programming

For continuous-time stochastic dynamic programming, the small, nontechnical Art of Smooth Pasting by Dixit is a wonderful option. It does a very effective job of conveying the basic intuition. Stokey'...
user avatar
9 votes

When Optimal Control fails (?)

I am posting this as an answer, because it continues on user @ivansml answer... which is the one that identified the catch here, a catch I naively have overlooked (although it is a narrow case, while ...
user avatar
8 votes
Accepted

Time costs and the St. Petersburg paradox

Consider the version of the paradox from Wikipedia: A casino offers a game of chance for a single player in which a fair coin is tossed at each stage. The pot starts at 2 dollars and is doubled ...
user avatar
  • 16.6k
7 votes
Accepted

In Blackwell's condition for T to be a contraction mapping, we require that satisfies discounting. What is the intuition of discounting?

Without discounting, you cannot show either $$ T(g + || f - g||) \leq Tg + \beta || f - g|| $$ or $$ T(f + || g - f||) \leq Tf + \beta || g - f|| $$ and thus you cannot demonstrate that $T$ is ...
user avatar
  • 1,189
7 votes
Accepted

A Cake Eating Problem in Continuous Time: Hamiltonian or HJB?

The comment by user @MaartenPunt is accurate. I don't think that in general one can identify situations where one should have a clear preference over one formulation over the other. It is more of a ...
user avatar
6 votes

References to learn continuous-time dynamic programming

Dynamic Programming & Optimal Control by Bertsekas Introduction to Modern Economic Growth by Acemoglu The Acemoglu book, even though it specializes in growth theory, does a very good job ...
user avatar
  • 714
6 votes
Accepted

Optimal stopping (reference request)

This is known as the McCall search model in economics. The original paper shows that the optimal stopping strategy rule is given by a "reservation wage", there is a threshold such that it is ...
user avatar
5 votes
Accepted

Prove the uniqueness of steady state

Rearranging the steady state equation $$ \overline{p}^{\alpha}=\alpha y\overline{p}^{\alpha-1}- \alpha\overline{p}^{\alpha}-\frac{a+1}{\sigma} $$ we get $$ (1 + \alpha)\overline{p}^{\alpha}=\alpha y\...
user avatar
  • 26.7k
5 votes
Accepted

Understanding subscripts in first order conditions of dynamic optimization problems

In an intertemporal maximization problem, we seek to find the optimal sequence of the control and the state variables. It is the recursive nature of the problem that permits us to consider a "typical"...
user avatar
5 votes
Accepted

Guess and Verify

Another somewhat canonical form is the value function for risk-sensitive preferences when consumption follows a random walk with drift (there are also versions including capital -- see Backus Ferriere ...
user avatar
  • 1,573
5 votes

When Optimal Control fails (?)

I think that the key question is whether this firm is the only firm in the economy. If it is then it is no longer correct for it to take $w$ as given as $w$ will be affected by its own capital ...
user avatar
5 votes

References to learn continuous-time dynamic programming

I think Kamien and Schwartz's Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management is pretty well known.
user avatar
  • 9,155
5 votes
Accepted

Dynamic programming in infinite horizon model

There are two interrelated maximisation problems. The first is the infinite horizon maximisation problem: $$ \begin{align*} v(k) = &\max_{a_1, a_2, \ldots} \sum_{t = 0}^\infty \delta^t F(k_t, c_t),...
user avatar
  • 8,642
5 votes
Accepted

How can I show convexity of this value function?

Suppose that $u(C,l)=\sqrt{C}-l^2$ and $f(l,A)=\big(l+g(A)\big)^2$, where $g$ is any function of $A$ that is not convex. Then $$u\big(f(l,A),l\big)=l+g(A)-l^2.$$ The optimal labor supply is given by $...
user avatar
4 votes

References to learn continuous-time dynamic programming

Controlled Markov Processes and Viscosity Solutions by Fleming and Soner includes a number of applications to Finance and Differential Games.
user avatar
4 votes
Accepted

Solution Method for Infinite-Horizon Maximization Problem

Your first question (regarding constraints on the parameters) can be answered through first and second derivative analysis. In order to satisfy strictly increasing, we need $u'>0$ and to satisfy ...
user avatar
  • 1,548
4 votes

A reference for most used utility functions in macroeconomic problems of intertemporal optimization

I think that this article might be helpful: “Exotic Preferences for Macroeconomists” http://www.nber.org/chapters/c6672.pdf They give a thorough explanation of many preference functionals and show ...
user avatar
4 votes
Accepted

What is unknown in Bellman Equation?

The original problem was probably of the form $$\max_{\{W_t\}_{t=1}^\infty}\sum_{t=0}^\infty \beta^t u(W_t-W_{t+1}),$$ $$\mbox{s.t. } W_{t+1}\in[0,W_t] \ \forall \ t, ~~ W_0 \mbox{ given}$$ When ...
user avatar
  • 4,158
4 votes

The Principle of Optimality and the Bellman Equation

Recall that the Principle of Optimality states that the solution to Our Bellman Functional Equation is the same as the solution to the sequential problem if: Assumption 1: $\Gamma(x)$ (our set of ...
user avatar
  • 7,715
3 votes

Solution Method for Infinite-Horizon Maximization Problem

Your first question, if it's literally correct, is easy: The only way for $u'$ to be positive for c=0 is for p=1. if p =1 then sign($\phi$)=sign($\theta$) so that the product is positive. But, since $...
user avatar
  • 2,590
3 votes
Accepted

Optimisation using value function

Starting with your original equation: $max_{c_t, m_t, b_t} E_0\sum_{t=0}^\infty U(c_t, m_t)$ s.t. (1) $y+\frac{m_{t-1}}{1+\pi_t}+\frac{1+i_{t-1}}{1+\pi_t}b_{t-1}=c_t+m_t+b_t+\tau_t$ Here: $R_{t-1} ...
user avatar
  • 98
3 votes

References to learn continuous-time dynamic programming

A really nice methodology for approximating the HJB is the upwind scheme, which I learnt quite quickly using Ben Moll et al's notes and codes The examples are continuous time versions of familiar ...
user avatar
  • 10.4k
3 votes

Bellman equation for this dynamic programming problem

The "second" constraint appears redundant and it confuses matters. Re-arrange the first one to obtain $$\tilde{a}_{t+1} = \big[\tilde a_t+(1-\delta )Y_t-\tilde{c}_t\big]R_t$$ This tells us that ...
user avatar
3 votes

Multiple equilibria: which one to select?

I'm not sure I follow the logic on that equation having infinitely many solutions and steady states. In any case, in what follows are some guidelines for equilibrium selection. It depends a lot on ...
user avatar
  • 10.4k
3 votes
Accepted

Dynamic programming, optimal consumption-savings (finite horizon) problem

Your value function is as follows: $$ V_t[w] = \max_{c_t \in[0,w]} \left\{u(c_t) + \frac{1}{2}V_{t+1}[\alpha(w_t - c_t)] + \frac{1}{2}V_{t+1}[\beta(w_t-c_t)] \right\} $$ with the terminal condition $$ ...
user avatar
3 votes
Accepted

The Cake Eating Problem with Depreciation (Modelling difficulties)

It would seem that the way you've formulated your production function/law of motion has introduced double counting into the problem. Note that substituting 1 and 2 into 3 gives: $$k_{t+1}=(1-\delta)(...
user avatar
  • 605
3 votes
Accepted

What is the result of the Bellman Equation

...the result of applying sup operator is a NUMBER... Read it carefully. The equation is $$ v(x_0) = \sup_{ \{x_t \}_{t \geq 1}} \cdots \quad (1) $$ This defines a function $v$, called the value ...
user avatar
  • 2,569
3 votes

Exercise 4.7 in SLP (dynamic programming)

It follows from Assumption 4.10. Let $\lambda y \in \Gamma(\lambda x)$ be the solution. Let $\delta = \frac{1}{\lambda}$. By Assumption 4.10, $$ \lambda y \in\Gamma(\lambda x) $$ implies $$ \delta \...
user avatar
3 votes

Resources to derive economic forecasts

You can find excellent examples of codes for DSGE models as well as VAR on QuantEcon. For example, here is an example of VAR model in Python, and here is an example of some simple DSGE model. The ...
user avatar
  • 43.5k

Only top scored, non community-wiki answers of a minimum length are eligible