9

I believe the problem is that the steady state may not exist, and the system instead exhibits steady growth (depending on parameters). The reason is because the model is equivalent to the standard consumption-saving problem with exogenous and constant interest rate. To see that, first consider the first order condition for labor choice $f_2(k,\ell) = w$ (...


8

For continuous-time stochastic dynamic programming, the small, nontechnical Art of Smooth Pasting by Dixit is a wonderful option. It does a very effective job of conveying the basic intuition. Stokey's more recent The Economics of Inaction is also decent, but for a practical-minded person it probably underperforms Dixit - its much greater length and ...


8

Consider the version of the paradox from Wikipedia: A casino offers a game of chance for a single player in which a fair coin is tossed at each stage. The pot starts at 2 dollars and is doubled every time a head appears. The first time a tail appears, the game ends and the player wins whatever is in the pot. Thus the player wins 2 dollars if a tail ...


6

I am posting this as an answer, because it continues on user @ivansml answer... which is the one that identified the catch here, a catch I naively have overlooked (although it is a narrow case, while the interesting par comes after. Nevertheless, it should have been dealt with). Indeed, with exogenous wage rate, and perfectly competitive optimization on ...


5

Dynamic Programming & Optimal Control by Bertsekas Introduction to Modern Economic Growth by Acemoglu The Acemoglu book, even though it specializes in growth theory, does a very good job presenting continuous time dynamic programming.


5

In an intertemporal maximization problem, we seek to find the optimal sequence of the control and the state variables. It is the recursive nature of the problem that permits us to consider a "typical" point in time and just one condition per variable. For each such problem, we need to find out (carefully) in how many distinct periods a specific ...


4

Another somewhat canonical form is the value function for risk-sensitive preferences when consumption follows a random walk with drift (there are also versions including capital -- see Backus Ferriere Zin 2014). $$c_t = \mu + c_{t-1} + \sigma_c \varepsilon_{t}$$ Begin with preferences given as Epstein-Zin with a certainty equivalence function of the form $\...


4

I think that the key question is whether this firm is the only firm in the economy. If it is then it is no longer correct for it to take $w$ as given as $w$ will be affected by its own capital accumulation decision. In this case you should make the substitutions that you made before your equation (2) while setting up the Hamiltonian. On the other hand if ...


4

I think Kamien and Schwartz's Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management is pretty well known.


4

Rearranging the steady state equation $$ \overline{p}^{\alpha}=\alpha y\overline{p}^{\alpha-1}- \alpha\overline{p}^{\alpha}-\frac{a+1}{\sigma} $$ we get $$ (1 + \alpha)\overline{p}^{\alpha}=\alpha y\overline{p}^{\alpha-1}- \frac{a+1}{\sigma}. $$ As $\alpha \in [0,1]$, the left hand side of the equation is increasing in $\overline{p}$ and the right hand side ...


4

I think that this article might be helpful: “Exotic Preferences for Macroeconomists” http://www.nber.org/chapters/c6672.pdf They give a thorough explanation of many preference functionals and show their usefulness with macroeconomic applications. Im not a specialist in macroeconomics, so I dont know the literature that well, but I think that the most ...


4

The original problem was probably of the form $$\max_{\{W_t\}_{t=1}^\infty}\sum_{t=0}^\infty \beta^t u(W_t-W_{t+1}),$$ $$\mbox{s.t. } W_{t+1}\in[0,W_t] \ \forall \ t, ~~ W_0 \mbox{ given}$$ When formulated that way there is no need to solve for any function $V$, but the infinite elements of the sequence ${\{W_t\}}_{t=1}^\infty$ are all choice variables. ...


3

Starting with your original equation: $max_{c_t, m_t, b_t} E_0\sum_{t=0}^\infty U(c_t, m_t)$ s.t. (1) $y+\frac{m_{t-1}}{1+\pi_t}+\frac{1+i_{t-1}}{1+\pi_t}b_{t-1}=c_t+m_t+b_t+\tau_t$ Here: $R_{t-1} =1+i_{t-1}$ and $1+\pi_t=\frac{P_t}{P_{t-1}}$ Note that in this problem, you have have two state variables, $m_{t-1}$ and $b_{t-1}$, and your main issue have ...


3

The "second" constraint appears redundant and it confuses matters. Re-arrange the first one to obtain $$\tilde{a}_{t+1} = \big[\tilde a_t+(1-\delta )Y_t-\tilde{c}_t\big]R_t$$ This tells us that wealth in the beginning of next period is fully determined by current-period decisions and known states, without any uncertainty whatsoever: we start with the ...


3

Your first question (regarding constraints on the parameters) can be answered through first and second derivative analysis. In order to satisfy strictly increasing, we need $u'>0$ and to satisfy strictly concave, we need $u''<0$. What does this actually mean? $$u'(\frac{}{})=\phi p\theta c_t^{p-1}e^{\theta c_t^p}>0$$ Since we know that $c_t^{p-1}e^{...


3

A really nice methodology for approximating the HJB is the upwind scheme, which I learnt quite quickly using Ben Moll et al's notes and codes The examples are continuous time versions of familiar heterogenous agents economies models such as Hugget and Aiyagari.


3

Controlled Markov Processes and Viscosity Solutions by Fleming and Soner includes a number of applications to Finance and Differential Games.


3

I'm not sure I follow the logic on that equation having infinitely many solutions and steady states. In any case, in what follows are some guidelines for equilibrium selection. It depends a lot on the context. Here are some criteria, in descending order of relevance Clean Ways Is any of the steady states unstable? If so, it's less likely to be the one ...


3

Without discounting, you cannot show either $$ T(g + || f - g||) \leq Tg + \beta || f - g|| $$ or $$ T(f + || g - f||) \leq Tf + \beta || g - f|| $$ and thus you cannot demonstrate that $T$ is a contraction mapping in the traditional proof. For intuition, note that Blackwell's sufficiency conditions for a contraction mapping is a fixed-point theorem ...


3

It would seem that the way you've formulated your production function/law of motion has introduced double counting into the problem. Note that substituting 1 and 2 into 3 gives: $$k_{t+1}=(1-\delta)(c_t+x_t)+x_t$$ Where investment in period t is counted twice. The correct law of motion is simply: $$ 3. \: k_{t+1}=(1-\delta)x_t $$ And the general form of ...


2

There is an old result in dynamic programming due to David Blackwell, according to which stationary problems allow for stationary best responses. So if you would gain by changing your behavior after a certain history, you would gain by changing it at every history corresponding to the same state. For the original reference, see the corollary to Theorem 1 ...


2

A deviation (one-shot or not) can certainly generate a sequence that differs from the optimal one for an arbitrary number of periods. You could treat a dynamic programming problem as a repeated game between one player and chance. The one-shot deviation principle should then carry over from repeated games to dynamic programming.


2

Your first question, if it's literally correct, is easy: The only way for $u'$ to be positive for c=0 is for p=1. if p =1 then sign($\phi$)=sign($\theta$) so that the product is positive. But, since $exp()>0$ and $p=1$, the first term of $u''$ cancels out and the only way for u'' to be negative is for $\phi$ to be negative. Therefore $\theta<0$ too. ...


2

Thanks to @Boaten I was able to find the solution. For those interested here are the steps for deriving the Fisher relation: Combining the FOC $[b_{t}]$ and the envelope for $[b_{t-1}]$ we get $U_{c_{t}} = \beta E_{t}\frac{U_{c_{t+1}}(1+i_{t})}{1+\pi _{t+1}}$ Assuming utility is log in both arguments this can be simplified as $\frac{c_{t+1}}{c_{t}} = \...


2

What you address (and what is done in your referred paper) is called Model Calibration Many economic models use this method to determine one or more parameters: Calibration is a strategy for finding numerical values for the parameters of artificial economic worlds. As the widespread use of this technique, let me explain it with a universal example and ...


2

Your value function is as follows: $$ V_t[w] = \max_{c_t \in[0,w]} \left\{u(c_t) + \frac{1}{2}V_{t+1}[\alpha(w_t - c_t)] + \frac{1}{2}V_{t+1}[\beta(w_t-c_t)] \right\} $$ with the terminal condition $$ V_{T}[w_T] = \max_{c_T \in [0,w_T]} u(c_T) $$ So, we can solve this via backward induction. Clearly, at the final period $T$, since $u$ is monotonic, we ...


2

In my experience, it's mainly just for cleanliness for results. Consider an infinite horizon repeated game, with discounted payoff representation (where I use $\delta = (1-\lambda)$ in your notation) $$ (1-\delta)\sum_{t=0}^{\infty}\delta^t R_t $$ where $0 < \delta < 1$. Suppose I play a strategy that gives me the same payoff, say $a$, for each ...


2

...the result of applying sup operator is a NUMBER... Read it carefully. The equation is $$ v(x_0) = \sup_{ \{x_t \}_{t \geq 1}} \cdots \quad (1) $$ This defines a function $v$, called the value function, which is a type of indirect utility function. For given $x = x_0$, the value of $v(x)$ is defined to be the sup on the RHS, taken over feasible ...


1

$$V(b_{t-1})=\max\limits_{C_{t},H_t,N_t} \Bigg\{\ln C_t+j\ln H_t-\dfrac{(N_t)^\eta}{\eta} +\beta E_tV(b_t) \Bigg\}$$ so $$\beta E_tV(b_{t})=\beta E_t\left[\max\limits_{C_{t+1},H_{t+1},N_{t+1}} \Bigg\{\ln C_{t+1}+j\ln H_{t+1}-\dfrac{(N_{t+1})^\eta}{\eta} +\beta E_{t+1}V(b_{t+1}) \Bigg\}\right]$$ But forwarding the budget constraint $$C_t + b_t +q_t(H_t-...


1

Upgrading an exchange of comments, a critical point in the question is the expression \begin{align} \frac{\partial u(\tilde x)}{\partial \rho}=\frac{1}{(\rho + u'(\tilde x) + 1)^2} \end{align} which is mistaken , because from \begin{align} u(\tilde x) = \frac{\rho + u'(\tilde x)}{\rho + u'(\tilde x) + 1}. \end{align} we obtain \begin{align} \frac{\...


Only top voted, non community-wiki answers of a minimum length are eligible