9
Most of the literature on "Strategic experimentation" (or Bandits) uses Poisson processes. Here players can use either a risky or safe arm and one of them generates a fixed stream of payoffs (usually the safe arm) the other one generates lump-sum payments whose arrival times are described by a Poisson process.
Some examples are:
Klein, Nicolas, and Sven ...
9
I am the last person that should be answering continuous time questions like these, but if there's no one else I guess I'll give it a shot. (Any correction of my dimly remembered continuous-time finance is very welcome.)
My impression has always been that this is best interpreted as a consequence of the martingale representation theorem. First, though, I'll ...
8
For continuous-time stochastic dynamic programming, the small, nontechnical Art of Smooth Pasting by Dixit is a wonderful option. It does a very effective job of conveying the basic intuition.
Stokey's more recent The Economics of Inaction is also decent, but for a practical-minded person it probably underperforms Dixit - its much greater length and ...
5
You can separate your function in three terms by writing
\begin{align}
& v(c_{t+\Delta},u_{t+\Delta},t+\Delta)-v(c_t,u_t,t) = \\
& v(c_{t+\Delta},u_{t+\Delta},t+\Delta)-v(c_t,u_{t+\Delta},t+\Delta) \\
+ & v(c_{t},u_{t+\Delta},t+\Delta)-v(c_t,u_t,t+\Delta) \\
+ & v(c_t,u_t,t+\Delta)-v(c_t,u_t,t)
\end{align}
When you divide by $\Delta$ and take ...
5
Dynamic Programming & Optimal Control by Bertsekas
Introduction to Modern Economic Growth by Acemoglu
The Acemoglu book, even though it specializes in growth theory, does a very good job presenting continuous time dynamic programming.
5
I've been meaning to post this for a long time. I came across this and thought it could add some insight. This example is from "Financial Asset Pricing Theory" by Munk.
Consider the following figure. How many assets do we need to have a complete market?
You might think that, because there are 6 different states here, we need at least 6 different ...
4
I think Kamien and Schwartz's Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management is pretty well known.
4
Here's my best guess. I haven't checked to thoroughly if this is right, but maybe it will help.
Evolution of population density
I understand the model as follows. $f(a,t)$
is the density of people of age $a$ at time $t$. Suppose at time $t=0$, the density of the population is $f_0(a)$. To model the aging process as well as the mortality rate, the density $...
3
Klette and Kortum (2004) develop a parsimonious model of innovating firms rich enough to confront firm-level evidence. It captures the dynamics of individual heterogenous firms, describes the behavior of an industry with simultaneous entry and exit, and delivers ageneral equilibrium model of technological change. At the root of the model is a Poisson process ...
3
A really nice methodology for approximating the HJB is the upwind scheme, which I learnt quite quickly using Ben Moll et al's notes and codes
The examples are continuous time versions of familiar heterogenous agents economies models such as Hugget and Aiyagari.
3
Controlled Markov Processes and Viscosity Solutions by Fleming and Soner includes a number of applications to Finance and Differential Games.
2
$\newcommand{\R}{\mathbb{R}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\F}{\mathbb{F}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\E}{\mathbb{E}}
%short command for inseting abbreviated "such that" in a math environment
\newcommand{\st}{\text{ s.t. }}
%text in a math environment "as"
\newcommand{\as}{\text{ as }}
%various referencing commands
\newcommand{\rref}[1]...
2
From time $0$ to time $t$ the profit flow is constant $\Delta$. The discounted value of the profit flow of any instance $s$ is $\Delta \cdot e^{-r s}$. To get the total discounted profit we will need the integral of this from $0$ to $t$:
$$
\int_0^t \Delta \cdot e^{-r s} \ ds = \left. \frac{\Delta \cdot e^{-r s}}{-r} \right]_0^t = \frac{\Delta \cdot e^{-r t}}...
2
I think the step
"...where $P(a, t) = exp(-d(a,t)\Delta)$ is the discrete time analogue of $d(a,t)$..."
is the problem.
In continuous time I guess we have
$$\dot m(a,t) = -d(a,t)m(a,t) \implies m(a,t) = m_0\exp \{-d(a,t)t\}$$
We then have, discretizing,
$$\frac {m(a,t+\Delta) - m(a,t)}{m(a,t)} = \frac {\exp \{-d(a,t)(t+\Delta)\}-\exp \{-d(a,t)t\}}{\...
2
In the model with technological progress the capital per effective worker remains constant, implies that capital per worker grows at the rate of exogenous rate of technological progress. See Barro and Martin book, Chapter 1.
2
I would leave this as a comment but I cant. You are on the right track.
Once you know $V_2(k)$ then you can plug that into to the first hjb and solve.
To solve for $V_2$ you need to find the optimal $i$ as a function of $k$. Then plug $i(k)$ into the 2nd HJB. That will give you a second order ode. Solving that will give you $V_2(k)$ and you go to 1.
1
The paper is not trying to say that equation (10) is derived from equation (8). Equation (8) tells us how household makes its optimal consumption and 'saving' decision (it gives us demand for investing the wealth into bonds/risky assets).
The equation (10) then tells us that given the household optimal decisions (which depend on utility (8)), those ...
1
More of a comment:
There should be an expectation operator in the statement of the problem, otherwise problem doesn't make sense.
That "...the deterministic and stochastic value function must be the same..." is not quite right. The value of $\sigma^2$ is crucial in the restriction
\begin{align}
\rho = \left(-n + \sigma^2\left(1 - \frac{\alpha\gamma}{2}\...
1
I will try to answer to your last question. I did not read the paper but in models with higher dimensions, it is always difficult to find an analytical solution.
If there exists an analytical solution (for a very basic model with a one-state variable), it is possible to derive the initial conditions for your control and state variable from your differential ...
1
Yes, it is correct. You can (for instance) write a Taylor expansion:
\begin{align*}
[1-(1-\frac{x}{un})^n]^x & = [1-e^{n ln(1-\frac{x}{un})}]^x \\
& = [1-e^{n (-\frac{x}{un} + o(\frac{1}{n}))}]^x \\
& = [1-e^{-\frac{x}{u} + o(1)}]^x \\
& \sim [1-e^{-\frac{x}{u}}]^x \text{ when } n \rightarrow +\infty
\end{align*}
1
An alternative approximating approach you could use as as check might be to say there are $X$ job offers in total and $u$ unemployed.
So the probability that an individual does not get a particular job offer is $\left(1-\dfrac{1}{u}\right)$ and so the probability the individual does not get any job offer is $\left(1-\dfrac{1}{u}\right)^X$ which is $\left(\...
1
You iterate towards a fixed point, so you want to reach a situation where plugging in your current iterated value produces itself. Now using your notation, we are told that we should calculate
$$V_{n+1}(a) = V_n(a) + \Delta$$
where
$$\Delta = \ u(c(a^*)) + \dfrac{\partial V_n(a)}{\partial a}da_t(a^*) - \rho V_n(a)$$
Insert the second into the first to ...
1
Applied Intertemporal Optimization by Klaus Wälde is a very very nice book, even for those who are not really familiar with mathematics.
The book treats deterministic and stochastic models, both in discrete and continuous time.
I would really say for this book "Dynamic Optimization for dummies". I was not familiar at all with dynamic optimization but this ...
Only top voted, non community-wiki answers of a minimum length are eligible
