Questions tagged [optimal-control]
The optimal-control tag has no usage guidance.
43
questions
1
vote
0
answers
37
views
How to add a linear contraint between state variables to a current time Hamiltonian?
Let's say I have an objective function $F$ with state variables $A,B,C$ and relative equations of motion, I can create the current time Hamiltonian with $H_C = F\{A,B,C\}+\alpha * \dot A+\beta * \dot ...
- 307
3
votes
1
answer
53
views
$H$ is a constant? Maximizing: $\int _0^Te^{-t}f(x,u)dt$ st $x_t=g(t,x,u)$ and $g$ is independent of $t$
$\max_{x(t),u(t)}\int _0^Te^{-t}f(x(t),u(t))dt$,
st derivative $x_t=g(t,x(t),u(t))$. Prove that $H$ is constant.
My try2:
consider the Hamiltonian
$$
H(x(t), u(t)) = e^{-t}f(x(t), u(t)) + \lambda(t) g(...
- 259
3
votes
1
answer
95
views
stochastic optimal control/derive the consumption process
(the paper link: p.32, Equation (26), (27))The output process: $d\log y_t = \sigma dZ_t$. The problem is:
$$\max_{C_t, B_t} \mathbb E_0 \int_0^\infty e^{-\rho t} \left(\frac{C_t^{1-v}}{1-v} + (y_t)^{...
- 485
3
votes
1
answer
73
views
stochastic optimal control/FOC/Reis(2021)
Reis (2021) `The constraint on public debt when $r < g$ but $g < m$' :
HJB:
$$\rho V(a, q) = \max_{c/a, k/a} [\log c + V'(a,q)[r + (mq-r)\frac{k}{a} - \frac{c}{a}] a + \frac{V''(a,q)}{2} (k/a)^2 ...
- 485
1
vote
0
answers
46
views
How is the constrained maximization problem in the Ramsey model solved?
Assuming that utility is forward discounted $ \rho $, lifetime utility = $ \int_{0}^{\infty} e^{-\rho t}u(c) \,dt $.
And in continuous time, we also know that $ \dot{k} = y - (\delta + n)k - c$.
...
- 69
3
votes
0
answers
84
views
Current best methods for solving dynamic optimization problems in high dimensional state spaces
I was wondering what the current best methods are for solving dynamic optimization problems in high dimensional state spaces. Let me lay out the common cases where I would do something like this in ...
- 385
5
votes
1
answer
241
views
What are the assumptions made about fixed points in the dynamics equations of Recursive macroeconomics?
I am new to Macroeconomics, but I understand the basics of Recursive Macroeconomic models--following the Ljungqvist and Sargent book. So I get the basic recursive problem to find a vector of ...
- 385
5
votes
1
answer
158
views
When does it make sense to use variational methods, versus dynamic programming, versus nonlinear control methods so solve DSGE models
I come from a statistics and applied math background, but have been looking at some problems related to macroeconomic DSGE models lately. So I am still trying to understand the ideas and economics ...
- 385
2
votes
0
answers
90
views
Optimal Control Problem with Bang-Bang Solution
I am reading Acemoglu, Robinson, Verdier 2017 jpe and stuck at the derivation of the optimal control problem in section IV.D.
The current-value Hamiltonian of the problem is $$H(m(t), u(t), \mu(t))=\...
- 2,339
1
vote
1
answer
67
views
Which industries are widely agreed upon to require central regulation on a national basis, regardless of economic or political system?
For example, the most straightforward cases of widely accepted centrally controlled regulation on a national basis would be the nuclear power generation and nuclear fuel supply industries. Where every ...
- 121
0
votes
0
answers
40
views
Why isn't the Federal Reserve interest rate set by a published algorithm?
The Federal Reserve sets the interest rate, presumably based on a number characteristics and trends in the economy: inflation, GDP growth, population growth, unemployment, housing prices, etc.
However,...
- 511
4
votes
2
answers
457
views
Multiple solutions to an HJB, how to pin down the optimal "viscosity" solution?
Consider the deterministic consumption-savings problem:
$
V(a_t)
=
\underset{c}{\max}
\int_{\tau =t}^{\tau = \infty} e^{-\rho (\tau - t) } u(c_{\tau}) d\tau
$ w/ $u(c)=\frac{c^{1-\gamma}-1}{1-\...
- 340
5
votes
1
answer
141
views
What is the relationship between the HJB and "Hamiltonian"? Why is the Hamiltonian H(p) inside the HJB?
Deterministic Optimal Control Problem
$
V(a_t)
=
\underset{c}{\max}
\int_{\tau =t}^{\tau = \infty} e^{-\rho (\tau - t) } u(c_{\tau}) d\tau
$
s.t.
$
\frac{da}{d\tau} = \left( r a_{\tau} - c_{\tau} \...
- 340
3
votes
1
answer
392
views
Current valued VS Present Valued Hamiltonian Differing Euler equations
I have been having some difficulties with recovering the same euler equation from the following optimal control problem when comparing the present valued hamiltonian to the current valued hamiltonian.
...
- 8,295
4
votes
0
answers
111
views
Solve the Ben-Porath Model (Optimal Control Problem)
Suppose we have a Ben-Porath style human capital investment model, in which the representative agent maximize her lifetime earnings: $$V(h, a)=\max \int_{a}^{R} e^{-r(t-a)}\left[ w h(t)(1-n(t))-px(t)\...
- 2,339
5
votes
2
answers
924
views
Euler equation in Continuous time VS Discrete time
I have seen the euler equation in discrete time for the baseline neoclassical growth model written as:
$$\frac{U'(c_{t+1})}{U'(c_{t})}=\frac{1}{\beta(1+r)}$$
however I have also seen the euler ...
- 8,295
7
votes
1
answer
595
views
Intuition behind Euler Lagrange equation in economics
When being exposed to the concept of the Euler Lagrange equation as a mathematical concept, many ideas from the physical sciences are used to explain its relevance in terms of choice of shortest path, ...
- 8,295
6
votes
1
answer
604
views
Deriving the Euler equation from a Continuous Time Dynamic Programming Problem (HJB)
Solving for the Euler equation in discrete time is farily straight forward with the use of the Benveniste Scheinkman theorem. However for the following standard Ramsey model:
$$\max \int_{0}^{\infty}e^...
- 8,295
4
votes
0
answers
173
views
What are the boundary value conditions for generic HJBs in economics?
Consider a routine continuous time optimization problem:
$
V(t,a_{t}) :=
\max \int_{\tau=t}^{\tau = T}
e^{-\rho (\tau -t)} u(c_{\tau})d\tau
$ $\text{ s.t. }$
$\dot{a}_{t} = y + ra_{t} - c_{t}$,
$a_{...
- 340
3
votes
2
answers
742
views
What is state space representation for DSGE modeling
I'm beginning with DSGE modeling, and a mathematical representation (perhaps trivial for most of the people that are more with this topic) is the space-state state representation of a dynamical model, ...
- 821
5
votes
0
answers
146
views
Optimization in discrete time
I have made optimizations in continuous time that belong to the control theory, for example one case:
$\max(\min)V[u(t)]=\int_0^Tf(t,x(t),u(t))dt$
constraint to: $\dot x=g(t,x(t),u(t))$
Where:
$x(t)$: ...
- 821
8
votes
1
answer
264
views
Continuous time optimization with two laws of motion (the Hamiltonian with two laws of motion)
How would we deal with a continuous time optimal control problem with two laws of motion? Suppose we have the following RCK like environment with human capital investment.
$$\max_{c(t),k(t),h(t)}\int_{...
- 8,295
5
votes
1
answer
3k
views
Current Value Hamiltonian VS Present Value Hamiltonian in Economics
I've been looking at a number of optimal control problems and have been wondering under what conditions one should use the current value Hamiltonian over the present value Hamiltonian.
Does it depend ...
- 8,295
1
vote
0
answers
71
views
Constancy of current value Hamiltonian and numeric computations
I am playing with a simple, continuous time optimal growth problem to learn optimal control. The social planner chooses $c_t$ to maximize:
$$\int_0^\infty e^{-\rho t}u(c_t)dt$$
where
$$\dot k_t=f(k_t)-...
- 11
2
votes
2
answers
106
views
Why do game theorists use a discounted payoff of this form?
Excuse the click-baity title. I notice the discounted payoff in the game theory literature usually takes the form
$$\sum_{t=1}^\infty\lambda(1-\lambda)^{t-1}R_t$$
This differs from the discounted ...
- 265
2
votes
0
answers
71
views
Finding optimal path in continuous time
I have the following optimal control problem
$$\max_{c_t,l_t} \int_0^{\infty} [ln(c_t)+\theta ln(1-l_t)]e^{-pt}dt$$
st. $$\dot{k_t}=k_t^{1/2}l_t^{1/2}-c_t-\beta l_t$$ $$k_0>0$$
I do big part of ...
- 120
4
votes
2
answers
557
views
Dynamic programming, optimal consumption-savings (finite horizon) problem
Let $w_t$ denote a consumer's wealth at time $t$ and $c_t$,
the amount she chooses to consume, so her savings exiting this time period are $w_t-c_t$. Given this savings decision, her savings $w_{t+1}$ ...
- 468
6
votes
1
answer
286
views
Optimal price function: application of calculus of variations
The problem, I am trying to solve is based on the paper by Rochet and Vila 1994 (see literature below). In fact, it is a variant of the seminal paper of Kyle 1985 in the finance/economics literature. ...
- 61
3
votes
1
answer
222
views
Solution method in Smith (2006) A Closed Form Solution to the Ramsey Model
I am trying to understand the way Smith demonstrates that the general solution to his equation (12) is (13) (see page 6).
(12) \begin{eqnarray}
\dot{z} &=& (1- \alpha)\left[1-\left(\delta +...
- 225
2
votes
1
answer
623
views
Optimal control theory: How to maximize Hamiltonian in this case?
The problem is to maximize $\int_0^1 y(t) + u(t)^2 dt$ where $y$ is state and $u$ is control.
Further we have $y' = u, y(0) = 5$.
I set up the Maximum Principle equations, but, in particular, I need ...
- 21
7
votes
1
answer
262
views
An Optimal Control Model: A Ridiculous Result for a Steady State
I was experimenting with a seemingly simple optimal control problem that generates a system of differential equations. When I calculate the values of the steady state of the system I get some very ...
- 133
7
votes
1
answer
357
views
Optimization: Dynamic Programming vs Kuhn-Tucker
Considering the standard utility maximization of representative household which lives forever, one may use dynamic programming and Kuhn-Tucker in case of discrete time. For instance, one would like to ...
- 101
6
votes
4
answers
3k
views
Does control system engineering have a place in economics?
Do central banks use some form of engineering-style PID control systems/feedback loops to implement monetary policy?
I'm an electrical engineering student taking microeconomics/macroeconomics and a ...
- 121
2
votes
1
answer
175
views
Analytically tractable Ramsey model: how to solve ODE for optimal trajectories
In Brunner and Strulik (2002) the authors claim, that the solution of
\begin{align}
\dot c &= \frac{c}{\sigma}(\alpha k^{\alpha-1} - \delta - \rho)\\
\dot k &= k^\alpha - \delta k - c
\end{...
- 1,559
4
votes
1
answer
150
views
Stability analysis and dimension of a dynamic control system
I have an optimal control problem where I have two control and one state variable.
$$max\int_{0}^{\infty}\left(u\left(c\right)-P_{M}M\right)e^{-\rho t}dt\tag{1}$$
where $P_{M}$ is the unit price of ...
- 2,115
7
votes
1
answer
168
views
Simple Derivation of Maximum Principle
Consider the simplest problem of optimal control
\begin{align}
&\max_u\int^T_0{F(y,u)dt}\\
\text{s.t.} \quad&\dot y = f(y,u)\\
& y(0) = y_0\\
& y(T)~~\text{free}
\end{align}
...
- 1,559
2
votes
1
answer
93
views
Optimal Stopping
When we solve Bellman equations, I normally would think of the Blanchard Kahn technique. But in the case that I have an optimal stopping problem, or where the decision that the agent has to take is to ...
- 1,631
8
votes
4
answers
508
views
Autonomous or non-autonomous optimal control system?
I have a following system with endogeneous discounting.
$c,k$ and $h(k)$ are consumption, capital and endogeneous discount function based on physical capital. (the properties of this function are not ...
- 2,115
15
votes
1
answer
578
views
Stochastic growth in continuous time
Literature: See Chang (1988) for theoretical part and Achdou et
al. (2015) for numerical part respectively.
Model
Consider the following stochastic optimal growth problem in per capita notation.
\...
- 1,559
1
vote
0
answers
57
views
discount factor on rival and non-rival goods
I am asking to myself if we must use a different discount rate on rival and non rival goods. In standard Solow-Swann model, we use generally the discount factor $\rho$.
More formally, let's give the ...
- 2,115
3
votes
0
answers
67
views
Converging Trajectories and Sufficiency for Optimality
(The question is loosely relatet to this thread.)
In the paper "Feedback Equilibria for a class of non-linear Differential Games" by Mäler et al. it is stated (p. 14)
In fact sufficiency is ...
- 1,559
20
votes
0
answers
868
views
How do I use the Malliavin calculus to solve for the optimal trading strategy in the classic Merton problem?
How do I use the Malliavin calculus to solve for the optimal trading strategy in the classic Merton problem?
In Duffie's book "Dynamic Asset Pricing," he outlines the "Martingale method" of solving ...
- 9,345
24
votes
3
answers
894
views
When Optimal Control fails (?)
In order to "ask my question", I have to solve a model first. I will omit some steps but still, this will unavoidably make this post very long -so this is also a test to see whether this community ...
- 33.4k