When does it make sense to use variational methods, versus dynamic programming, versus nonlinear control methods so solve DSGE models

Question

I come from a statistics and applied math background, but have been looking at some problems related to macroeconomic DSGE models lately. So I am still trying to understand the ideas and economics concepts, but I have a pretty good grasp of linear and nonlinear optimal control methods.

My fundamental question is when do economists use variational methods to solve DSGE models, versus using dynamic programming or linear optimal control methods, versus using nonlinear optimal control methods? It seems like economists use all of the above methods, but I was not sure when and why they choose one method over the other.

I was looking at some videos that present a nice simple DSGE RBC model, and solve it using variational methods--meaning the Lagrange multiplier method. The solution is analytical and derived first order conditions for the system like I learned as an undergraduate. The basic problem they are working on is the infinite and finite time horizon optimization problem:

$$ max_{c, h} \quad \mathbb{E}(\sum^{\infty}_{t=0} \beta^tu(c_t, h_t)) $$

In this case, $c_t$ is the consumption at time $t$, and $h_t$ is the labor provided at time $t$. Of course we can add budget constraints and such to this problem, as the author did in the videos.

However, when looking at the McCandless "ABCs of RBCs" book, the author in chapter 4 uses dynamic programming methods. Now dynamic programming methods like value iteration don't work for high dimensional problems--meaning that they are too computationally expensive.

I could not tell if McCandless gets into linear quadratic control methods and Kalman filters to solve the optimization problem above--I did not find references to "Kalman" in the index. However, Kalman filter methods and nonlinear control methods can be used to solve the objective function--especially since that function $u(\cdot)$ is usually nonlinear.

Hence I was hoping that someone could explain how economists choose which method to solve the optimization problem above.

Answer 1

The main two tools for economists solving infinite-horizon constrained optimization problems in discrete-time, as in your example problem, are:

1. Karush–Kuhn–Tucker (KKT) conditions - which is a variational method, as you noticed, and work for linear and non-linear problems.
2. Dynamic programming - which can also be used to solve linear and non-linear problems, as well.

Bellman's Principle of Optimality gives conditions for the the former approach (sequence problem - SP) to give the same solution as the latter (functional equation problem - FP), that is, SP=FE.

When looking for an analytical solution the former approach is often more convenient. It is often natural to write a problem as a sequence problem and the first-derivative approach does not require knowledge about the functional form of the solution or potentially tedious iteration.

However, the second approach has advantages when one wants to take advantage of a computer to solve a potentially more complicated problem with the use of a computer. While multiple state variables makes solving dynamic programming problems more computationally intensive, this has not deterred many economists and many notable economic models are highly dimensional (see Krusell and Smith (1998) for an early example). Deep-learning methods are also being used to speed up the solving of these large-scale problems.

Another benefit of the dynamic programming approach is that it is easier to adapt to uncertainty. An example given here.

Lastly, it is possible to extend your constrained optimization problem to a greater degree of complexity, such that you may want to use a more advanced technique, such as the Kalman Filter. An example of this may be a problem in which the state variable is unobserved to the agent.

Note: there exist continuous-time substitutes to these methods (Pontryagin's Maximum Principle and the Hamiltonian-Jacobi-Bellman equations).