# When does it make sense to use variational methods, versus dynamic programming, versus nonlinear control methods so solve DSGE models

I come from a statistics and applied math background, but have been looking at some problems related to macroeconomic DSGE models lately. So I am still trying to understand the ideas and economics concepts, but I have a pretty good grasp of linear and nonlinear optimal control methods.

My fundamental question is when do economists use variational methods to solve DSGE models, versus using dynamic programming or linear optimal control methods, versus using nonlinear optimal control methods? It seems like economists use all of the above methods, but I was not sure when and why they choose one method over the other.

I was looking at some videos that present a nice simple DSGE RBC model, and solve it using variational methods--meaning the Lagrange multiplier method. The solution is analytical and derived first order conditions for the system like I learned as an undergraduate. The basic problem they are working on is the infinite and finite time horizon optimization problem:

$$max_{c, h} \quad \mathbb{E}(\sum^{\infty}_{t=0} \beta^tu(c_t, h_t))$$

In this case, $$c_t$$ is the consumption at time $$t$$, and $$h_t$$ is the labor provided at time $$t$$. Of course we can add budget constraints and such to this problem, as the author did in the videos.

However, when looking at the McCandless "ABCs of RBCs" book, the author in chapter 4 uses dynamic programming methods. Now dynamic programming methods like value iteration don't work for high dimensional problems--meaning that they are too computationally expensive.

I could not tell if McCandless gets into linear quadratic control methods and Kalman filters to solve the optimization problem above--I did not find references to "Kalman" in the index. However, Kalman filter methods and nonlinear control methods can be used to solve the objective function--especially since that function $$u(\cdot)$$ is usually nonlinear.

Hence I was hoping that someone could explain how economists choose which method to solve the optimization problem above.

• What exactly do you mean by solving DSGE models? Are you always assuming parameters are known, or do they have to be estimated?
– BrsG
Commented Jul 19, 2022 at 11:15
• Are you talking about solving these models analytically on paper or actually programming the model? With analytical solutions people just use what they are most comfortable with, if you are talking about programming then its a method that is sufficiently quick enough and easiest to apply
– 1muflon1
Commented Jul 19, 2022 at 11:16
• @BrsG By solving I mean estimating the consumption and labor vectors $c_t, h_t$. In a real world problem, I would probably look to estimate the control variables. By estimation, mean the potentially recursive optimization of those vectors. Estimating the parameters is important, but there are well known methods for that: I can use gradient descent to minimize the squared loss between the model predictions and the ground truth data, plus any additional constraints I want to impose on the solution. First I just want to efficiently estimate those two vectors. Commented Jul 19, 2022 at 13:59
• @1muflon1 I am thinking about computational or numerical solutions. The variational method is analytical, but requires some conditions to work--the existence of the first derivatives, etc. Like I was saying, I wanted to precisely understand what I gain by using more complex DP or Kalman or even nonlinear control methods? Do I gain accuracy because I can estimate more complex functions? Do I have less variance in the computed solution versus the analytical solution? Or are there other issues as well? Commented Jul 19, 2022 at 14:03

The main two tools for economists solving infinite-horizon constrained optimization problems in discrete-time, as in your example problem, are:

1. Karush–Kuhn–Tucker (KKT) conditions - which is a variational method, as you noticed, and work for linear and non-linear problems.
2. Dynamic programming - which can also be used to solve linear and non-linear problems, as well.

Bellman's Principle of Optimality gives conditions for the the former approach (sequence problem - SP) to give the same solution as the latter (functional equation problem - FP), that is, SP=FE.

When looking for an analytical solution the former approach is often more convenient. It is often natural to write a problem as a sequence problem and the first-derivative approach does not require knowledge about the functional form of the solution or potentially tedious iteration.

However, the second approach has advantages when one wants to take advantage of a computer to solve a potentially more complicated problem with the use of a computer. While multiple state variables makes solving dynamic programming problems more computationally intensive, this has not deterred many economists and many notable economic models are highly dimensional (see Krusell and Smith (1998) for an early example). Deep-learning methods are also being used to speed up the solving of these large-scale problems.

Another benefit of the dynamic programming approach is that it is easier to adapt to uncertainty. An example given here.

Lastly, it is possible to extend your constrained optimization problem to a greater degree of complexity, such that you may want to use a more advanced technique, such as the Kalman Filter. An example of this may be a problem in which the state variable is unobserved to the agent.

Note: there exist continuous-time substitutes to these methods (Pontryagin's Maximum Principle and the Hamiltonian-Jacobi-Bellman equations).

• thanks for you response, it was very helpful. The thing that I am still confused about is the idea of nonlinear system. So if the model for the dynamics or utility function is nonlinear--which is often the case--then what do you do? If you want to use a Linear Quadratic Control method, the you would need to linearize around the fixed point right? But then the control method would only optimize the consumption and labor vectors to remain at that fixed point. That is the way that optimal control would work in applied math world, but I am not sure how Econ models deal with that. Commented Jul 20, 2022 at 17:53
• I guess part of the question is whether the objective function $u(c_t, h_t)$ has only one fixed point, or whether it has more than one fixed point? If there is only one fixed point, then linearizing around that fixed point makes sense. But if there are more than one fixed point, then linear control will not help to design a control law to move from one fixed point to another. Commented Jul 20, 2022 at 17:59
• There is no problem with the utility being nonlinear, as long as we have monotonicity and discounting (Blackwell’s sufficiency conditions) the function we are solving will be a contraction mapping and have a unique fixed point. For a primer on how to solve these problems I recommend: researchspace.auckland.ac.nz/bitstream/handle/2292/190/…
– Fića
Commented Jul 21, 2022 at 3:25
• If you are looking for a more thorough treatment of why these methods work you could refer to a more advanced source, such as: Stokey, N., Lucas, R.E., with Prescott, E., Recursive Methods for Economic Dynamics, Harvard University Press (1989)
– Fića
Commented Jul 21, 2022 at 3:26