8
$\begingroup$

In dynamic programming, the method of undetermined coefficients is sometimes known as "guess and verify." I've periodically heard there are canonical guesses one might make.

In particular, I've seen

$V(k) = A + B\ln(k)$

$V(k) = \frac{Bk^{1-\sigma}}{1-\sigma}$

The former applies to log utility while the latter is related to CRRA preferences. What other canonical guesses exist, and are these generally tied to the particular form of the return function?

Edit: For those not familiar with dynamic programs, what we're trying to do here is come up with closed-forms for the coefficients (e.g. $A$ and $B$). To over-simplify, the functional equation typically takes the generic form $V(k) = \max\bigl\{F(k,u) +\beta V\bigl(g(k,u)\bigr)\bigr\}$, where $g(\cdot,\cdot)$ describes the evolution of the state variable $k$. Essentially, the value of being in state $k$ today depends on today's return function $F(k,u)$ and some discounted value of whatever $k$ is going to be tomorrow $\beta V\bigl(g(k,u)\bigr)$. $u$ represents whatever other non-state variables you think influence the return.

Sometimes it's possible to get a closed-form solution for $V(k)$ (...note: we don't just solve for $V(k)$ since the right-hand side is a maximized quantity). This typically involves knowing something about the return function $F(k,u)$ and then making a guess about the functional form of $V(k)$. We can then iterate to see if our guess yields a closed-form solution for $V(k)$. In particular, this would include closed-forms for the coefficients in the guess (hence the method of undetermined coefficients).

$\endgroup$
  • $\begingroup$ It depends on what kind data you have. In general almost every function can be taken. But if you think that the data are distributed like a utilitiy function, than you can take $U(x,y)=x^{\alpha}\cdot y^{\beta}$ In this case you can linearize the equation: $ln(U)=\alpha\cdot ln( x)+\beta \cdot ln(y)$ To estimate the coefficients $\alpha$ and $\beta$ you can apply the method of least squares: en.wikipedia.org/wiki/Least_squares $\endgroup$ – callculus Jul 13 '15 at 13:51
  • $\begingroup$ @calculus He isn't asking about estimating $\alpha$ and $\beta$. He is asking about dynamic programming and the method of guess and verify as a method to obtain the value function that corresponds with specific utility functions. $\endgroup$ – cc7768 Jul 13 '15 at 13:58
  • $\begingroup$ @cc7768 This question is not very specific. I don´t know what the OP meant by dynamic programming in this context. I just wanted to give some hints. I had the impression that the OP was not sure what he was asking. The OP can make an edit for clarification. $\endgroup$ – callculus Jul 13 '15 at 14:38
4
$\begingroup$

Another somewhat canonical form is the value function for risk-sensitive preferences when consumption follows a random walk with drift (there are also versions including capital -- see Backus Ferriere Zin 2014).

$$c_t = \mu + c_{t-1} + \sigma_c \varepsilon_{t}$$

Begin with preferences given as Epstein-Zin with a certainty equivalence function of the form $\mu_t(x) = E_t[x_{t+1}^\alpha]^{\frac{1}{\alpha}}$:

$$V_t = \left( (1 - \beta) C_t^{\rho} + \beta \mu_t(V_{t+1}) \right)^{\frac{1}{\rho}}$$

then letting $\rho \rightarrow 0$ gives us

$$V_t = C_t^{1 - \beta} \left[\mu_t(V_t) \right]^{\beta}$$ $$V_t = C_t^{1 - \beta} \left[E_t[V_t^{\alpha}]^{\frac{1}{\alpha}} \right]^{\beta}$$

Taking logs gives us risk-sensitive preferences as presented in Hansen Sargent 1995, Tallarini 2000, etc...

Define $U_t = \log(V_t)/(1-\beta)$ and $\theta = \frac{-1}{(1-\beta) \alpha}$ then we see that:

$$U_t = \log(C_t) - \beta \theta \log \left[ E_t \left[ \exp \left( \frac{-U_{t+1}}{\theta} \right) \right] \right]$$

The form of this value function can be guessed as:

$$U_t = \gamma_0 + \gamma c_t$$

References:

  • David Backus, Axelle Ferriere, and Stanely Zin. Risk and Ambiguity in Models of Business Cycles. Carnegie-Rochester-NYU Conference. 2014.
  • Lars Ljunqvist and Thomas J. Sargent. Recursive Macroeconomic Theory, 3rd Edition. 2013.
  • T.D. Tallarini Jr. Risk-sensitive real business cycles. Journal of Monetary Economics. 2000.
  • L.P. Hansen and T.J. Sargent. Discounted linear exponential quadratic gaussian control. IEEE Trans Automatic Control. 1995.

Additional Comment: The two cases you present are more or less covered by the guess $V(k) = A + B \frac{k^{1-\sigma}}{1 - \sigma}$ since this reduces to logs as $\sigma \rightarrow 1$. The guesses are certainly tied to the particular form of the return function as the value function is related to the one period return (reward) function repeatedly obtained throughout an infinite history (if consumption were constant then it would reduce to a geometric sum).

| improve this answer | |
$\endgroup$
  • $\begingroup$ Good point on log preferences as a special case. This is a great answer, and I'll plan to keep this open a little longer to see if others have other canonical forms, too. $\endgroup$ – Pat W. Jul 14 '15 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.