Stochastic dynamic programming with history dependent shocks

Question

Consider a simple stochastic dynamic programming growth model Bellman with full depreciation,

$$V(K_{t},Z_{t})=\ln(C_{t})+\beta\mathbb{E}[V(K_{t+1},Z_{t+1})|Z_{t}]$$ where $$C_{t}=Z_{t}K_{t}^\alpha+(1-\delta)K_{t}-K_{t+1}$$ and $$Z_{t+2}=\rho_1Z_{t+1}+\rho_2Z_t+\phi\varepsilon_{t+1}+\varepsilon_{t+2};\varepsilon_{t+2}\sim N(0,\sigma^2).$$

Notice that the innovations/shocks follow a higher order stochastic process, with a moving average component. This could really be any structure, and my question remains the same.

If we use a standard process to solve for the Euler equation (forgive my abuses of time subscript notation), we have $$\frac{1}{C_{t}}=\beta\mathbb{E}_t[\frac{\alpha Z_{t+1}K_{t}^{\alpha-1}}{Z_{t+1}K_{t+1}^\alpha-K_{t+2}}].$$

And by the method of undetermined coefficients, we can conjecture the policy function to be $K_{t+1}=sZ_{t}K_{t}^\alpha \implies K_{t+2}=sZ_{t+1}K_{t+1}^\alpha $. Substituting this into the EE we have, $$\frac{1}{C_{t}}=\beta\mathbb{E}_t[\frac{\alpha Z_{t+1}K_{t}^{\alpha-1}}{Z_{t+1}K_{t+1}^\alpha-sZ_{t+1}K_{t+1}^\alpha}]$$

From which we can simplify to, $$\frac{1}{C_{t}}=\beta\mathbb{E}_t[\frac{\alpha}{K_{t+1}(1-s)}]$$ which is non stochastic in the expectation. I won't continue, as I think I can ask my question now.

Question: Most if not all of the DP problems I have seen have assumed either AR1 or iid shocks. I know that with a higher-order AR process that you need to redefine the state space such as the one from above: $$Z_{t+2}=\rho_1Z_{t+1}+\rho_2Z_t+\phi\varepsilon_{t+1}+\varepsilon_{t+2}$$ becomes some first order vector autoregressive process of the AR(2) process $$A_{t+1}=HA_t+\phi E_{t+1}.$$ where $A_{t+1}=[Z_{t+1}\quad Z_t]'$ and $H$ contains the coefficients of past shocks.

I don't see in my short derivation above where I would actually need to complicate things with the VAR. We make no assumptions about the stochastic process before Z' cancels out and we would need to take an expected value of it. It seems to me that the standard approach may work with higher order processes, but my intuition (maybe false) is telling me that it can't be that simple. Am I making a false assumption about the stationarity of the value function? If I am fine to do it using the standard approach from advanced macro, then what assumptions must we make about the stochastic process for this to be valid? Any ideas would be great, Thanks!

edit: If I use CRRA rather than log, then the expectation still has future values of Z in it, so then do I just need to expand the state space to include these? $$V(K_t,Z_t,Z_{t-1},\varepsilon_{t-1})=U(C_t)+\beta\mathbb{E}[V(K_{t+1},Z_{t+1},Z_t, \varepsilon_t)|Z_t,Z_{t-1},\varepsilon_{t-1}]$$

Answer 1

You need to distinguish two cases:

1. Your analytical example solves the Brock/Mirman model, which is uses log utility and full depreciation $\delta=1$. With log utility, income and substitution effects offset each other. Full depreciation shuts off additional wealth effects. As a consequence, there is certainty equivalence: the agent behaves as if expected values would occur with certainty. This implies that the specific structure of the stochastic process for $Z$ is irrelevant. However, it is a very special case that.
2. In pretty much all other setups (like the CRRA case you refer to), the agent must form expectations about the future using the complete stochastic process. That considerably enlarges the state space. You can use arbitrary ARMA processes (which are all covariance stationary) as you suggest above. Representing them as a VAR(1) simplifies notation and facilitates numerical solutions. These approaches usually involve discretizing the exogenous process to obtain a finite state Markov chain as in e.g. Tauchen (1986). Here, it is convenient to only deal with a generic VAR(1) process as the input without having to adjust codes to the specific ARMA structure.