3

I guess you learned about MDP from Reinforcement Learning since RL tries to solve MDP without parameters. For MDP without considering Microeconomics, (Indeed MDP is a decision-making process.) "Markov Decision Processes: Discrete Stochastic Dynamic Programming" by Martin Puterman If you want an Economics based book, "Recursive Methods in Economic ...


2

My understanding is that the standard advice is to randomly select a seed value, and then keep that seed for your entire analysis. This allows the same computer code to give identical results each time. See, for example, the Stata manual: Stata’s random-number generation functions, such as runiform() and rnormal(), do not really produce random ...


2

The value iteration operator is a contraction with respect to the supremum norm. Your example probably provides a counterexample for the statement that it's a contraction with respect to the Manhattan norm.


2

You could do a binomial-tree approximation to the process for $z_t$ and then have a different process control the number of steps you take on the tree. This preserves recombining property and it is essentially the method explored in On the Computation of Continuous Time Option Prices Using Discrete Approximations (Amin (1991)) We develop a class of ...


2

Let the $n$ states of the finite-state markov chain be denoted by $\{x_1,...,x_n\}$ and let $\vec e = [e(x_1), ..., e(x_n)]'$. Now, first note that because $X_{t+1} \mid X_t$ is independent of $W_{t+1}$, we can write \begin{align*} \exp(\eta) e(x) &= E[\exp(D'x + x' F W_{t+1})] E[ e(X_{t+1}) \mid X_t = x] \\ &= \exp(D'x + x' F F' x)\, E[ e(X_{t+1}) ...


Only top voted, non community-wiki answers of a minimum length are eligible