Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [markov-chain]

The tag has no usage guidance.

0
votes
0answers
21 views

Textbooks addressing MDP's in microeconomics/game theory/decision theory?

I have learned about the MDP formalism in the context of machine learning. I know that micro theorists also study MDP's, and agent behaviour in this context (both single and multi-agent I think). I ...
1
vote
0answers
20 views

Markov switching

I want to build some kind of two-stage Markov switching model but with two time-series. My general idea was to calculate some kind of dependence between two time-series in two different periods of ...
1
vote
0answers
39 views

Can a MS-VAR(1) be discretized to a Markov-chain the same way a VAR(1) has a discrete approximation?

It is well known via Tauchen (1986) that a VAR(1) process can be approximately fairly well (in the absence of persistence) by a discrete Markov-chain if the innovation is distributed standard normal. ...
2
votes
0answers
60 views

Equilibria for multi-round 'Markov' games?

I'm interested in zero-sum symmetric games which have the following form. Each player has a counter which starts at 0. Each turn, a player may choose from a fixed set of actions. A player's counter is ...
4
votes
1answer
152 views

Markov decision processes, contractions and value iteration

I am reviewing Markov decision processes (MDP) and there is something I am missing with respect to the contraction argument. I am pretty sure it is a silly mistake somewhere (maybe computational), but ...
4
votes
1answer
237 views

How to approximate stochastic volatility model with finite-state Markov chain?

A common practice when computing solutions to stochastic dynamic optimization problems is to approximate an exogenous forcing process $z_{t+1} = \rho z_t + \sigma \epsilon_{t+1}$ with a finite-state ...
3
votes
1answer
53 views

How can I write a conditional expectation of finite state markov process in matrix notation

NOTE: This question is related to the econometric method explored in the following two questions: Multiplicative factorization of stochastic growth time series--solving for an eigenfunction/...
2
votes
0answers
38 views

Have any chemical reactions been used to simulate CTMC for economic simulations

Assumption and Question Given that many economic models can be viewed as Continuous Time Markov Chains, and that many chemical reactions are also described as such, have any experiments been ...