# Questions tagged [markov-chain]

The tag has no usage guidance.

15 questions
Filter by
Sorted by
Tagged with
1 vote
17 views

### Normalization for model comparisons

I have a time series applying the Markov Switching model, which is estimated in about 15 different versions. One or two of the time series had to be normalized in order to converge. That is 1-2 out of ...
• 21
1 vote
17 views

### Credit profile bank dataset for Machine learning

Can you advise please any credit profile Bank free available datasets for Markov chain and machine learning computational experiments? For my student masters thesis. Thank you.
66 views

### Relation between Markov Perfect Nash Equilibrium and Markovian evolution of the state

I am studying dynamic games and I'm fundamentally confused about the relation between Markov Perfect Nash Equilibrium and Markovian evolution of the state. Before illustrating my doubt, let me ...
• 368
66 views

### Stable outcomes of a dynamic process

Consider a variable $Y_t$ that evolves over time according to the fixed relation $Y_{t+1} = f(Y_t)$. Clearly, fixed points of $f$ are absorbing states of the process. To assess stability, we may now ...
• 886
70 views

### Easy introduction to Markov processes and dynamic programming [closed]

I am taking advanced macro course in Fall. Could you please advise me a simple introduction to Markov processes and dynamic programming? I mean easy. Thanks!
• 113
41 views

### Hybrid Phillips curve under two-state Markov process

I am studying a New Keynesian model with a hybrid Phillips curve which reads \begin{align*} \pi_t=\beta E_t \pi_{t+1} + \varpi \pi_{t-1}+ \kappa x_t. \end{align*} The demand shock in the IS curve ...
• 148
1 vote
426 views

### Should the seed be set in forecasting returns in a MCMC

Suppose we are estimating a linear model utilizing a Markov Chain Monte Carlo technique such as a Gibbs sampler, drawing from the posteriors in a Bayes framework. Suppose the full sample of data we ...
• 871
1 vote
168 views

### Textbooks addressing Markov decision processes in microeconomics/game theory/decision theory?

I have learned about the Markov decision process (MDP) formalism in the context of machine learning. I know that micro theorists also study MDP's, and agent behaviour in this context (both single and ...
• 845
1 vote
30 views

### Markov switching

I want to build some kind of two-stage Markov switching model but with two time-series. My general idea was to calculate some kind of dependence between two time-series in two different periods of ...
• 11
1 vote
62 views

### Can a MS-VAR(1) be discretized to a Markov-chain the same way a VAR(1) has a discrete approximation?

It is well known via Tauchen (1986) that a VAR(1) process can be approximately fairly well (in the absence of persistence) by a discrete Markov-chain if the innovation is distributed standard normal. ...
73 views

### Equilibria for multi-round 'Markov' games?

I'm interested in zero-sum symmetric games which have the following form. Each player has a counter which starts at 0. Each turn, a player may choose from a fixed set of actions. A player's counter is ...
• 175
285 views

### Markov decision processes, contractions and value iteration

I am reviewing Markov decision processes (MDP) and there is something I am missing with respect to the contraction argument. I am pretty sure it is a silly mistake somewhere (maybe computational), but ...
320 views

### How to approximate stochastic volatility model with finite-state Markov chain?

A common practice when computing solutions to stochastic dynamic optimization problems is to approximate an exogenous forcing process $z_{t+1} = \rho z_t + \sigma \epsilon_{t+1}$ with a finite-state ...
• 1,096
71 views

### How can I write a conditional expectation of finite state markov process in matrix notation

NOTE: This question is related to the econometric method explored in the following two questions: Multiplicative factorization of stochastic growth time series--solving for an eigenfunction/...
• 9,355
1 vote