Questions tagged [markov-chain]
The markov-chain tag has no usage guidance.
15 questions
1
vote
0
answers
18
views
Normalization for model comparisons
I have a time series applying the Markov Switching model, which is estimated in about 15 different versions. One or two of the time series had to be normalized in order to converge. That is 1-2 out of ...
- 21
1
vote
0
answers
18
views
Credit profile bank dataset for Machine learning
Can you advise please any credit profile Bank free available datasets for Markov chain and machine learning computational experiments? For my student masters thesis. Thank you.
- 11
2
votes
1
answer
69
views
Relation between Markov Perfect Nash Equilibrium and Markovian evolution of the state
I am studying dynamic games and I'm fundamentally confused about the relation between Markov Perfect Nash Equilibrium and Markovian evolution of the state. Before illustrating my doubt, let me ...
- 378
3
votes
1
answer
67
views
Stable outcomes of a dynamic process
Consider a variable $Y_t$ that evolves over time according to the fixed relation $Y_{t+1} = f(Y_t)$. Clearly, fixed points of $f$ are absorbing states of the process. To assess stability, we may now ...
- 888
2
votes
0
answers
70
views
Easy introduction to Markov processes and dynamic programming [closed]
I am taking advanced macro course in Fall. Could you please advise me a simple introduction to Markov processes and dynamic programming? I mean easy. Thanks!
- 113
2
votes
0
answers
42
views
Hybrid Phillips curve under two-state Markov process
I am studying a New Keynesian model with a hybrid Phillips curve which reads
\begin{align*}
\pi_t=\beta E_t \pi_{t+1} + \varpi \pi_{t-1}+ \kappa x_t.
\end{align*}
The demand shock in the IS curve ...
- 148
1
vote
1
answer
435
views
Should the seed be set in forecasting returns in a MCMC
Suppose we are estimating a linear model utilizing a Markov Chain Monte Carlo technique such as a Gibbs sampler, drawing from the posteriors in a Bayes framework. Suppose the full sample of data we ...
- 871
1
vote
1
answer
173
views
Textbooks addressing Markov decision processes in microeconomics/game theory/decision theory?
I have learned about the Markov decision process (MDP) formalism in the context of machine learning. I know that micro theorists also study MDP's, and agent behaviour in this context (both single and ...
- 837
2
votes
0
answers
31
views
Markov switching
I want to build some kind of two-stage Markov switching model but with two time-series. My general idea was to calculate some kind of dependence between two time-series in two different periods of ...
- 21
1
vote
0
answers
64
views
Can a MS-VAR(1) be discretized to a Markov-chain the same way a VAR(1) has a discrete approximation?
It is well known via Tauchen (1986) that a VAR(1) process can be approximately fairly well (in the absence of persistence) by a discrete Markov-chain if the innovation is distributed standard normal.
...
2
votes
0
answers
75
views
Equilibria for multi-round 'Markov' games?
I'm interested in zero-sum symmetric games which have the following form. Each player has a counter which starts at 0. Each turn, a player may choose from a fixed set of actions. A player's counter is ...
- 175
4
votes
1
answer
289
views
Markov decision processes, contractions and value iteration
I am reviewing Markov decision processes (MDP) and there is something I am missing with respect to the contraction argument. I am pretty sure it is a silly mistake somewhere (maybe computational), but ...
- 5,237
4
votes
1
answer
323
views
How to approximate stochastic volatility model with finite-state Markov chain?
A common practice when computing solutions to stochastic dynamic optimization problems is to approximate an exogenous forcing process $z_{t+1} = \rho z_t + \sigma \epsilon_{t+1}$ with a finite-state ...
- 1,096
3
votes
1
answer
71
views
How can I write a conditional expectation of finite state markov process in matrix notation
NOTE: This question is related to the econometric method explored in the following two questions:
Multiplicative factorization of stochastic growth time series--solving for an eigenfunction/...
- 9,385
1
vote
0
answers
52
views
Have any chemical reactions been used to simulate CTMC for economic simulations
Assumption and Question
Given that many economic models can be viewed as Continuous Time Markov Chains, and that many chemical reactions are also described as such, have any experiments been conducted ...
- 3,555