# Transition Matrix: Discrete -> Continuous Time

I have the code corresponding to Tauchen (1986) (Python equivalent of this), which generates a discrete approximation of a discrete time AR(1) process.

For example, if you set up grid size as 3, it gives you a vector of productivities

[A_1, A_2, A_3,]


and a matrix of transition probabilities

A_11, A_12, A_13
A_21, A_22, A_23
A_31, A_32, A_33


Where row i, column j gives you the probability of transitioning from i to j, and it satisfies that the sum of each row is approximately one.

I am wondering how I can transform this to a continuous time equivalent of the transition matrix; a set of poisson probabilities controlling the flow rates between the states.

All I remember in this regard is that we can get the linear approximation to the poisson probabilities using

$$Prob(i \to j) = \lim_{\Delta\to0} \exp(-\lambda_{ij}\Delta) \approx 1-\lambda_{ij}\Delta$$

But I can't see how that helps me transforming that former matrix to the $\lambda$s... I'm looking forward to any suggestion.

Suppose $B$ is an $n\times n$ matrix of Poisson transition rates, where $B_{ij}\geq 0$ for $i\neq j$ denotes the rate at which state $i$ transitions to state $j$, and $B_{ii}\leq 0$ gives the rate at which state $i$ transitions to all other states. Each row of $B$ sums to 0.

Then if $p(t)$ denotes the probability distribution at time $t$, by definition of $B$ we have the ODE $$\dot{p}(t) = Bp(t)$$ We know what the solution to this kind of ODE looks like: $p(t)=e^{Bt}p(0)$, where $e^{Bt}$ is the matrix exponential of $Bt$. So, if we want $B$ to generate the Markov transition matrix $A$ after $t=1$, we need to have $e^B=A$.

In principle, to get $B$, we need to invert the matrix exponential, taking the matrix logarithm of $A$. The problem is that each matrix has many matrix logarithms - the logarithm in one-dimensional complex space has infinitely many branches, and this is compounded when we're talking about matrices in $n$-dimensional space. Most of these logarithms will not be satisfactory Poisson transition matrices: maybe they won't be real, or the entries won't have the right signs. Yet it is possible that more than one of them will be: in some cases there is more than one Poisson $B$ corresponding to a Markov $A$, just as in some cases there is no Poisson $B$ corresponding to $A$. It's messy.

Fortunately, there is a situation where life is relatively simple, and it almost certainly includes your own case: when all the eigenvalues of $A$ are positive, distinct reals. In this case, there is only one logarithm of $A$ that will be real, and it's easy to compute: you just diagonalize the matrix as $A=V\Sigma V^{-1}$ and take the real logarithm of the eigenvalues, getting $B=V\Omega V^{-1}$, where $\omega_{ii} = \log(\sigma_{ii})$. Indeed, you don't need to do this yourself: if you use the command $\text{logm}(A)$ in Matlab (presumably Python too), it will give you precisely this $B$.

Given this $B$, all you have to do is to verify that it's actually a Poisson matrix. The first requirement, that the rows all sum to zero, is satisfied automatically due to the construction of $B$.** The second requirement, that the diagonal elements are negative and the off-diagonal elements are positive, does not always hold (I think), but it's easy for you to check.

To see this in action, I'll consider an $A$ for a 3-state Markov process that resembles a discretized AR(1). $$A = \begin{pmatrix}0.5 & 0.4 & 0.1 \\ 0.2 & 0.6 & 0.2 \\ 0.1 & 0.4 & 0.5\end{pmatrix}$$ Now, if I type $B=\text{logm}(A)$ into Matlab, I get $$B = \begin{pmatrix}-0.86 & 0.80 & 0.06 \\ 0.40 & -0.80 & 0.40 \\ 0.06 & 0.80 & -0.86\end{pmatrix}$$ This is indeed a valid Poisson transition matrix, as we can easily check that the rows sum to zero and have the right signs - so this is our answer.

The case with positive eigenvalues is pretty important, since it spans all cases where there is not some kind of oscillatory behavior in the Markov chain (which would require negative or complex eigenvalues), presumably including your discretized AR(1).

More generally, the $\text{logm}$ command on Matlab will give us the principal matrix logarithm, an analogue of the principal scalar logarithm that takes all eigenvalues to have imaginary part between $-\pi$ and $\pi$. The problem is that this is not necessarily the logarithm we want, and by looking at it we might miss a Poisson $B$ that does generate $A$. (That's why the positive eigenvalue case, where we didn't have to worry about this, was so nice.) Still, even in these other cases it can't hurt to try and see if it works.

By the way, this problem of seeing whether there is a $B$ that generates some Markov matrix $A$ has been studied extensively. It is called the embeddability problem: see some overview and references in this excellent survey article by Davies. I'm not an expert on technical aspects of the problem, though; this answer is based more on my own hackish experience and intuition.

I feel obligated to close by seconding ecksc's comment and saying that there might be better, more direct ways to convert a discretely fitted AR(1) into a finite-state continuous time process - rather than just taking the matrix obtained via the Tauchen method and making it continuous. But I don't personally know what that better way is!

**Explanation (though I'm rusty): $A$ has a unique Perron-Frobenius eigenvalue of 1, and since $A$ is stochastic the right eigenvector of this eigenvalue is the unit vector $e$. This is still the right eigenvector, now with an eigenvalue of 0, when we take the matrix logarithm.

Can't comment, or I'd ask for more specifics first. If you're trying to convert an AR(1) process fitted against a discrete time series to a continuous time process, I found a relevant resource here on page 4.

The computations are provided for estimating the coefficients of a CAR(2) process from an AR(2) process, but of course you can substitute a 0 for the second coefficient to get your conversion.

If you're trying to convert a discrete time Markov Chain to continuous time, it's going to be more complicated and I'll have to do some more reading before I can give more help. :) In the mean time, here's some good reading material I found regarding continuous time Markov Chains.