I seem to be having trouble in comprehending what it means for a time series to be covariance stationary. Specifically, with the third condition that for any $t,h$ the $cov(x_t,x_{t+h})$ only depends on $h$ and not $t$.

Would anyone have any examples of how a time series might be covariance stationary or any examples of non-covariance stationary time series?


2 Answers 2


Let $\epsilon_1, \epsilon_2, ... ,\epsilon_t,...$ denote i.i.d. stochastic variables, each having a standard deviation of $\sigma$.

Let $$ x_t = \sum_{i = 1}^t \epsilon_i. $$ Then $$ cov(x_t,x_{t+h}) = cov\left( \sum_{i = 1}^t \epsilon_i, \sum_{i = 1}^{t+h} \epsilon_i \right). $$ Because of the i.i.d. property $$ cov\left( \sum_{i = 1}^t \epsilon_i, \sum_{i = 1}^{t+h} \epsilon_i \right) = \sum_{i = 1}^t cov\left(\epsilon_i,\epsilon_i\right) + \sum_{i = 1}^t\sum_{j \neq i} cov\left(\epsilon_i,\epsilon_j\right) $$ and furthermore because of independence $$ \sum_{i = 1}^t cov\left(\epsilon_i,\epsilon_i\right) + \sum_{i = 1}^t\sum_{j \neq i} cov\left(\epsilon_i,\epsilon_j\right) = \sum_{i = 1}^t \sigma + \sum_{i = 1}^t\sum_{j \neq i} 0 = t \cdot \sigma. $$ Now we have shown that for this time series $$ cov(x_t,x_{t+h}) = t \cdot \sigma $$ depends on $t$ hence it is not covariance stationary.

For a covariance stationary time series, let $$ z_t = \epsilon_{t-1} + \epsilon_{t}. $$ By performing the same analyis, you can conlude that for $t > 1$, $$ cov(z_t,z_{t+h}) = \left\{ \begin{array}{ll} \sigma & \text{ if } h = 1 \\ 0 & \text{ if } h > 1. \end{array} \right. $$ Since this depends on $h$, but not on $t$, the time series $z_t$ will be covariance stationary (after $t = 1$).

  • $\begingroup$ Would you be able to maybe give an example or two of how this might come up in applications, rather than more of a math explanation? $\endgroup$
    – M. Damon
    Commented Sep 19, 2018 at 16:31
  • 1
    $\begingroup$ Hi: Any stationary ARIMA model is covariance stationary. Take an ARIMA(2,0,0), calculate the covariance and you'll see that it's only a function of h. Same thing for any of them, as long as they are stationary.. $\endgroup$
    – mark leeds
    Commented Sep 19, 2018 at 17:56
  • $\begingroup$ @markleeds Yes, that's why I chose that as a simple example. If you want to provide an explanation for a general case, write it up as an answer, I'll vote for it! $\endgroup$
    – Giskard
    Commented Sep 19, 2018 at 19:01
  • $\begingroup$ @denesp: My apologies. I didn't read your answer carefully. I imagine that there must be a general proof but I'm not able to come up with one, atleast in some reasonable amount of time. ( and maybe not even in an unreasonable amount of time ). I'll google and and see if anything is out there. $\endgroup$
    – mark leeds
    Commented Sep 20, 2018 at 0:12
  • $\begingroup$ @denesp: I think 4.5, 4.6 and 4.7 of link below is sort of a proof because, since any stationary arima model can be written in form of a wold decomposition and wold says that any covariance stationary process can be written that way, then, any stationary arima model is covariance stationary. ( but check me on that. I may be hand waving. statement is true for sure ). sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/tutorials/… $\endgroup$
    – mark leeds
    Commented Sep 20, 2018 at 0:27

@ M Damon: If you're just starting with time series ( actually, even if you're not ) , think it's best to forget about Wold Decomp and ARIMA models and all that jibber jabber. I re-read your question and I think the best way to think of covariance stationary is the following.

You're sitting in the audience and there's a stage that you look at. The stage is such that the beginning of the curtain to the end of the curtain is of length h. You also have the ability to see the h numbers ( picture each number taking up a unit length of space ) that come out ( some stochastic process causes them to arrive on stage ) in the h second interval so that they perfectly fit on stage from the left side of the stage all the way to the right.

So, at time $t = 1, X_1$ arrives on the stage. Then, the $X_2$ arrives on stage at $t = 2$, to the right of $X_1$ ( from your vantage point ). So, after $h$ seconds, you have $X_1, \ldots X_h$ on stage and ordered from left to right.

You are sitting there at time $t=h$ ( picture time stopping for a moment ) and since the observations are from some stochastic process, there is a covariance structure between those h elements on stage. Whatever it is, doesn't matter. Maybe it's $\theta^i$ where $i$ is the lag between two $X_t$ say $X_t$ and $X_{t-i}$.

Next, time starts again and you close your eyes for another $h$ seconds so that a new set of $h$ elements arrive on stage and the old ones leave. Now you're at time $2h$ and you open your eyes. So, it's time $t = 2h$, but the covariance structure between the new $h$ elements on stage does not change from what it was earlier. So, the specific elements themselves don't matter when it comes to the covariance of any two elements. All one needs to know is the distance between any two elements in order to know what the covariance ( or correlation ) of them is. 2h plays no role. All that matters for knowing the covariance of any two elements on stage is their distance. This is the practical meaning of the term "covariance stationary".

Note that stage could have been bigger or smaller. $h$ was just random number picked.

This assumption is made and is useful because ( informally speaking ) it kind of allows one to estimate model parameters in an unbiased way with only one set of time series observations. If this assumption was not made, then it would be kind of like having a sample size of $n=1$ because the stochastic realization is one element. I hope that helps.

  • $\begingroup$ Thanks for this nice explanation! I'm new to Time Series and want to clarify - isn't covariance always between two series or two datasets? Here you mention "covariance of any 2 elements" - how would you get covariance of just two numbers? What am I missing? Do you mean the corresponding Random Variables? $\endgroup$ Commented Mar 5 at 18:09
  • $\begingroup$ @user2450223. Your welcome. I mis-spoke. I should have said covariance at some lagged value. So, $cov(x_{t},x_{t-h})$ would mean the covariance at lag h. So, really what I meant is the covariance at lag $h$ for any integer value of $h$. My mistake. If you need a good text and tell me your background, I can possibly recommend one. $\endgroup$
    – mark leeds
    Commented Mar 6 at 19:10
  • $\begingroup$ One more thing: Note that you have more than one observation at say lag $i$ because during the first stage, you have $h$ elements. So, you have $(x_{t}, x_{t-i})$ and $(x_{t+1}, x_{t-i+1})$ and $(x_{t+2}, x_{t-i+2})$ etc. So, for any given set of $h$ observations on the stage, you have many observations at say lag $i$. $\endgroup$
    – mark leeds
    Commented Mar 6 at 19:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.