Just wanted to ask a qualitative question in terms of the AR(1) having an infinite MA representation.

Firstly, here, in reverse order: I can understand why it is important to ensure that an MA process has an infinite AR representation (i.e. the MA/white noise process is unobservable).

However, I struggle intuitively to see why an AR (1) can be specified as an infinite MA process. In my own mind, a stationary AR(1) process has an autoregressive parameter which dies out and a zero-mean white noise error term.

However, I struggle to see why it is useful to convey the AR (1) in purely MA terms, if these MA terms need explained by an infinite AR in the first instance.

Would appreciate some clarity on the matter - or I hope at least it sparks some interesting conversation.

  • 2
    $\begingroup$ I do not quite understand what the question is. Do you want to know what the Wold decomposition is useful for? $\endgroup$ May 30 at 22:55
  • $\begingroup$ Hi Michael, thanks for your response. Firstly, yes: I think a very clear explanation of Wold's decomposition would also be helpful. And secondly, regarding my lack of clarity in the question, apologies. I was trying to say: (1) an AR representation of an MA is clearly important for estimating the MA part. (2) However, I am not quite sure why it is sound to interpret an AR as an infinite MA. $\endgroup$
    – EB3112
    May 31 at 7:28

Consider the AR process. $$ y_t = \phi_0 + \phi_1 y_{t-1} + \varepsilon_t \tag{1}. $$

The infinite MA represenation is given by:

$$ y_t = \frac{\phi_0}{1 - \phi_1} + \sum_{j = 0}^\infty \phi_1^{j} \varepsilon_{t-j} \tag{2} $$ One reason why $(2)$ is useful is to compute, for example, the mean, variance or covariance for the $y_t$ terms. Assume that $\varepsilon_t$ terms are i.i.d. with variance $\sigma^2$.

Using $(1)$ to compute the mean gives: $$ \mathbb{E}(y_t) = \phi_0 + \phi_1 \mathbb{E}(y_{t-1}). $$ This is not a closed form solution, so it would have to be solved recursively. However, if we use $(2)$ we get immediately: $$ \mathbb{E}(y_t) = \frac{\phi_0}{1 - \phi_1}. $$ Also, if we want the compute the variance using $(1)$ it would look something like this: $$ \mathbb{E}((y_t - \mathbb{E}(y_t))^2) = \left[\phi_0 + \phi_1\mathbb{E}(y_{t-1}) + \varepsilon_t - \frac{\phi_0}{1 - \phi_1} \right]^2, $$ Which does not seem very useful. However, using $(2)$ we get: $$ \begin{align*} \mathbb{E}((y_t - \mathbb{E}(y_t))^2) &= \mathbb{E}\left[\sum_{j = 0}^\infty\sum_{k = 0}^\infty \phi_1^j \phi_1^k\varepsilon_{t-j}\varepsilon_{k-j}\right],\\ &= \mathbb{E}\left[\sum_{j = 0}^\infty \phi_1^{2k}(\varepsilon_{j})^2\right],\\ &= \frac{\sigma^2}{1 - \phi_1^2} \end{align*} $$ Again, computing the covariance starting from $(1)$ will be quite cumbersome, but using $(2)$ we easily obtain $(\ell > t)$: $$ \begin{align*} \mathbb{E}((y_t - \mathbb{E}(y_t))(y_{\ell}- \mathbb{E}(y_\ell))) &= \mathbb{E}\left[\sum_{j = 0}^\infty \sum_{k = 0}^\infty \phi_1^j \phi_1^k \varepsilon_{t-j} \varepsilon_{\ell - k}\right]\\ &= \mathbb{E}\left[\sum_{j = 0}^\infty \phi_1^j \phi_1^{\ell - t + j} \sigma^2\right],\\ &= \sigma^2 \phi^{\ell - t} \sum_{j = 0}^\infty \phi_1^{2j},\\ &= \sigma^2 \frac{\phi^{\ell - t}}{1 - \phi^2} \end{align*} $$

  • $\begingroup$ Thank you @tdm. I appreciate your rigour there! $\endgroup$
    – EB3112
    May 31 at 7:31
  • 1
    $\begingroup$ very nice tdm. just a heads up that you have a typo in 2) in that the bottom subscript in summation should be zero. It's not a bug deal but it might confuse someone trying to see how to go from 2) to the equation with the double sum. $\endgroup$
    – mark leeds
    May 31 at 13:25
  • $\begingroup$ @mark leeds Thanks, I'll correct it. $\endgroup$
    – tdm
    May 31 at 13:27
  • $\begingroup$ With respect to calculating expectation and variance I would still find it easier to use (1) and simply impose stationarity $\mathbb E[y_t] = \mathbb E[y_{t-1}]$ and $\mathbb V[y_t] = \mathbb V[y_{t-1}]$. I mean I guess the infinite MA-sums only converge if we have stationarity anyway so no generality is lost or am I wrong? $\endgroup$ May 31 at 23:53
  • $\begingroup$ @Jesper Hybel That's indeed another option. Please edit the answer (or add one) if you think it's useful. $\endgroup$
    – tdm
    Jun 1 at 5:25

Take the simple example of a univariate AR(1) process, without a constant, and $0 < \phi <1 $. $$ y_t = \phi y_{t-1} + \varepsilon_t $$

It is true that the impact of any shock $\varepsilon_t$ diminishes over time. However, lagged values of $y$ also still carry the impact of earlier shocks. To see this, do a recursion. $$ y_t = \phi y_{t-1} + \varepsilon_t = \phi(\phi y_{t-2}+\varepsilon_{t-1}) + \varepsilon_t = \cdots = \phi^h y_{t-h-1} +\sum_{j=0}^h \phi^j \varepsilon_{t-j} $$

As $0<\phi<1$, this recursion can be extended to infinity. $$ y_t = \sum_{j=0}^{\infty} \phi^j \varepsilon_{t-j} $$

In plain English: the current value of $y$ is determined by the the history of all historical shocks. This is simply and implication applying recursion of the AR(1) process to infity.

Note that the cumulative impact of the shocks doesn't die out, on the contrary it's always present.

Consider $$ \frac{\delta}{\delta \varepsilon_{t-i}} y_t = \frac{\delta}{\delta \varepsilon_{t-i}} \sum_{j=0}^{\infty} \phi^j \varepsilon_{t-j} = \sum_{j=i}^{\infty} \phi^j =\sum_{j=0}^{\infty} \phi^{j+i} =: \Phi_i $$

$\Phi_i$ is the total cumulative marginal impact of a shock occurring at $t-i$. In other words it captures the impact not just up to time $t$, but beyond.

Generally, we can get the maximum cumulative marginal impact (I am sure there are better ways to phrase this) as

$$ \Phi := \sum_{j=0}^{\infty} \phi^j $$

So, one of the key information we get from the MA representation of a AR process (among other things) is this kind of impact metric.

  • $\begingroup$ Thanks @BrsG - this is of course a richer/better explanation than my own $\endgroup$
    – EB3112
    Jul 20 at 18:27

I thought I'd add an answer here to my original question on 'why an MA representation is required' having recently discovered it.

The short answer is: that an MA representation allows us to see how shocks decay.

Hope this is somewhat useful to anyone who was struggling with the conceptual point (rather than the algebraic point), as I was.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.