Understanding the construction of stochastic processes

Question

I've seen stochastic processes modeled/constructed in the following way.

Consider the probability space $(\Omega, \mathcal F, Pr)$ and let $\mathbb S$ be the (measurable) transformation $\mathbb S: \Omega \rightarrow \Omega$ that we use to model the evolution of the sample point $\omega$ over time. Also, let $X$ be the random vector $X: \Omega \rightarrow \mathbb R^n$. Then, the stochastic process $\{ X_t: t=0,1,...\}$ is used to model a sequence of observations via the formula $ X_t(\omega) = X[\mathbb S^t(\omega)] $ or $ X_t = X \circ \mathbb S^t. $

How should I understand the sample points $\omega \in \Omega$ and the transformation $\mathbb S$ in this construction? (Could $\omega$ be something like a sequence of shocks in certain cases?)

For more concreteness, how would I write these two processes in this notation?

Process 1: $$ X_{t+1} = \rho X_t + \varepsilon_{t+1} \tag{1} $$ where $X_0 = 0$.

Process 2: $$ X_{t+1} = \varepsilon_{t+1} \tag{2} $$

Answer 1

This construction you describe is not fully general. In fact it characterizes strictly stationary time series. You see that it's shift-invariant. This operator $S$ is essentially a shift operator.

For comparison, here's the usual definition of, let's say discrete-time, processes:

Definition A stochastic process is a sequence $\{ X_t \}$ of Borel measurable maps on a probability space $( \Omega, \mathcal{F}, \mu )$.

Now for what you're describing, you have a fixed Borel measurable map $X: \Omega \rightarrow \mathbb{R}^n$. It's the underlying measure that is evolving according to $S$. The map $S$ induces a new "push-forward measure" (in measure-theoretic parlance) on $\Omega$ by just taking preimages: define a measure $\mu_S$ by

$$ A \in \mathcal{F} \stackrel{\mu_S}{\mapsto} Pr(S^{-1}(A)). $$

So the random vector $X: ( \Omega, \mathcal{F}, \mu_S) \rightarrow \mathbb{R}^n$ is $X \circ S$ by construction. They induce the same push-forward measure on $\mathbb{R}^n$. Do this with $S^t$ for each $t$ and you have your time series.

As for your question about $\omega$, inspecting the proof for the other direction should clarify this---i.e. any strictly stationary time series must necessarily take this form for some $( \Omega, \mathcal{F}, Pr)$, $X$, and $S$.

The basic point is that, from a general point of view, a stochastic process is a probability measure on the set of its possible realizations. This is seen in, for example, Wiener's construction of Brownian motion; he constructed a probability measure on $C[0, \infty)$. So in general, an $\omega$ is a sample path and $\Omega$ consists of all possible sample paths.

For example, take the two processes you named above. They are strictly stationary, if let's say the innovations are Gaussian. (Any covariance-stationary time series driven by Gaussian innovations is strictly stationary.) The construction would then start by taking $\Omega$ to be the set of all sequences, $\mathcal{F}$ the $\sigma$-algebra generated by coordinate maps, and $Pr$ the appropriate measure. For the white noise process (2), $Pr$ is just a product measure on an infinite product.

Reference This characterization/construction by shift of strictly stationary time series is mentioned in White's Asymptotic Theory for Econometricians.

Answer 2

He's just thinking of $\mathbb{S}$ as being a deterministic and $\omega$ as being unobservable. Then we observe $X(\omega)$ as a form of incomplete information about $\omega$. $\mathbb{S}$ and $X$ then help us deduce a joint probability distribution over $\{X_t\}_{t=0}^\infty$.

Answer 3

It is possible to consider cases of $\omega$ being a point in infite dimensional space, e.g. sequence of shock, but such interpretation would be unproductive, as then you will get no simplifications when compared to direct specification of the process on filtered probability space and only produced unwanted additional entities to complicate matters.

This approach is much better suited for applications to points in finite dimensional space. Then by this approach you will construct time homogeneous Markov process and $\omega$ shall be interpreted as a point in it's state space, say, current position of the process, or several last positions. The considerations on interpretation of S shall be postponed until examples are discussed.

Henceforce I presume that $\epsilon_t$ is an iid sequence of random variables on the probability space defined in the question. Then second process can be defined as follows:

$\omega \in R,$ $S(\omega) = \omega, $ $X(S^t(\omega)) = S^t(\omega).$ Upper index here denotes here multiple application of the operator.

First example is an elaboration upon the first one:

$\omega \in R^2,$ $S((\omega_1,\omega_2)) = (\rho \cdot \omega_1 + \omega_2,\omega_2), $ $X(S^t(\omega)) = (S^t(\omega))_1.$ Lower index here denotes here respective component of the corresponding vector.

As we have seen, the operation S itself is rather ambiguous and dificult to reasonably interpret. The point to be noted, however, is that it defines measure preserving transformation and taking an image under it produces the set with the same measure. So this function dynamics of measure on our state space in time.