1
$\begingroup$

Suppose we are estimating a linear model utilizing a Markov Chain Monte Carlo technique such as a Gibbs sampler, drawing from the posteriors in a Bayes framework. Suppose the full sample of data we have is from time $t = 1, \dots,\tau\dots, T$, and we want to forecast returns one period ahead using a recursive window starting at time $t = \tau$ until time $t = T$. We have trained the model from time $t=1$ to time $t=\tau$, then we want to forecast returns out of sample at time $t=\tau+1$:

\begin{equation} y_{\tau +1} = \beta' \mathbf{x}_\tau + \sigma \varepsilon_{\tau + 1}, \; \; \varepsilon_{\tau + 1} \sim \mathcal{N}(0,1) \end{equation}

When I tried this without setting the seed at each period the error term $\varepsilon_{\tau+1}$ was so stochastic across the $I$ iterations of the MCMC algorithm that $y_{\tau+1}$ was essentially a random walk. When I set the seed at each period such that the $I$ iterations in the Gibbs sampler used the same $\varepsilon_{\tau+1}$ I found much better results in terms of RMSFE.

Should we set the seed in the random number generator in each period $\tau$, such that all $I$ iterations of evaluating $y_{\tau+1}$ use the same error term $\varepsilon_{\tau+1}$ when computing this forecast?

$\endgroup$
  • $\begingroup$ By seed, do you mean $X_0$? Do you mean seeding the random number generator? If not, can you please update your question to be more precise about what you mean? Thanks. $\endgroup$ – BKay Jul 15 at 19:14
  • $\begingroup$ Thank you for pointing this out, I mean the random number generator. But I think I have realized the answer to this is to not include the random term in the forecast equation at all. $\endgroup$ – Sunhwa Jul 15 at 19:16
2
$\begingroup$

My understanding is that the standard advice is to randomly select a seed value, and then keep that seed for your entire analysis. This allows the same computer code to give identical results each time.

See, for example, the Stata manual:

Stata’s random-number generation functions, such as runiform() and rnormal(), do not really produce random numbers. These functions are deterministic algorithms that produce numbers that can pass for random. runiform() produces numbers that can pass for independent draws from a rectangular distribution over[0,1); rnormal()produces numbers that can pass for independent draws from N(0,1). Stata’s random-number functions are formally called pseudorandom-number functions.The sequences these functions produce are determined by the seed, which is just a number andwhich is set to 123456789 every time Stata is launched. ... If you record the seed you set, pseudorandom results such as results from a simulation or imputed values from mi impute can be reproduced later. Whatever you do after setting the seed, if you set the seed to the same value and repeat what you did, you will obtain the same results ... It does not really matter how you set the seed, as long as there is no obvious pattern in the seeds that you set and as long as you do not set the seed too often during a session

However, if you are using parallelization to do many separate simulations then you need to make sure that you do not use the same seed, or a time based seed, on every node:

Properly seed your generator. Even the state-of-the-art Mersenne Twister ran into problems early on because the authors had neglected the issue of proper seeding....This rule is VITAL if you are going to run parallel simulations on a Beowulf cluster for example.The simplest way to seed a RNG is to take something like the current time e.g. using the time() function found in Unix and most C libraries, which returns a 32-bit integer giving the number of seconds since 1st Jan 1970. ...Hundreds of jobs on different nodes will be starting at almost exactly the same time –therefore many of your jobs will be starting with exactly the same seed and therefore those that have the same seed will generate exactly the same results(assuming your code has no bugs in it).

Good Practice in (Pseudo) Random Number Generation for Bioinformatics Applications

$\endgroup$
  • 1
    $\begingroup$ Yup, plus: always run your analysis against different seeds as a robustness check. Present your results for the one seed, but if they’re highly sensitive to the seed value, you have a problem. $\endgroup$ – dismalscience Jul 21 at 1:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.