0
$\begingroup$

Let's say I have a time series model (VAR model for example). How can I know that my forecast is good ? I could use the R2 but is there something else? I also know I could just use in sample forecasting and compare my model to the observed data, but is there a problem with this method ?

Thank you

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ How are you defining quality? Do you wish to compare forecasts from one model with those with another? In sample forecasting is comparing the $hat y$ with $y$..this is similar to checking $R^2$. In any case, you need to first define quality $\endgroup$
    – Dayne
    May 9, 2022 at 10:14
  • 1
    $\begingroup$ Similar topics are popular on Cross Validated Stack Exchange. $\endgroup$
    – Richard Hardy
    May 9, 2022 at 13:52
  • $\begingroup$ This seems squarely on-topic for Cross Validated, perhaps even a better fit there than here because there is nothing specific to economics in this question; the connection to economics seems to be that economists are among the many people who study time series, VAR models, and forecasting. Given the lack of (upvoted) answers, is a migration warranted? Possible? $\endgroup$
    – Dave
    Jun 5 at 12:27

1 Answer 1

Reset to default
0
$\begingroup$

Forecasting is a very interesting topic for economists. There are many techniques, but basically the key is to compare your forecast against a base model, which could be validation data (test data), a naïve model (really basic estimation) or a more complex model.

Forecasters tend to use the Root Mean Squared Error (RMSE) or Mean Absolute Percent Error (MAPE) statistics to compare models, but there are also others like the Akaike Information Criterion (AIC), Bayesian Information Criterio (BIC), etc. You can google them out easily.

Finally, in the case of VAR model, an actual forecast of the data is not the most useful tool, but the Impulse Response Function (IRF) which is basically a simulation of what could happen (response) to a variable if another changes due to a shock (impulse). This response is dynamic (not constant) over time, to the difference of the typical regression coefficients.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.