What methods - inspired by Haavelmo’s Structural Econometrics - can show that a partial equilibrium model is unreliable?

Question

According to Spanos 2014 Revisiting Haavelmo's Structural econometrics: Bridging the gap between theory and data Dynamic Stochastic General Equilibrium models are statistically inadequate, in such an order of magnitude as to render them useless.

The most serious problem with DSGE models is that they are often statistically inadequate. Advocates of DSGE modeling, however, have put forward a number of arguments to explain away the unreliability of their evidence, such as ‘the price one has to pay for rigorous and policy-relevant theoretical models’ (Lucas, 1980, p. 696), or simply inevitable: “The models constructed within this theoretical framework are necessarily highly abstract. Consequently, they are necessarily false, and statistical hypothesis testing will reject them. This does not imply, however, that nothing can be learned from such quantitative theoretical exercises.” (Prescott, 1986, p. 10)

Part of my research involves using a partial equilibrium models to simulate a particular segment of an economy.

Considering the above, what methods would you start with to show that a given equilibrium model is statistically unreliable?

Answer 1

I can think of two big ways.

1. Formal tests of model fit: If you are using SMM, GMM or indirect influence check out the J-stat. If you are using maximum likelihood you want a likelihood ratio. Bayesian model testing is similar to likelihood ratios, but more complicated. Many of the DSGE models were rejected on these bases (you can see Sargent discussing it briefly in this interview). These tests are all very similar.
2. Out of sample performance: Estimate the model, then use it to predict out of sample data. This is a tough test, but it is super useful. They aren't so common in economics, but statisticians use them a lot (often times they leave out part of the data and try the model out there). You probably want your model to pass a formal test in sample before you do an out of sample test.

All these tests rely on the same, extremely basic, intuition: If the model is good then it should fit the data well. If the model does not fit the data well it is probably not good.