Is there a go-to resources on the use of the scientific method in economics?

Question

I once had a physics teacher who said (paraphrased):

for any formula or principle you use on an exam, you have to be able to provide a real world test that would demonstrate the theory's validity. In other words, the test must show the theory accurately describes some real world phenomena.

I would like to better understand how economist think about "testing" their theories. For instance, psychology undergrads at my local university take a psych research methods course. I would like something along these lines for economics. But more than this, I want something that goes beyond just discussing research methods and actually assesses what makes high quality economic science. In contrast to psychology, many claims in economics are hard to test because doing so is either impossible, intractable, or impractical. Therefore, I think a good understanding of how to do economic science is crucial to learning it to figure out whether a given author is adhering to the scientific method.

Remark:

By science, I mean

CLAIM: the earth's acceleration is $g$

TEST:

We will drop a ball and measure how long it takes to hit the ground.

(1) measure a height $d$ off the surface of the earth

(2) time the ball as it drops from $d$ to the surface

(3) compute the acceleration from $d = v_0t + \frac{1}{2} a t^2$

and I will find the acceleration of the earth is $g$.

Answer 1

Economists often do something similar to what your physics teacher taught. One of my graduate econometrics teachers taught us to begin our empirical investigations by asking what the ideal experiment to test them would be. And a quick web search confirmed this is a widespread sentiment:

Esther Duflo, PhD ’99, a prominent antipoverty researcher at MIT whom Angrist advised when she was a graduate student, says his relentless concentration on the problem of selection bias—for instance, the possibility that the better-rated high school is populated with better students—spurred her to go beyond his techniques and conduct actual field experiments. “Once you ask the question right, you can ask what the ideal experiment to answer it is,” she explains

The Natural Experimenter By Peter Dizikes on January 2, 2013

For a long time I have maintained that economics is unfairly criticised for the use of unrealistic assumptions. (See Cartwright 1989, 1998). I can summarise my view by comparing an economic model to a certain kind of ideal experiment in physics: criticising economic models for using unrealistic assumptions is like criticising Galileo’s rolling ball experiments for using a plane honed to be as frictionless as possible. This defence of economic modelling has a bite, however. On the one hand, it makes clear why some kinds of unrealistic assumptions will do; but on the other, it highlights how totally misleading other kinds can be – and these other kinds of assumptions are ones that may be hard to avoid given the nature of contemporary economic theory.

The Vanity of Rigour in Economics: Theoretical Models and Galilean Experiments by NANCY CARTWRIGHT

So what is, intuitively, an experiment? The key idea is control: experimenting involves observing an event or set of events in controlled circumstances. It is useful to distinguish at least two important dimensions of control: (1) control over a variable that is changed or manipulated by the experimenter, and (2) control over other (background) conditions or variables that are set by the experimenter. Both dimensions involve the idea of design or manipulation of the experimental conditions: the experimental laboratory is in some intuitive way an “artificial” situation compared to what is likely to happen in the “natural” or “real” world. 3 An experiment is usually designed with the aim of getting a clear-cut answer to a fairly specific scientific question. As we shall see, many different kinds of questions can be answered in the laboratory. But typically, such questions regard the mutual dependence of some variables or quantities, and in particular the causal relations holding between them.

Experimentation in Economics by Francesco Guala