Seminal papers that later were proven to contain errors

Question

I was reading on institutions, and I came across Acemoglu, Johnson and Robinson (AJR) paper on The Colonial Origins of comparative development: An Empirical Investigation, and this paper seemed so 'perfect' that spawned a lot of other papers based on the data of ancient settler mortality rate and how it was used by AJR as an IV for the current quality of Institutions in former colonies.

However, as I began searching more on this paper, I found out that the paper is in fact ridden with errors, specially on data treatment. (Albouy's 2012 comment on the same paper, published by The American Economic Review)

I would like to know, if, besides this paper on development economics, are there any other seminal papers, in any area of economics, that later on were proven to have crucial mistakes/errors that invalidated the paper conclusions?

Answer 1

My favorite example is the initial formulation of Arrow's impossibility theorem in the first edition of Arrows' "Social Choice and Individual Values" (1951). In the first edition, Arrow claimed that, together with 4 other conditions, the following domain condition

``The domain $\mathcal{D}$ is sufficiently extensive so that there exists at least one free triple of alternatives. (A triple is called free if all conceivable combinations of individual orderings of this triple actually occur in $\mathcal{D}$" (rewording from Blau(1957))

implied that there exists no social welfare function $S : \mathcal{D} \rightarrow \mathcal{R}$, where $\mathcal{R}$ is the set of all possible orderings (i.e. complete and transitive binary relations) over the set of alternatives $A$.

This was later showed to be false by Blau (1957) The Existence of Social Welfare Functions" Econometrica Vol. 25, No. 2 (Apr., 1957), pp. 302-313 who provided a counter-example.

Blau also showed (among other things) that the theorem could be corrected by replacing the above domain condition by the following condition

Universal domain : the domain $\mathcal{D}$ of the social welfare function contains every possible profile of preferences over the set of alternatives $A$ (with $|A| \geq 3$).

Arrow later corrected this mistake in the second edition of Social Choice and Individual Values (1963), and the formulation of Arrow's theorem using the Universal domain condition has now become standard.

This being said, the initial error in the first edition of Arrow's book was rather minor, and the solution proposed by Blau does not reduce in any strong sense the importance of Arrow's result and approach. Intuitively, the conclusion remains that on a vast domain of relevant economic problems, no social welfare function satisfies a set of rather basic and reasonable conditions.

So this might not be exactly the kind of errors you were looking for (definitely a seminal paper though!), but I like the example so much I could not resist posting it . If such brilliant people like Arrow make these kind of mistakes, I guess it takes a little bit of the pressure off for everyone else?

Answer 2

Kydland and Prescott's seminal paper on RBC theory uses a log-log specification on consumption-leisure preferences, arguing it is the only one that matches long-run constant share of working hours (one of the Kaldor facts).

This is false. In fact, there's a whole class of additively-separable utility functions (King-Rebelo-Plosser, which were discovered (published) in the same decade) where income and substitution effects from labor income cancel and do not affect the working hours decision.

Why it is relevant? Well, because their log-log specification gives them a huge Frisch elasticity, which is the only reason why they match labor. The capital stock (as opposed to investment) doesn't vary that much over the business cycle. It is then circular logic to get TFP shocks as Solow residuals from the data, feed them into a model where capital does not move that much, and observe that you get back $AF(K,L)$ (where $K$ is almost constant and $A$ comes from the data).

Don't get me wrong, it is still a very important paper, as it is the basis of most modern macro. But people at that time were astonished how good RBC fits the data. Well, if you don't fit labor data, the rest isn't that surprising.

Answer 3

In a 1929 paper, Harold Hotelling introduced what became the standard model of spatial competition. Two firms position themselves on an interval, which induces a certain demand structure, and then compete in prices. The model was influential and widely taught. The message was that firms diferentiate minimally, both locate in the center.

But in 1979 (!) paper, d'Aspremont, Gabszewicz, and Thisse pointed out that the model has actually no equilibrium under Hotelling's original specifications. There is an important discontinuity that Hotelling and everyone afterwards ignored.

Answer 4

An influential paper by Angrist and Krueger (1991) used quarter of birth as an instrument for effect of schooling on earnings. Since compulsory schooling stops when you reach a certain age (so many dropout when they can). However, it turned out that quarter of birth is not a good instrument, correlated with family background and thus also earnings.

Buckles and Hungerman (2013) show that date of birth is correlated with characteristics of the mother. So the instrument does not satisfy the exogeneity assumption (since mother characteristics are clearly correlated with wage).

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3777829/

Answer 5

This is freebee, but I'm gonna grab it: Reinhard and Rogoff (2010, AER pp) argued that there is critical level of government debt-to-gdp ratio at around 90%, arguing that countries that cross this level of leverage typically grow less.

Ignoring the whole point of correlation versus causality, UMass student + coauthors showed that this result only holds when

• Using a specific, controversial weighting method that, iirc, was not emphasized in their paper
• Specific and controversial sample selection (as above)
• An error in their excel calculations that ignored some of the observations

Only when all these three coexist, one gets the 90% as a specific intolerance level. Otherwise, while higher debt-to-gdp ratios might correlate with smaller growth, there has been no study that shows a structural break / discontinuity at 90%.

Answer 6

Donohue & Levitt, 2001 showed a causal link between the abortion rate and the crime rate 20 years later. When abortion was legalized in the 1970s, a whole generation of unwanted births were averted, leading to a drop in crime nearly two decades later when this phantom generation would have come of age.

Legalised abortion accounts for as much as 50% of the drop in crime.

The problem with the paper was that the authors didn't do what they said they did. As part of their analysis, they subjected the data to a battery of tests. They claim to have controlled for a variety of effects which in fact, due to a coding error, they didnt. The Economist explains it as does The Wall Street Journal.

Donohue and Levitt did not run the test they thought they had—an “inadvertent but serious computer programming error”, according to Messrs Foote and Goetz... Fixing that error reduces the effect of abortion on arrests by about half, using the original data, and two-thirds using updated numbers.

Another error identified in the paper was that they used arrest rate totals instead of a population controlled variable.

These errors drastically cheapen the conclusion of a very prominent and controversial paper.

Answer 7

There is a book, The Halo Effect: . . . and the Eight Other Business Delusions That Deceive Managers (Rosenzweig (2007)), that highlights the methodological problems with a number of prominent management theory books including In search of excellence : lessons from America's best-run companies, Good to Great and Built to Last. They mostly amount to problems with selection bias.

Answer 8

The paper "A price theory of multi-sided platforms" (AER 2010) by E. Glen Weyl has a mistake which was pointed out by Tan and Wright (AER 2018). I think his paper was the first top-5 publication on two-sided platforms can be seen as quite influential. Other than the insulating tarifs, a key contribution was that profit maximization leads to classical and Spence distortions. Spence distortion means that only the externalities with respect to marginal (instead of all) users are internalized. Using American Express as an example, he writes

Given its limited ability to price discriminate, AmEx fails to fully internalize the preferences of loyal users, putting too little effort into attracting merchants and charging them a higher price than would be socially optimal.

Tan and Wright correct him

Weyl’s result only implies the marginal incentive is for AmEx to set its price too high to merchants when evaluated at the socially optimal allocation. This reflects that AmEx doesn’t take into account that loyal cardholders value additional merchants more than marginal cardholders (i.e., the existence of the Spence distortion). But under monopoly pricing, there will tend to be too few cardholders, which means that the marginal cardholders’ valuations are higher than would be the case in the socially optimal solution. Thus, the fact that AmEx focuses on marginal cardholders rather than loyal cardholders in setting fees to merchants may turn out to be second best.

The mistake in Weyl's proof in Theorem 3, equations (7) and (11). He uses the FOC for the efficient price in the FOC for the monopoly price as if these FOC are evaluated at the same allocation. It probably happened because he did not use different indices to denote the socially / privately optimal allocations.

Answer 9

Crawford and Sobel (1982) provide the basic setting for the literature on cheap talk, and the model is taught in graduate-level microeconomics around the world. Theorem 1 of the paper shows the existence of partition equilibria in a general model of strategic information transmission. In 2021, Econometrica published a corrigendum to the proof of Theorem 1: the theorem itself is correct, but the original proof contains errors. Authors of the corrigendum provide a counter-example to statements in the original proof, and then come up with a correct proof.

This case is surprising first because the "later" is really late: the correction came 39 years and more than five thousand cites later. The other interesting part of the story is that the error was discovered at an undergrad seminar!

This note is based on the discussion in the undergraduate research seminar organized by the second author, of which the first author is a member.

(I realize I am almost 7 years late to this question, but since this question showed up again recently near the top of the Economics SE, I thought it would be nice to include this example.)

Answer 10

1. Acemoglu's textbook "Introduction to Modern Economic Growth" proposition 15.12 was proven to be mistaken, which was acknowledged by Acemoglu according to the author.

2. RADER T. The existence of a utility function to represent preferences, Review of Economic Studies, (1963)'s main theorem is correct but the proof is wrong.

The proof of this mistake was published in: MEHTA G.,A remark on a utility representation theorem of Rader. Economic Theory, 9, 1997, 367–370.

A new corrected version of the proof was published in: Isler, Romano. "Semicontinuous utility functions in topological spaces." Rivista di Matematica per le Scienze economiche e sociali 20.1 (1997): 111-116.

3. Arrow's impossibility theorem originally contained a mistake, which is corrected in Blau (1957 Econometrica) and a subsequent paper by Arrow himeself.

4. Debreu (1954) originally contains a proof mistake, which is later corrected by Debreu in his International Economic Review paper.