6
$\begingroup$

I have read several experimental papers which ask people to make decisions which only matter with a certain probability. For example, individuals might be asked to submit bids in an auction, but told that their bids only 'count' with a certain probability $p$ (with probability $1 - p$, their payoff is 0 regardless of what they do). Alternatively, if we wanted to learn whether an individual's valuation for a good exceeds £10, we could offer to sell her that good for £10; but tell her that the good will be sold to her (and the payment taken) only with a certain probability. In both cases, the idea is that individuals should act as they would if $p = 1$ (i.e. the mechanism is 'incentive compatible') which then allows researchers to run these experiments more cheaply than they would be able to otherwise.

Is there any evidence about the general validity of this approach? More concretely, is there evidence as to whether the outputs of such experiments vary with $p$? Intuitively, I would have expected that subjects become inattentive if $p$ is very low (since in absolute terms, it doesn't matter much what they do), which could then add noise to the elicited estimates.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ There is some evidence to the contrary. At least it seems to me that the setting in the Allais Paradox is similar, though not identical. $\endgroup$
    – Giskard
    May 12 '20 at 19:26
  • 1
    $\begingroup$ Right, I would think that this procedure assumes EU (and also zero decision making costs) $\endgroup$
    – Itzhak Rasooly
    May 13 '20 at 8:04
1
$\begingroup$

Interesting question. I don't have any empirical evidence to offer, so I can't answer your question if that's what you're asking for.

But it's easy to provide "theoretical evidence", as it were, that your concerns may be well-founded, even without leaving the EU model.

You simply need to add a decision cost --- or a cost to figure out your true WTP --- to the decision-theoretic model of your participants. Then, as the EU stakes decrease (because $p$ decreases), participants are less likely to invest in the introspection cost of discovering their true WTP. As a consequence, participants' behavior become a much noisier indication of their WTP:

  • Some of your participants will decide to buy the good although their true WTP is lower than the price. This could happen if the participant decided not to invest in introspection and therefore got a sufficiently upward-biased signal of their true WTP.
  • Other participants will decide not to buy the good although their true WTP is above the price. This could happen if the participant decided not to invest in introspection and as a consequence got a sufficiently downward-biased signal of their true WTP.

Assuming the cost of introspection does not change with $p$, participants become less likely to invest in introspection as $p$ decreases (because decreasing $p$ decreases the stakes, and therefore the benefits of introspection), and you'll get more of this noise as $p$ decreases.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I saw your comment "Right, I would think that this procedure assumes EU (and also zero decision-making costs) " after writing this answer. So yes, as you suggested, my point is that adding decision costs (even without dropping EU) makes the accuracy of the approach questionable. Given some budget constraint on the cost of the experiment, there's maybe a trade-off here between getting a few accurate observations (with high $p$) versus a lot of noisy observations (with low $p$). $\endgroup$
    – Martin Van der Linden
    May 14 '20 at 14:18
0
$\begingroup$

Freeman and Mayraz (2018) document that the presentation of safe-vs-risky decisions in a choice list has a distinct effect on people's decisions, whether or not payment is based on predeterminted or randomly selected items from that list. The abstract of their paper reads:

Choice lists with random incentives are widely used for preference elicitation. It is commonly assumed that subjects choose the same option in each question as they would have if it were the only question, but recent findings challenge this assumption. We conduct a large sample experiment varying incentives and presentation independently, and examine choices both near and away from certainty. We consistently find more risk taking when a choice between a safe prize and a risky lottery is embedded in a choice list than when it is presented on its own. This difference remains when we inform subjects of the paid choice in advance, implying that isolation fails not because of the random incentives scheme, but simply because the choice appears in a list together with others. We conjecture that subjects are uncertain about their preferences, reduce this uncertainty through considering the choices that confront them, and make cautious decisions in the interim. Other conditions and non-choice data support this interpretation. Our results open up the possibility that preferences inferred from choice lists offer a better indication of informed preferences than preferences inferred from single choices.

In a subsequent paper, Freeman, Halevy and Kneeland (2019) provided a non-EU theory that rationalizes the finding.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.