3 The role of experiments
Nobel Memorial Prize winner Alvin Roth (1995) categorised the three major uses of experiments in economics as follows:
Speaking to theorists: These are experiments designed to test the predictions of theories in a precise, controlled and measured environment that allows the observations to be interpreted in relation to the theory. The experiment is used to fill gaps that are hard to fill with raw empirical data. They are intended to feed back into the theoretical literature, help refine theoretical ideas, and discover irregularities that can help formulate new theories. Experiments can also help to distinguish among theories.
For example, game theory was a major driver of the emergence of experimental economics from the 1950s. The body of theory was tested, ultimately leading to fields such as behavioural game theory that seek to explain the gap between the original theoretical predictions and the empirical evidence as to how people play games.
Kahneman and Tversky’s testing of expected utility theory also falls into this category. They tested the theory in controlled experiments to see where it might hold or not. The failure of the theory to predict some of the phenomena fed into new theories, such as Kahneman and Tversky’s own prospect theory.
Searching for facts: This involves studying the effects of variables about which existing theory may have little to say. It can also be thought of as the search for irregularities or the isolation of the cause of previously observed regularities.
Some of Kahneman and Tversky’s (1979) early work also falls into this category. They created experiments to test whether people would depart from the rules of logic and probability, identifying a rich set of irregularities that are now the common fodder of behavioural economics.
Whispering in the ears of princes: Experiments can facilitate a dialogue between experimenters and policymakers or business decision makers. The questions in these experiments are motivated by the types of questions raised by regulatory agencies, government service providers, marketing and pricing teams, or business leaders. Experiments are used to test interventions relating to government policy and programs, and business strategy and tactics.
The work of the Behavioural Insights Team is perhaps the best-known example of this, although experiments have been used in both business and policy for almost as long as there has been a discipline of experimental economics. That said, they are rarely used relative to their potential.