6 Lab versus field
A common claim against laboratory experiments is that they do not enable conclusions to be drawn about the “real world” Those critiques tend to rest on three foundations:
- Lab experiments are conducted in a “sterile environment” with artificial surroundings and tasks.
- Lab experiments often do not use realistic commodities or stakes. They often involve abstract rewards of materially smaller amounts than may be at stake in the outside world.
- Lab experiments don’t use “real people”. We are not interested primarily in the behaviour of students.
These three critiques mean that lab experiments can have limited relevance for predicting field behaviour unless the aspects of behaviour being studied are general across environments, stakes and subject pools.
Field experiment, by contrast:
- Involve the subject pool that is of interest (i.e. not undergraduates)
- Can actually be cheaper than lab experiments in that it is easier to get large samples in the field
- May be better environment for testing size of change. The sterility of the lab often means that the scale of the observed phenomena is not a reliable indicator of the expected external change.
- Use realistic stakes.
Lab experiments are not, however, without their advantages:
- Lab experiments can give experimenters more control. In the lab, it is easier to give strict instructions that are then followed.
- Lab experiments can also give more transparency about the subject pool (even if they are undergraduate students or workers on Amazon Turk). New subject pools may have idiosyncrasies beyond their field relevance.
- Lab experiments are typically more replicable. They are easier and cheap(er) to replicate. This can provide comfort against surprising results.
- Lab experiments are often simple more feasibile. It is often not possible to experiment in half the market.
The result of this balance of costs and benefits means that lab and field experiments should be seen as being methodologically complementary. Each can enable insights that the other can’t. Experiments in one can be used to inform the other.
We will return to the question of lab versus field experiments in more detail later when we explore the generalisability of experiments.