How does the order of questions influence people’s answers?
|Image credit: wavebreakmedia via shutterstock
Rights information: http://shutr.bz/1iBPwOy
Quantum theory, developed about a century ago to explain the puzzling behavior of elementary particles, could also help explain seemingly irrational aspects of human reasoning.
The mathematics behind this highly successful physics theory has now provided a way to explain why people respond differently to survey questions depending on the questions’ ordering, scientists report June 16 in the Proceedings of the National Academy of Sciences.
Human reasoning is notoriously fickle, inconsistent and full of seemingly obvious fallacies. A prime example of such apparently irrational decision-making is the “order effect”: Researchers routinely find that the sequence in which they ask survey questions affects how people respond to them. In a 1997 Gallup poll, for instance, when surveyors asked people if they thought Bill Clinton was honest and trustworthy, roughly seven percent more respondents answered “yes” if they were first asked whether Al Gore was honest and trustworthy.
Standard probability theory, long thought to underlie human reasoning, holds that people should answer questions consistently regardless of question order. Such theories cannot account for order effects, so researchers typically consider such phenomena to be unwanted statistical noise. To minimize this noise, survey designers often split respondents into groups and give them different sequences of questions.
But cognitive scientist Zheng Wang of The Ohio State University in Columbus and colleagues suspected that notions of probability borrowed from quantum physics theory might help explain this seemingly irrational reasoning pattern.
“What our research tries to show is that … maybe quantum probability is a better way [than classical probability theory] to explain human cognition and judgment,” Wang said.
Wang and her colleagues noted a similarity between the order effect and a principle in quantum theory called commutation. According to that principle, the outcomes of certain pairs of measurements, when performed in succession, will differ depending on which measurement is performed first. For instance, if a scientist measures a particle’s position and then its momentum, he or she will get a different result from what he or she would have found by measuring the particle’s momentum and then its position. Physicists say position and momentum are “noncommuting.”
Similarly, in the Clinton-Gore example, being asked about Al Gore seemed to change how some people judge Clinton’s trustworthiness, and vice versa. This apparent noncommutativity arises because human opinions about complex issues are not hard-wired, said Wang; rather, they are cobbled together depending on the immediate situation. “Human judgment and decision-making really are context-dependent,” she said.
Wang and her colleagues realized that if the same mathematics underlying quantum commutation could explain the order effect in social science research, survey data should additionally reveal a symmetry they call “quantum question equality.” This symmetry would mean that when the question order is reversed, the number of people agreeing with both “Clinton is trustworthy” and “Gore is trustworthy” should increase by the same amount that the number of people disagreeing with both statements decreases. Likewise, the increase in respondents responding “no” to Clinton and “yes” to Gore would mirror the decrease in people responding “yes” to Clinton and “no” to Gore.
To test this symmetry, Wang and her colleagues analyzed data from 70 well-designed surveys, including the Clinton-Gore poll, three other Gallup polls and 66 surveys conducted by the Pew Research Center between 2001 and 2011. The team also analyzed data from two experiments in which they deliberately reversed the order of survey questions.
The researchers found the quantum question equality in every instance in which two questions were asked back-to-back but in different orders to different groups of respondents. The team also found, as predicted, that the symmetry did not appear in a survey in which two questions were reversed but in which a third question was inserted in between. Wang said that her team’s theory is the first mathematical model to predict both order effects and quantum question equality.
The study’s broad dataset and mathematical precision make it “a really compelling empirical demonstration” of quantum cognition theory, said psychologist Emmanuel Pothos at City University London. “It’s a powerful validation of the applicability of these ideas in cognitive modeling.”
Wang stresses that her team does not claim that the human brain is actually a quantum computer. Rather, she said, behavior consistent with quantum theory could emerge even if neural circuitry obeys only classical physics laws.
Her team’s results do, however, suggest that scientists may need to revise long-held notions of what constitutes fallacious or irrational reasoning, Wang said.
“It’s not just noise,” she said. “It’s really something essential to human beings.”
–Gabe Popkin, Inside Science News Service
Gabriel Popkin (@gabrielpopkin) is a freelance science and environmental writer based in the Washington, D.C. area. He has written for Science News, ScienceNOW, Johns Hopkins Magazine and other publications.