Possible Flaw in Psychological Studies: The Monty Hall Effect

A fascinating piece from The New York Times: (Science/Findings) And Behind Door No. 1, a Fatal Flaw By JOHN TIERNEY Published: April 8, 2008 The Monty Hall Problem has struck again, and this time it’s not merely embarrassing mathematicians. If the calculations of a Yale economist are correct, there’s a sneaky logical fallacy in some of the most famous experiments in psychology. The economist, M. Keith Chen, has challenged research into cognitive dissonance, including the 1956 experiment that first identified a remarkable ability of people to rationalize their choices. Dr. Chen says that choice rationalization could still turn out to be a real phenomenon, but he maintains that there’s a fatal flaw in the classic 1956 experiment and hundreds of similar ones. He says researchers have fallen for a version of what mathematicians call the Monty Hall Problem, in honor of the host of the old television show, “Let’s Make a Deal.” Here’s how Monty’s deal works, in the math problem, anyway. (On the real show it was a bit messier.) He shows you three closed doors, with a car behind one and a goat behind each of the others. If you open the one with the car, you win it. You start by picking a door, but before it’s opened Monty will always open another door to reveal a goat. Then he’ll let you open either remaining door. Suppose you start by picking Door 1, and Monty opens Door 3 to reveal a goat. Now what should you do? Stick with Door 1 or switch to Door 2? Before I tell you the answer, I have a request. No matter how convinced you are of my idiocy, do not immediately fire off an angry letter. In 1991, when some mathematicians got publicly tripped up by this problem, I investigated it by playing the game with Monty Hall himself at his home in Beverly Hills, but even that evidence wasn’t enough to prevent a deluge of letters demanding a correction. Before you write, at least try a few rounds of the game, which you can do by playing an online version of the game. Play enough rounds and the best strategy will become clear: You should switch doors. This answer goes against our intuition that, with two unopened doors left, the odds are 50-50 that the car is behind one of them. But when you stick with Door 1, you’ll win only if your original choice was correct, which happens only 1 in 3 times on average. If you switch, you’ll win whenever your original choice was wrong, which happens 2 out of 3 times. Now, for anyone still reading instead of playing the Monty Hall game, let me try to explain what this has to do with cognitive dissonance. For half a century, experimenters have been using what’s called the free-choice paradigm to test our tendency to rationalize decisions. This tendency has been reported hundreds of times and detected even in animals. Last year I wrote a column about an experiment at Yale involving monkeys and M&Ms. The Yale psychologists first measured monkeys’ preferences by observing how quickly each monkey sought out different colors of M&Ms. After identifying three colors preferred about equally by a monkey — say, red, blue and green — the researchers gave the monkey a choice between two of them. If the monkey chose, say, red over blue, it was next given a choice between blue and green. Nearly two-thirds of the time it rejected blue in favor of green, which seemed to jibe with the theory of choice rationalization: Once we reject something, we tell ourselves we never liked it anyway (and thereby spare ourselves the painfully dissonant thought that we made the wrong choice). But Dr. Chen says that the monkey’s distaste for blue can be completely explained with statistics alone. He says the psychologists wrongly assumed that the monkey began by valuing all three colors equally. Its relative preferences might have been so slight that they were indiscernible during the preliminary phase of the experiment, Dr. Chen says, but there must have been some tiny differences among its tastes for red, blue and green — some hierarchy of preferences. If so, then the monkey’s choice of red over blue wasn’t arbitrary. Like Monty Hall’s choice of which door to open to reveal a goat, the monkey’s choice of red over blue discloses information that changes the odds. If you work out the permutations (see illustration), you find that when a monkey favors red over blue, there’s a two-thirds chance that it also started off with a preference for green over blue — which would explain why the monkeys chose green two-thirds of the time in the Yale experiment, Dr. Chen says. Does his critique make sense? Some psychologists who have seen his working paper answer with a qualified yes. “I worked out the math myself and was surprised to find that he was absolutely right,” says Daniel Gilbert, a psychologist at Harvard. “He has essentially applied the Monty Hall Problem to an experimental procedure in psychology, and the result is both instructive and counter-intuitive.” Dr. Gilbert, however, says that he has yet to be persuaded that this same flaw exists in all experiments using the free-choice paradigm, and he remains confident that the overall theory of cognitive dissonance is solid. That view is shared by Laurie R. Santos, one of the Yale psychologists who did the monkey experiment. [Sounds like a little cognitive dissonance going on with Drs. Gilbert and Santos!] “Keith nicely points out an important problem with the baseline that we’ve used in our first study of cognitive dissonance, but it doesn’t apply to several new methods we’ve used that reveal the same level of dissonance in both monkeys and children,” Dr. Santos says. “I doubt that his critique will be all that influential for the field of cognitive dissonance more broadly.” Dr. Chen remains convinced it’s a broad problem. He acknowledges that other forms of cognitive-dissonance effects have been demonstrated in different kinds of experiments, but he says the hundreds of choice-rationalization experiments since 1956 are flawed. Even when the experimenters use more elaborate methods of measuring preferences — like asking a subject to rate items on a scale before choosing between two similarly-ranked items — Dr. Chen says the results are still suspect because researchers haven’t recognized that the choice during the experiment changes the odds. (For more of Dr. Chen’s explanation, see TierneyLab.) “I don’t know that there’s clean evidence that merely being asked to choose between two objects will make you devalue what you didn’t choose,” Dr. Chen says. “I wouldn’t be completely surprised if this effect exists, but I’ve never seen it measured correctly. The whole literature suffers from this basic problem of acting as if Monty’s choice means nothing.”