Clustering illusion
Encyclopedia
The clustering illusion refers to the tendency erroneously to perceive small samples from random distributions to have significant "streaks" or "clusters", caused by a human tendency to underpredict the amount of variability
likely to appear in a small sample of random or semi-random data due to chance.
Thomas Gilovich
found that most people thought that the sequence looked non-random, when, in fact, it has several characteristics maximally probable for a "random" stream, such as an equal number of each result and an equal number of adjacent results with the same outcome for both possible outcomes. In sequences like this, people seem to expect to see a greater number of alternations than one would predict statistically
. The probability of an alternation in a sequence of independent random binary events is 0.5, yet people seem to expect an alternation rate of about 0.7. In fact, in a short number of trials, variability and non-random-looking "streaks" are quite probable
. For example, it strikes most people as improbable that if heads comes up four times in a row during a sequence of coin flips. However, in a series of 20 flips, there is an absolute 50% chance of getting four heads in a row (Gilovich).
Daniel Kahneman
and Amos Tversky
explained this kind of misprediction as being caused by the representativeness heuristic
(which itself they also first proposed). Gilovich argues that a similar effect occurs for other types of random dispersions, including 2-dimensional data such as seeing clusters in the locations of impact of V-1 flying bomb
s on London during World War II
or seeing streaks in stock market
price fluctuations over time.
The clustering illusion was central to a widely reported study by Gilovich, Robert Vallone and Amos Tversky
. They found that the idea that basketball
players shoot successfully in "streaks", sometimes called by sportcasters as having a "hot hand" and widely believed by Gilovich et al.'s subjects, was false. In the data they collected, if anything the success of a previous throw very slightly predicted a subsequent miss rather than another success.
Using this cognitive bias
in causal reasoning may result in the Texas sharpshooter fallacy
. More general forms of erroneous pattern recognition are pareidolia
and apophenia
.
An example of this in real life is how casinos make available to players a list of numbers on the roulette tables of previous outcomes. By allowing players to see the numbers generated before, the players have a sense of control over the outcome of the next numbers when in fact all numbers are completely random. This is usually coupled with the gambler's fallacy
.
Statistical dispersion
In statistics, statistical dispersion is variability or spread in a variable or a probability distribution...
likely to appear in a small sample of random or semi-random data due to chance.
Thomas Gilovich
Thomas Gilovich
Thomas D. Gilovich is a professor of psychology at Cornell University who has researched decision making and behavioral economics and has written popular books on said subjects. He has collaborated with Daniel Kahneman, Lee Ross and Amos Tversky....
found that most people thought that the sequence looked non-random, when, in fact, it has several characteristics maximally probable for a "random" stream, such as an equal number of each result and an equal number of adjacent results with the same outcome for both possible outcomes. In sequences like this, people seem to expect to see a greater number of alternations than one would predict statistically
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
. The probability of an alternation in a sequence of independent random binary events is 0.5, yet people seem to expect an alternation rate of about 0.7. In fact, in a short number of trials, variability and non-random-looking "streaks" are quite probable
Probability
Probability is ordinarily used to describe an attitude of mind towards some proposition of whose truth we arenot certain. The proposition of interest is usually of the form "Will a specific event occur?" The attitude of mind is of the form "How certain are we that the event will occur?" The...
. For example, it strikes most people as improbable that if heads comes up four times in a row during a sequence of coin flips. However, in a series of 20 flips, there is an absolute 50% chance of getting four heads in a row (Gilovich).
Daniel Kahneman
Daniel Kahneman
Daniel Kahneman is an Israeli-American psychologist and Nobel laureate. He is notable for his work on the psychology of judgment and decision-making, behavioral economics and hedonic psychology....
and Amos Tversky
Amos Tversky
Amos Nathan Tversky, was a cognitive and mathematical psychologist, a pioneer of cognitive science, a longtime collaborator of Daniel Kahneman, and a key figure in the discovery of systematic human cognitive bias and handling of risk. Much of his early work concerned the foundations of measurement...
explained this kind of misprediction as being caused by the representativeness heuristic
Representativeness heuristic
The representativeness heuristic is a psychological term describing a phenomenon wherein people judge the probability or frequency of a hypothesis by considering how much the hypothesis resembles available data as opposed to using a Bayesian calculation. While often very useful in everyday life, it...
(which itself they also first proposed). Gilovich argues that a similar effect occurs for other types of random dispersions, including 2-dimensional data such as seeing clusters in the locations of impact of V-1 flying bomb
V-1 flying bomb
The V-1 flying bomb, also known as the Buzz Bomb or Doodlebug, was an early pulse-jet-powered predecessor of the cruise missile....
s on London during World War II
World War II
World War II, or the Second World War , was a global conflict lasting from 1939 to 1945, involving most of the world's nations—including all of the great powers—eventually forming two opposing military alliances: the Allies and the Axis...
or seeing streaks in stock market
Stock market
A stock market or equity market is a public entity for the trading of company stock and derivatives at an agreed price; these are securities listed on a stock exchange as well as those only traded privately.The size of the world stock market was estimated at about $36.6 trillion...
price fluctuations over time.
The clustering illusion was central to a widely reported study by Gilovich, Robert Vallone and Amos Tversky
Amos Tversky
Amos Nathan Tversky, was a cognitive and mathematical psychologist, a pioneer of cognitive science, a longtime collaborator of Daniel Kahneman, and a key figure in the discovery of systematic human cognitive bias and handling of risk. Much of his early work concerned the foundations of measurement...
. They found that the idea that basketball
Basketball
Basketball is a team sport in which two teams of five players try to score points by throwing or "shooting" a ball through the top of a basketball hoop while following a set of rules...
players shoot successfully in "streaks", sometimes called by sportcasters as having a "hot hand" and widely believed by Gilovich et al.'s subjects, was false. In the data they collected, if anything the success of a previous throw very slightly predicted a subsequent miss rather than another success.
Using this cognitive bias
Cognitive bias
A cognitive bias is a pattern of deviation in judgment that occurs in particular situations. Implicit in the concept of a "pattern of deviation" is a standard of comparison; this may be the judgment of people outside those particular situations, or may be a set of independently verifiable...
in causal reasoning may result in the Texas sharpshooter fallacy
Texas sharpshooter fallacy
The Texas sharpshooter fallacy is a logical fallacy in which pieces of information that have no relationship to one another are called out for their similarities, and that similarity is used for claiming the existence of a pattern. This fallacy is the philosophical/rhetorical application of the...
. More general forms of erroneous pattern recognition are pareidolia
Pareidolia
Pareidolia is a psychological phenomenon involving a vague and random stimulus being perceived as significant. Common examples include seeing images of animals or faces in clouds, the man in the moon or the Moon rabbit, and hearing hidden messages on records played in reverse...
and apophenia
Apophenia
Apophenia is the experience of seeing meaningful patterns or connections in random or meaningless data.The term was coined in 1958 by Klaus Conrad, who defined it as the "unmotivated seeing of connections" accompanied by a "specific experience of an abnormal meaningfulness", but it has come to...
.
An example of this in real life is how casinos make available to players a list of numbers on the roulette tables of previous outcomes. By allowing players to see the numbers generated before, the players have a sense of control over the outcome of the next numbers when in fact all numbers are completely random. This is usually coupled with the gambler's fallacy
Gambler's fallacy
The Gambler's fallacy, also known as the Monte Carlo fallacy , and also referred to as the fallacy of the maturity of chances, is the belief that if deviations from expected behaviour are observed in repeated independent trials of some random process, future deviations in the opposite direction are...
.