How to Think Critically III: Randomness

I’ve written before that the human mind is a pattern-seeking engine. We are wired by evolution to seek cause-and-effect relationships in the world around us, and when there are relationships to be found, we often do very well. The problem is when there are no causal relationships to be found. People in such situations often develop what we call superstitions, erroneous conclusions about what sorts of causes are correlated with desired outcomes. In other words, we do not have an instinctive grasp of the null hypothesis: we are reluctant to infer randomness.

What follows is an array of 1024 random bits – not pseudo-random, as computers usually produce, but truly random. They came from the HotBits server, which generates a steady stream of bytes by timing the radioactive decay of atoms of cesium-137. According to our best understanding of quantum physics, radioactive decay is a fundamentally probabilistic process – in other words, it is random. Even a hypothetical superintelligence with total knowledge of all facts in the universe prior to that point could not predict whether the next bit in HotBits’ byte stream was going to be a 0 or a 1.


Though this bitstream contains plenty of alternation, it also contains a surprising amount of clustering – including, in red, some runs of 7, 8 or even 9 consecutive 0s or 1s. “Surprising” is, of course, a relative term in this context. In statistical terms, this is far from surprising. But research has shown that human beings consistently tend to underestimate how much repetition and clustering there will be in a random sequence. We are not good random number generators. (In a psychic staring experiment, Rupert Sheldrake claimed to have obtained positive results, but detailed analysis showed that his supposedly random sequences of staring and not staring were biased against repetition – precisely the way we might expect people to guess.)

This human tendency is known as the gambler’s fallacy: the belief that independent events can influence each other’s probabilities. It is aptly named, for casinos make millions of dollars each year because of it. People who reason that a coin which has come up heads ten times in a row is “due” to come up tails are committing this fallacy; so are people who reason that a roulette wheel which has landed on red several consecutive times must be on a “hot streak” upon which they can capitalize. These are independent events with no “memory” of what preceded them. In a random stream of bits, the chance of the next bit being 0 is always 1/2, regardless of how many consecutive 0s came before it.

But the gambler’s fallacy has more subtle effects than staring experiments or roulette wheels. Due to the human reluctance to infer randomness, people often conclude that events which coincide must be causally connected. Sometimes this is a valid assumption; other times, it almost certainly is not:

The maths tells us that in respect of probability, people are much like tosses of a coin: we will see occasional clusters, even of cancer.

Tell that to the residents of Wishaw near Sutton Coldfield where a huge mobile phone mast overshadowed 35 of the houses, until someone pulled it down one bonfire night.

…”I was going through chemotherapy,” she says, “when I started meeting my neighbours in the hospital.

“It was then we started looking at the mobile phone mast and now we know for certain that it is responsible.”

In any random sample of data, we will inevitably see coincidental clustering. And the larger the pool of data is, the larger the clusters will tend to be. In a country the size of the U.K., it is very likely indeed that cancer cases, even rare forms of cancer, may cluster together purely by chance in at least a few places. Of course, it is certainly possible that a local cluster of cancer cases may have a common cause, and that possibility should not be overlooked.

The problem arises when detailed scrutiny turns up no common cause, yet sufferers understandably in search of an answer persist in believing there must be one, refusing to consider the possibility of random chance. The result, as we see here, is that innocent parties often get blamed by people who believe there simply must be some connection – such as concluding that cell phone towers cause cancer, despite the fact that these towers do not emit ionizing radiation which can damage DNA. A similar explanation was probably at play in medieval witch trials, where any unusual run of bad luck would often be blamed on black magic practiced by the town outcast. It is also seen today in so-called “healing” shrines, where the few spontaneous recoveries among vast crowds of suffering pilgrims are attributed to the miraculous powers of the shrine, rather than the inevitable occasional chance event.

Though the gambler’s fallacy is indeed a fallacy, there is a superficially similar phenomenon that is real: it is called regression to the mean. This principle states that a statistical sample whose characteristics significantly deviate from its parent data pool is more likely to be followed by a sample that conforms more closely.

The difference between the gambler’s fallacy and regression to the mean is subtle, but important. Consider again the random bit stream printed above. If I draw a sample from this pool of data and it contains a large imbalance of 0s over 1s, say, it is more likely than not that my next sample will have closer to an even split. That is regression to the mean. But it does not follow that any individual item in the sample is more likely to be a 0 or a 1 (that is the gambler’s fallacy) – it only follows that large groups of such items are, in a statistical sense, predictable, even though individual items are not.

Failure to understand this principle has given rise to many superstitious notions. Consider the “Sports Illustrated Jinx” – the belief that athletes who appear on the cover of Sports Illustrated often suffer a slump in performance thereafter, and that the former causes the latter. In all likelihood, this is an example of regression to the mean. An athlete who appears on Sports Illustrated is probably there because they have lately had an unusual – statistically unlikely – run of good performance. Since such lucky streaks are the exception and not the norm, regression to the mean tells us that the athlete’s subsequent performance is more likely to be closer to their average performance, which is perceived as a decline.

In both these cases, human beings are often brought to grief by inferring hidden causes when we should infer random chance. Our inability to innately comprehend statistical principles causes us to commit this fallacy. By treating chance as the default explanation, which it should always be, and learning to ask for evidence of an underlying cause before dismissing randomness as an explanation, we could shed many unnecessary superstitious hindrances.

Other posts in this series:


Dystopia Journal #9: Stormy Weather
"Also note how some Republicans think that red-state cities like Houston are much more deserving ..."

Dystopia Journal #9: Stormy Weather
"NO! It is all caused by the reptoids and their weather control spaceships speeding up ..."

Dystopia Journal #9: Stormy Weather
"You won't get many takers for the land swap. Everyone knows that the hurricanes are ..."

Dystopia Journal #9: Stormy Weather

Browse Our Archives

Follow Us!

What Are Your Thoughts?leave a comment