How to understand statistics

When was the last time you read a controversial statement, accompanied by a ‘convincing’ number, and wondered whether it could really be true? Statistics are vital in helping us tell stories and yet in recent years we have come to doubt them more than ever.

In How to Make the World Add Up, Tim Harford draws on his experience as both an economist and presenter of the BBC’s radio show ‘More or Less’. He takes us deep into the world of disinformation and obfuscation, bad research and misplaced motivation to find those priceless jewels of data and analysis that make communicating with numbers worthwhile. With characters ranging from the art forger who conned the Nazis to the stripper who fell in love with the most powerful congressman in Washington, to famous data detectives such as John Maynard Keynes, Daniel Kahneman and Florence Nightingale. He reveals how we can evaluate the claims that surround us with confidence, curiosity and a healthy level of scepticism.

Tim believes that using ten simple rules for understanding numbers – plus one golden rule – if we keep our wits about us, think carefully about the way numbers are sourced and presented, we can look around us and see with crystal clarity how the world adds up.

In this extract, Tim discusses RULE ONE – SEARCH YOUR FEELINGS.

The aim of this book is to help you be wiser about statistics. That means I also need to help you be wiser about yourself. All the statistical expertise in the world will not prevent you believing claims you shouldn’t believe and dismissing facts you shouldn’t dismiss. That expertise needs to be complemented by control of your own emotional reactions to the statistical claims you see.

In some cases there’s no emotional reaction to worry about. Let’s say I tell you that Mars is more than 50 million kilometres, or 30 million miles, away from the Earth. Very few people have a passionately held belief about that claim, so you can start asking sensible questions immediately.

For example: is 30 million miles a long way? (Sort of. It’s more than a hundred times further than the distance between Earth and the moon. Other planets are a lot further away, though.) Hang on, isn’t Mars in a totally different orbit? Doesn’t that mean the distance between the Earth and Mars varies all the time? (Indeed it does. The minimum distance between the two planets is a bit more than 30 million miles, but sometimes Mars is more than 200 million miles away.) Because there is no emotional response to the claim to trip you up, you can jump straight to trying to understand and evaluate it.

It’s much more challenging when emotional reactions are involved, as we’ve seen with smokers and cancer statistics. Psychologist Ziva Kunda found the same effect in the lab, when she showed experimental subjects an article laying out the evidence that coffee or other sources of caffeine could increase the risk to women of developing breast cysts. Most people found the article pretty convincing. Women who drank a lot of coffee did not.

We often find ways to dismiss evidence that we don’t like. And the opposite is true, too: when evidence seems to support our preconceptions, we are less likely to look too closely for flaws.

The more extreme the emotional reaction, the harder it is to think straight. What if your doctor told you that you had a rare form of cancer, and advised you not to look it up? What if you ignored that advice, consulted the scientific literature, and discovered that the average survival time was just eight months?

Exactly that situation confronted Stephen Jay Gould, a palaeontologist and wonderful science writer, at the age of forty. ‘I sat stunned for about fifteen minutes . . .’ he wrote in an essay that has become famous. You can well imagine his emotions. Eight months to live. Eight months to live. Eight months to live. ‘Then my mind started to work again, thank goodness.’

Once his mind did start to work, Gould realised that his situation might not be so desperate. The eight months wasn’t an upper limit; it was the median average, which means that half of sufferers live longer than that. Some, possibly, live a great deal longer. Gould had a good chance: he was fairly young; his cancer had been spotted early; he’d get good treatment.

Gould’s doctor was being kind in trying to steer him away from the literature, and many of us will go to some lengths to avoid hearing information we suspect we might not like. In another experiment, students had a blood sample taken and were then shown a frightening presentation about the dangers of herpes; they were then told that their blood sample would be tested for the herpes virus. Herpes can’t be cured, but it can be managed, and there are precautions a person can take to prevent transmitting the virus to sexual partners – so it would be useful to know whether or not you have herpes. Nevertheless, a significant minority, one in five, not only preferred not to know whether they were infected but were willing to pay good money to have their blood sample discarded instead. They told researchers they simply didn’t want to face the anxiety.

Behavioural economists call this ‘the ostrich effect’. For example, when stock markets are falling, people are less likely to log in to check their investment accounts online.7 That makes no sense. If you use information about share prices to inform your investment strategy, you should be just as keen to get it in bad times as good. If you don’t, there’s little reason to log in at all – so why check your account so frequently when the market is rising?

It is not easy to master our emotions while assessing information that matters to us, not least because our emotions can lead us astray in different directions. Gould realised he hadn’t been thinking straight because of the initial shock – but how could he be sure, when he spotted those signs of hope in the statistics, that he wasn’t now in a state of denial? He couldn’t. With hindsight, he wasn’t: he lived for another twenty years, and died of an unrelated condition.

We don’t need to become emotionless processors of numerical information – just noticing our emotions and taking them into account may often be enough to improve our judgement. Rather than requiring superhuman control over our emotions, we need simply to develop good habits. Ask yourself: how does this information make me feel? Do I feel vindicated or smug? Anxious, angry or afraid? Am I in denial, scrambling to find a reason to dismiss the claim?

If you’d like to read more about understanding statistics, or just want to read some really interesting stories, Tim’s book is out now!