You may have noticed that there was a fuss in the media over the summer about the relationship between coffee and depression. “Coffee is good for you” screamed the Daily Mail. The Guardian insisted on “Why drinking coffee makes women full of beans”. Even the BBC ran with the headline “Coffee may prevent depression, scientists say”. As you may be able to tell, this story was of course based on a study which came to the exact conclusions the headlines stated: that coffee was brilliant at protecting women against depression.
The study in question was a solid one. Published in the Archives of Internal Medicine, it became the first large scale study to look at the relationship between coffee consumption and mental health. Data from an entire 50,739 women from the Nurses’ Health Study were used, and each of the participants had never previously been diagnosed with depression. The inclusion of only women in the sample is important, I should add. It has been consistently found that women are twice as likely to be diagnosed with depression as men, making the sample both statistically more viable and a little biased at the same time.
After looking at each woman’s average consumption of caffeine (in tea, soft drinks, chocolate and, yes, coffee), it was found that women who consumed more caffeinated coffee were less likely to develop depression during a ten year follow up. On a side note, the study also looked at decaf and found absolutely no association there.
So far so good. While the paper didn’t exactly find a relationship between caffeine and depression – there was no such effect for other sources of caffeine, although this could have been due to coffee being far more popular than any of the other caffeine-full choices in the distinctly non-tea drinking US sample – there was certainly a relationship between caffeinated coffee and a lack of depression. So what was wrong with the headlines?
The problem is this: we can’t say that the coffee caused women to be less susceptible to depression. This wasn’t an issue in the study itself – as the authors of the paper quite accurately pointed out, the study “only suggests the possibility of such a protective effect”. Instead, it was an issue surrounding how the study was perceived and reported.
Many of us carry the assumption that if there’s a correlation between two things then there must be a causal connection. Not only that, we assume the connection must be the one we find most intuitive, as if our world view is always the most accurate account of what‘s going on. When we find out violent children play more violent video games than other kids, we talk about how dangerous the video games are and how they‘re creating violent behaviour. Whereas if we measured the personalities of people who never missed an episode of Stephen Fry’s Planet Word to those who preferred X-Factor (not that there’s any law against watching both), we might conclude that any correlation is due to how personality influences our preferred entertainment rather than the other way around. In the same way, when we hear about a correlation between a stimulant containing beverage and a disorder involving low mood and low energy… well, you get the idea.
But just because the causation seems logical, there’s nothing to say the relationship that’s so willingly trotted out is a correct one. An idle speculation, for example, might be that those women who are tended towards depression don’t want to upset their already fragile sleep patterns with large amounts of coffee. There’s nothing particularly to support this view, and I’m not suggesting there is – just because we can imagine something doesn’t mean that it’s true. But a correlation gives no clues as to what might be influencing what.
But even if causation might go the other way than expected, does that still mean there’s a causal relationship when we find a correlation? Well, that’s where things get a little tricky. Because, as we know, a correlation may indeed point to a simple, clear-cut, causal connection. Or it may not.
A slippery fact in causation is that the relationship may well be real, but it isn’t easy to say that one causes the other. They may even be a circular relationship where both cause the other, something less paradoxical than it sounds. Going back to our Planet Word vs. X-Factor example, whether you have more conversations about semantics or the sorry death of pop music that week may depend on which programme you watch. But knowing that one of your friends is thrilled by Stephen Fry/Louis Walsh (from your conversation earlier in the week) may prompt you to watch the relevant show in order to talk sensitively about your friend‘s adoration the next week. There’s causation there, but it’s tangled.
An equal possibility is that the two factors don’t cause each other at all, but are linked by a third factor. Lack of coffee and depression might be linked by sociability, for example, or employment. Or, finally – and I hate to say this – sometimes a correlation occurs when there’s no meaningful link between the two factors involved.
There’s a famous observation about storks and babies which seems to be too irresistible to pass on. Based on some data from Oldenburg, Germany from the years 1930–36, the growth of the city population correlated with the number of storks seen around the area in that particular year. While it’s tempting to imagine the headlines should such a link be shown today (“Confirmed: Storks really do bring babies”), it’s difficult to put it down as anything but meaningless.
When it comes down to it, there’s only one way to know if one thing causes another: experiment. If the women were randomly divided up into two groups, assigned an amount of coffee (or placebo decaf) to drink and followed up over a number of years for incidents of depression. The randomisation should even out any differences between the two groups so, if you did find that the highly caffeinated group were less depression prone, you could say then that the coffee was the only factor that could possible have caused the lower depression rates.
“But why wasn’t an experiment done in the first place?” I hear you cry. “Was this observational study a complete waste of time?” The answer’s simple: an experiment like this is massively expensive and inconvenient (and how many people are going to let anyone, even serious people in white coats, dictate their coffee consumption for the next decade?). An observation study can put in the groundwork and establish that there may be a relationship – which the study in question has done admirably. The experimental study is the next step on, the thing that builds from the work already done.
So the lesson? Correlation doesn’t imply causation. It’s a cliché in itself, but worth remembering and repeating each time a health story is caught by the headlines before it’s quite ripe enough to deserve them. But there’s a deeper story too. While science itself relies on the build up and tearing down of data, “Possible link between Coffee and Depression” isn’t nearly as interesting as “Depression held at bay by Coffee Breaks”. It’s a pity really: while even the most simple of statistical rules remain overlooked by those who report the results to the wider world, it’s no wonder that so many people are willing to dismiss science as fickle and irrelevant.