Analysis: Statistics don’t lie, but politicians often do
By Dr Matthew Ashton
In the aftermath of the Budget the airwaves were swamped by a small army of politicians debating what it meant. Listening to their arguments the thing that immediately became clear was that very few of them actually understood the statistics they were throwing around. Many of them had obviously been briefed with a set of figures and then told to recite them like parrots. As a result the debates were fairly meaningless affairs that quickly became a contest of 'my statistics are better than your statistics'.
Government policy should ideally be based on fact, yet facts are tricky things. Take, for instance, unemployment. In theory this should be a relatively easy thing to measure. How many people in the UK of working age don't have a job? But there are over a dozen different ways of measuring it, depending on what answer you want. The unemployment statistics in various countries don't always take account of the 'hidden' unemployed; people who have given up looking for work, are on training programmes or are underemployed.
These subtleties are rarely discussed by politicians for fear that they'll be accused of nit-picking in order to avoid a difficult issue. They've sometimes used this to their advantage, though. The way unemployment is measured in the UK has been changed many times over the years, usually to the current government's advantage.
There's a famous Dilbert cartoon where the boss of the office gathers the workers for a meeting to discuss a new study. He's outraged because the study shows that 40% of sick days are being taken on a Monday and Friday. He thinks this is clear evidence of people attempting to take a four-day weekend, until a worker points out that statistically this is perfectly normal as the average is 20% per day. While this might be a silly example is does illustrate a wider point where politicians often make arguments without understanding either the statistics, or how they've been produced. We've all seen politicians quoting academic studies in the media. How many of them have actually taken the time to read the studies properly? It's quite easy to state that 'this study shows…' without understanding or mentioning that it was only based on a sample size of a few hundred people. That's even before you get into the problem of them confusing anecdote for fact.
Even when they do understand the statistics they sometimes don't play fair by taking them out of context. For example if you're talking about average crime rates in the UK you get very different figures if you quote the average between 1970 and 2012 than you do if you quote them between 1995 and 2012. Far too often politicians claim something along the lines of 'If you look at the figures for the past few years…'. Generally when they do this it means they've very carefully cherry-picked the data set to best suit their argument.
Not only is this intellectually dishonest, it also makes it very hard to refute as you don't know their terms of reference. A few years ago I saw a politician on TV stating that crime had fallen amongst areas with high levels of ethnic minorities. What he failed to include was the fact that he was quoting the statistics, not for the number of crimes, but for the number of crimes reported (for a more accurate result he should have used data from the British Crime Survey). Equally it could be argued that ethnic minorities sometimes don't report crimes to the police because they view them as either being institutionally racist, or unable to solve them, so again the figure is open to interpretation. Politicians very rarely mention the fact that statistics and how they're gathered can vary enormously from place to place. As a result whenever politicians try to make a point about how certain figures in the UK are better than abroad, it's often meaningless because the two sets of data are not comparing like with like.
Finally politicians don't always announce where they got their data from. This can make a huge difference. If you quote statistics about the number of accidents involved in the extraction of natural gas, but it comes from a report funded by the energy companies themselves, that immediately opens it up to scrutiny. Whenever any politician or public figure quotes a statistics, the immediate question that should be asked is where did this data come from? Who collected it? What methodology did they use? Might they be biased in a particular direction?
All of these things matter because it undermines the public's trust in the facts that government policy is meant to be based on. For instance crime has been on average decreasing across the UK since the mid-1990s. However you wouldn't think this if you talked to the public or the press. In part this is because of numerous governments attempts to alter the figures in their favour, leading to distrust whenever politician's wheel out another set of statistics. Whenever people mention statistics the first quote that comes to mind is 'lies, damn lies and statistics'. Well statistics don't lie, after all they're just numbers. Unfortunately politicians do.
Dr Matthew Ashton is a politics lecturer at Nottingham Trent University. Visit his blog.
The opinions in politics.co.uk's Comment and Analysis section are those of the author and are no reflection of the views of the website or its owners.
-01.png){kind=spacer}