I started writing this blog back in 2005, largely because I thought the media’s treatment of opinion polls was so unremittingly awful. One problem was the media’s tendency to treat whatever poll they had commissioned themselves as it if were the gospel truth, while ignoring polls commissioned by other papers. In voting intention at least they have improved on this – I quite often see journalists putting reports of voting intention figures in the context of other polls that have shown similar or contrasting trends in support. However, the problem is still rife with other polling questions – newspapers will write a whole story based on a single question in a poll they have commissioned, ignoring the evidence from many other polls on the same subject.
There is a classic example in the Independent today and their treatment of last night’s ComRes poll. I don’t wish to criticise the Indy too much- they have not misrepresented the poll in anyway, it is reported in an entirely fair and accurate way. Nevertheless, but taking a single poll question in isolation it ends up creating a shallow and one-sided picture of public opinion.
ComRes’s poll yesterday 72% of people agreed with the statement “It is time for the Coalition to change its economic policy to be focused more on promoting growth and less on spending cuts”. Taken in isolation, that suggests overwhelming opposition to the government’s economic policy and support for Labour’s alternative.
However, we don’t have to take it in isolation, as we have lots of other evidence too. I’ve written in the past about the shortcomings of “do you agree or disagree with this statement” questions – they risk skewing answers in the direction of the statement. For example, in December ComRes asked whether people agreed with the statement that “The Government should not increase public borrowing any further and its top priority should be to pay off the nation’s deficit as soon as possible” and found 74% of people agreed. Taken in isolation that would have suggested overwhelming support for the government’s position… except that ComRes also asked if people agreed that “The Government should borrow more in the short term to increase economic growth as much as possible even if it means reducing the deficit more slowly” and found that 49% of people agreed. In other words, 23% of people agreed both that the government should not borrow any more, and also that they should borrow more. ComRes’s findings in that poll suggest that the picture is not as clear as the single question today would suggest.
Unsurprisingly given the importance of the question, other polls and companies have come at the same question from different angles. Populus this month read out two sentences summarising the government view and the Labour view on the economy and cuts (without identifying them as such), and asked people which they most agreed with. They got an almost even split, 48% in favour of the government’s stance, 49% in favour of Labour’s stance.
YouGov do a similar question as a semi-regular tracker, asking people to say if the government should stick to its present strategy of reducing the deficit, even if this means growth remains slow, or whether the government should change its strategy to concentrate on growth, even if that means the deficit stays longer or gets worse. The last time YouGov asked that, also this month, showed 33% supported the present strategy and 39% wanted to change (29% weren’t sure or didn’t want either), so slightly more support for Labour’s stance than the Conservative one.
If you take a broad overview of all the polling evidence you end up with quite a mixed picture – certainly opinion seems to be moving in the direction of more of an emphasis on growth and less on the deficit, but the public remain quite evenly split. Looking at other polls people are opposed to the cuts, and they want to see more emphasis on growth. But they also want to see the deficit reduced, and think the cuts are necessary in order to do that. Ask them if they want to have their cake they say yes, ask them if they’d like to eat it they also say yes. Taking just a single polling question doesn’t give this broad picture at all – and indeed, depending on what the question was could produce entirely contradictory pictures.
The question we should ask ourselves is this – when the media talk about opinion polls, should they actually be doing their best to explain and illustrate the public’s opinion on an issue, taking all the available evidence into account, even if it ends up being muddy, confused, unclear and possibly quite dull? Or should they be plucking out single findings and trying to weave them into a sensational story?