Rob Hayward, the former Tory MP turned psephologist, gave a presentation at ComRes on Monday which has stirred up some comment about whether the polls are underestimating Conservative support.
Historically the polls have tended to underestimate Conservative support and/or overestimate Labour support. It was most notable in 1992, but was a fairly consistent historical pattern anyway. Since the disaster of 1992 this bias has steadily reduced as pollsters have gradually switched methods and adopted some form of political control or weighting on their samples. In 2010 – at last! – the problem seemed to have been eliminated. I hope that the polling industry has now tackled and defeated the problem of Labour bias in voting intention polls, but it would be hubris to assume that because we’ve got it right once the problem has necessarily gone away and we don’t need to worry about it anymore.
In his presentation Rob compared polls last year with actual elections – the polls for the European elections, for the by-elections and for the local elections.
I looked at how the polls for the European election did here and have the same figures as Rob. Of the six pollsters who produced figures within a week or so of the election five underestimated Conservative support. The average level of Tory support across those polls was 22.2%, the Tories actually got 23.9%. The average for Labour was 27%, when they actually got 25.4%.
Looking at by-elections, Rob has taken ten by-election polls from 2014 and compared them to results. Personally I’d be more wary. By-election campaigns can move fast, and some of those polls were taken a long time before the actual campaign – the Clacton polls, for example, were conducted a month before the actual by-election took place, so any difference between the results and the polling could just as likely be a genuine change in public opinion. Taking those polls done within a week or so of the actual by-elections shows the same pattern though – Conservatives tend to be underestimated (except in Heywood and Middleton), Labour tends to be overestimated.
Finally in Rob’s presentation he has a figure for polls at the local elections in 2014. I think he’s comparing the average of national Westminster polls at the time with Rallings and Thrasher’s NEQ, which I certainly wouldn’t recommend – the Lib Dems for example always do better in local election NEQ than in national polls, but it’s because they are different types of election, not because the national polls are wrong). As it happens there was at least one actual local election poll from Survation.
Survation local election: CON 24%, LAB 36%, LDEM 13%, UKIP 18%, Others 10%
R&T local election vote: CON 26%, LAB 36%, LDEM 11%, UKIP 16%, Others 12%
Comparing it to the actual result (that is, the actual total votes cast at the local election, which is what Survation were measuring, NOT the National Equivalent Vote) these figures were actually pretty good, especially given the sample size was only 312 and that it will be skewed in unknown ways by multi-member wards. That said, the pattern is the same- it’s the Conservatives who are a couple of points too low, Labour spot on.
So, Rob is right to say that polls in 2014 that could be compared to actual results tended to show a skew away from the Conservatives and towards Labour. Would it be right to take a step on from that and conclude that the national Westminster polls are showing a similar pattern? Well, let me throw out a couple of caveats. To take the by-election polls first, these are conducted solely by two companies – Lord Ashcroft and Survation… and in the case of Survation they are done using a completely different method to Survation’s national polling, so cannot reasonably be taken as an indication of how accurate their national polling is. ICM is a similar case, their European polling was done online while all their GB Westminster polling is done by telephone. None of these examples includes any polling from MORI, Populus or ComRes’s telephone polling – in fact, given that there were no telephone based European polls, the comparison doesn’t include any GB phone polls at all, and looking at the house effects of different pollsters, online polls tend to produce more Labour-friendly figures than telephone polls do.
So what can we conclude? Well, looking at the figures by-election polls do seem to produce figures that are a bit too Laboury, but I’d be wary of assuming that the same pattern necessarily holds in national polls (especially given Survation use completely different methods for their constituency polling). At the European elections the polls also seemed to be a bit Laboury… but the pollsters who produced figures for that election included those pollsters that tend to produce the more Laboury figures anyway, and didn’t include any telephone pollsters. It would be arrogant of me to rule out the possibility that the old problems of pro-Labour bias may return, but for the time being consider me unconvinced by the argument.
UPDATE: Meanwhile the Guardian have published their monthly ICM poll, with topline figures of CON 30%(+2), LAB 33%(nc), LDEM 11%(-3), UKIP 11%(-3), GRN 9%(+4) – another pollster showing a significant advance for the Green party.
Filed under: ICM