Over at Political Betting Mike Smithson is taking John Rentoul to task for saying in his Sunday column that the polls could be overrepresenting Labour support by up to 5 points, and the Conservatives may only need a lead of 3 points to win. Mike is right to do so.

Polls back in 1992 vastly overestimated Labour. In 1997 most of them overestimated Labour, with the honourable exception of ICM, who in their final poll actually erred towards the Conservatives. In 2001 many of them still overestimated Labour, but the lessons were slowly being learnt. In 2005 all the pollsters were very close to the actual position, though the errors were still slightly in Labour’s favour.

The graph below shows the difference between the actual Labour/Conservative lead at each of the last four elections, and what the last 8 published polls of the campaign showed. If there was not a systemic skew to the polls we would expect to see as many errors in favour of the Conservatives as in favour of Labour – as you can see, there is one lone instance of a eve-of-poll election underestimating Labour’s position (ICM in 1997).

However, what’s also clear is how much better the polls have got – in 1992 they all grossly erred in Labour’s favour. By 2005 they were all very close to the real position (the one tall bar in 2005 is a MORI poll, but their final poll of the campaign was much closer, so no one disgraced themselves).

If the polling companies behaved in exactly the same way as they did in 2005 we would expect a skew in the Conservative/Labour gap of about 2 points in Labour’s favour. However, they aren’t behaving in the same way: to give two examples, ICM have shifted their weighting to be marginally more favourable to the Conservatives (weighting past vote to a point 75% of the way towards the real result, rather than 50% as last time); Ipsos MORI are weighting by public and private sector employment, which also shifts things in the Conservatives favour. ComRes have adopted past vote weighting. Other companies have reformed their methods in ways where it harder to say what the partisan effect will be, but no one is resting on their laurels.

On past performance and the changes made since then, I would expect the polls to be pretty close to the actual result at the next election or even, in some cases, to overestimate Conservative support. Of course, the unknown quantity is how well the polls will reflect the large shift in political support since the last election. All three of the elections since the 1992 debarcle have been Labour victories, and the pollsters have been gradually honing their techniques under those circumstances. Aside from mid term polls like European and London mayoral elections, where some pollsters have done far better than others, this will be the first time the post-1992 polling methods are tested in an environment with the Conservatives ahead.

=

We should be getting ICM’s monthly poll for the Guardian tonight. My laptop has thrown a wobbly, so I may not be as quick on the uptake as usual, but I will report when I get the chance.


23 Responses to “Do the polls over-estimate Labour support?”

  1. Like any research method polling is very good at following trends as they occur but do not pick up changes in behavious that lie underneath the polling. There could be a case for saying that polls overstate Conservative support because of “shy” Labour supporters (either higher non-response or outright fibbing).

    The ‘feel’ (a guide to nothing I know) on the doorsteps is that there has been a significant switch away from Labour – hard to find someone admitting to supporting them at the moment. But the actual polling results suggest Labour are polling slightly better than in May 2008 (when they got stuffed).

    Finally, we can’t know until afterwards about polling accuracy

  2. Labour supporters can only hope that people change their mind once they step in front of the ballot box and ‘stick with what they know’.

    Did this happen in ’92?
    Is there any evidence that a significant % of folk switch at the last second – thus skewing polls?

    I know of no such evidence.

  3. One often hears the argument that the Tories should be much further ahead in the polls than they are at present to be sure of victory – coupled with comparisons to Labour’s position in the polls in 1996.

    What your article shows is that not only was Labour’s lead in the polls in 1996 inflated, but also, since the polling organisations have improved their methodology, the current reported leads are probably (we will only know for sure after the election) more accurate.

    Put another way, if a Poll today shows a Con lead of say 15%, then that is equivalent to a poll showing a Lab lead in 1996 of 20-22%.

    Given that we have had Polls in the past year showing Tory leads of 20% or more, there is indeed a parallel with the leads Labour appeared to have in 1986.

    1997 in reverse is a likely outcome for 2010.

  4. Sorry, should have said leads Labour appeared to have in 1996 !

    Interestingly, Labour did have leads of up to 25% in 1986, only to lose by a substantial margin a year later.

    1986-87 is often used as clear evidence to support the thesis that there is a swing back to the government as an election approaches – with 1982-83 and 1990-92 also adduced in support. Actually, if one looks at the pattern over the past fifty years it is more of a case of a swing to Conservative as an election approached. Note also that this swing in the final 6-18mths of a parliament is actually distinct and separate from the question of polling accuracy during the campaign itself.

  5. @Anthony – “However, what’s also clear is how much better the polls have got” – or lucky? forgive me, but considering standard variations between the polls, any result that is near to the actual should be considered more luck than design :-)

    For example ICM was hailed as being so close, but actually it might have been considered a rogue poll (happy to be corrected)

  6. ASC – “Like any research method polling is very good at following trends as they occur but do not pick up changes in behavious that lie underneath the polling.”

    Absolutely. For instance Anthony states that Ipsos Mori weight by public or private sector employment (presumably public sector being more likely to vote Labour). I do a certain amount of work for the public sector and there is a definite mood of disenchantment with the government at the moment. If this translates into a change of voting behaviour, Mori’s polls could still be overstating Labour’s support. I’m not picking on Mori, it’s just an example to back up the argument that pollsters’ sophisticated weighting can easily come unstuck.

    I wonder if it might be just as accurate to say something like “We overstated Labour support by 2% last time for whatever reason, so we’ll just knock 2% off whatever our polls say the Labour vote will be”?

  7. The problem is the ‘shy tory’ factor. You just can’t measure that. They didnt in 1992, and they won’t in 2010.

  8. Keir –

    Reasonable point, the problem with the focus that is put on the final eve-of-election polls is that they are subject to normal sample error. You could get a pollster with exemplorary methods who just has the bad luck to get a skewy sample on their final pre-election poll and ends up looking awful (to some extent Populus probably suffered like that at the last election: their final poll was further out than most of their rivals, yet they carried out another pre-election poll for Michael Ashcroft that was much closer. Unfortunately for them, it’s the one in the Times they were judged upon). In contrast, you could get a pollster producing biased, volatile crap who, by the pure luck of the draw, gets a sample that produces figures close to the actual result. Such is the problem with judging companies based on one data point every 4 years :)

    In this case though, we can be pretty certain ICM didn’t just get lucky in 1997. They weren’t just showing a lower Labour lead in that poll, it was very consistent. If you look back at the data throughout the 1992-1997 Parliament ICM were showing lower Labour leads than other companies. While it is theoretically possible that Labour had massive 20 or 30 point leads all through the Parliament, but it shot downwards at the very last minute after the final polls had closed to a figure that was, co-incidentally, close to ICM’s… it seems infinitely more likely that ICM were right and the other companies were wrong.

  9. “Labour supporters can only hope that people change their mind once they step in front of the ballot box and ’stick with what they know’.

    Did this happen in ‘92?
    Is there any evidence that a significant % of folk switch at the last second – thus skewing polls?

    I know of no such evidence.”

    it is only ancedotal but a freind of mine in the 1992 elections had been telling me through out the campaign he would vote labout when i saw him the next day and gloated he said he voted tory nquite shocked by this i asked why he said

    “when i woke up on the morning i was voting labour”
    “when i got dressed i was voting Labour”
    “when i got to the polling booth and got my voting papers i was voting Labour”
    “when i got into the voting booth i got a vision of Neil Kinnock on the steps of downing street and i voted tory!”

    I have heard several other people who said the same thing and i generally believe a lot of people did not make their mind up until they got into the polling booths this is backed up by the exit poll on the bbc that predicted a tory majority of 20 votes.

    Inceidentally it was also the nght where i first worried about bbc bias to Labour when dimbleby said in introduction

    “worse case scenario a small tory majority best case scenario a hung council” i know that is bye the bye and don’t want to start a debate on bbc objectivty.

    can i ask general question being sad does anyone know where i can buy past election night coverage?

  10. Could it be that Labour supporters are more likely to allow themselves to be polled than supporters of other parties, for whatever reason, and thus a ‘fog of war’ effect prevents pollsters from accurately representing the electorate (I know that x% of people polled will vote Labour, but I do not know what % of people I didn’t poll will)?

  11. yes by about 28%

  12. in 92′ they did not stick to what they knew.

    property was low and wanted to vote labour but could not afford them.

    the opposite now applies.
    they may well be saying they vote labour,but cant afford them so the tory vote will be higher than the polls on election day.

  13. The new ICM poll is out over at the Guardian, the tories are down 2 points, meanwhile the Lib Dems continuing to tise, they are up 2.

  14. In fact compared with the most recent ICM poll published just over a week ago both Labour and the Tories are up 2 whilst the LibDems are down 4!

  15. Anthony
    Do you remember if the exit poll on the night of the election in 1992 was correct or not or did that poll still give a lead to Labour?

  16. Glenn B
    I recall 1992 Election night very well! I was broadcating for a BBC Local Radio station on their Election Results programme and remember that the BBC exit poll predicted a VERY Hung Parliament indeed – giving the Tories 301 seats to 298 for Labour..I believe it was based on a Tory lead in the national vote of 4%. The actual results were ,of course, very different – the Tory lead was 7.6% and a majority for John Major of 21seats – 65 ahead of Labour.

  17. Sorry to kep coming back on the issue of Local Govt results.

    My belief is that there is a relationship between the number of Cllrs held by the parties before a GE and the result. In short as currently Labour will have far fewer Cllrs than the Tories and possiby also the Lib Dems then the actual swing against them will be proportionately higher. Or to put it another way the Polls are, I suggest, likely to overstate, in the present circumstances, the Labour share.

    Is there any evidence to suport this theory?

  18. It seems to me that there ought to be a difference between a poll done during an election campaign and one done mid-parliament (basically, like now) it’s because (you would think) people think more about the issues, make up their minds, etc. At the moment people are often too busy to think about politicians so much.

    This is part of the reason why local council elections don’t predict general elections too well.

  19. Over-estimating at what point in time though?

    Are they over-estimating at the moment, away from a General Election campaign, when the government is unpopular?
    Probably not –

    Would they over-estimate towards the end of an election campaign?
    Possibly. Although the changes in methods may have reduced it.

    Sorry to go back to 1979, but the pattern then seemed to be the government recovering slowly during the campaign, but doing a bit worse than expected against the polls in the last week.

  20. maybe time has clouded my memory.

  21. Could you imagine if the tories always did worse in elections that their opinion polls suggested?
    Why would a left wing party do worse in elections than the polls suggest? I mean why would Kinnock and Foot leaders of left wing less pro american left parties do worse in elections than surveys suggest. Especialy under the Reagan CIA era.

  22. The issue seems to be about fashion. When the tories were out of fashion people did not like to say they supported them. It could be argued now that the tories are in fashion and the polls may be overestimating their support because people are now ‘shy labour’