The Guardian’s monthly ICM poll is out (I’m not sure why it’s turned up at lunchtime – or perhaps I just missed it last night!). Topline figures are CON 37%(nc), LAB 36%(nc), LDEM 17%(+1), Others 10%. The Conservatives retain a one point lead. ICM do tend to show some of the most positive figures for the Conservatives (for reasons I’ll address below), but last month their 1 point Conservative lead did look like something of an outlier. It seems, however, that the difference is more systemic.

Now, looking at people’s reaction to this I’ve already seem plenty of comments remarking upon the big contrast between different companies figures – it probably deserves some explanation. Let’s start by comparing the sort of figures the different companies have produced over the last three months.

In their daily polls YouGov have been showing Labour leads of around 8 points, with the Conservatives firmly on around 36%, Labour on 42-44%, the Lib Dems on 9-10%.

Populus, who in most senses have methodology very close to that of ICM, have been showing the Conservatives between 34-39% (they dropped 5 points in their last poll, so it’s a big range), Labour firmly around 39-40%, the Lib Dems around 9-11%.

Ipsos MORI again have shown some variation in Conservative support, ranging from 32%-37% in their recent polls, have Labour more steady between 39%-42% and have the Lib Dems on 9-11%.

ComRes run parallel polls – telephone ones for the Indy and online ones for the Independent on Sunday. In their phone polls the Conservatives have been between 34-37%, Labour between 37-40%, the Lib Dems 11-13%. In their online polls the Conservatives have been between 36-38%, Labour have been between 37-40%, the Lib Dems at 10-11%.

The difference between an 8 point Labour lead and a 1 point Tory lead is large, but as Bob Worcester is want to say, look at the shares, not the lead which exaggerates variation. As you can see, with some variation and a couple of outliers like that 32% from MORI, the pollsters are largely showing the same pattern with the Conservatives, everyone has them at either around or slightly below the 37% they got at the election.

The big difference is the Lib Dems, where most companies have them around 10% or 11%, YouGov marginally lower, normally on 9% and ICM quite drastically higher, on 17%. For Labour, most companies have them between 37%-40%, the exceptions being YouGov who have them in the low 40s, and ICM who have them at 36%.

Now, one major reason behind the difference is topline adjustment – what happens to people who say “don’t know” when asked how they’ll vote. Pollsters treat these people in different ways. At one end of the scale, YouGov just ignore them; YouGov figures are based solely upon people who say how they would vote in a general election. Then we get pollsters who ask a squeeze question to people who don’t answer (are you leaning towards any of them, who would you vote for if it was a legal requirement). At the other end of the scale ICM (and, to a lesser extent, Populus) essentially make educated guesses about how these people would vote, and include them in the figures. Evidence from past elections shows that when push comes to shove “don’t knows” are likely to end up voting for the party they voted for at the previous election, hence ICM reallocate 50% of people who say don’t know to the party they voted for in 2010.

Almost by definition, this tends to be helpful to parties that have lost support since the last election, and harmful to those who have gained it. In this month’s poll it had a particularly sharp effect – before adjustment ICM’s topline figures were CON 37%, LAB 39%, LDEM 15%. The effect of reallocating don’t knows was to increase Lib Dem support from 15% to 17%, and decrease Labour support from 39% to 36%, producing a Tory lead.

Of course, that doesn’t explain all the difference. ICM would still be showing the Lib Dems on 15%, significantly higher than anyone else, and there would still be variation between the companies on other factors. There are various other explanatory factors at play – for example, I’ve seen it hypothesized that ICM’s question wording, which says “an election in your area” increases Lib Dem support. I am very sceptical of this explanation, but it is potentially a factor and is worthy of investigation. Another factor is likelihood to vote – most companies weight or filter by how likely people say they are to vote, YouGov only do this during election campaigns, when it reduces their level of Labour support.

Beyond that, there are probably factors connected with weighting – both the targets that people weight towards (primarily what assumptions they make about past vote and false recall) and also when the data is collected (YouGov and ComRes’s online polls can use stored data on panellists, polls conducted by telephone collect the data at the time the survey is conducted).

I draw no conclusions about what is right or wrong. When I first started this blog I always sought to explain the reasons behind the differences and let people make their own decisions, rather than say which polls I thought were right or wrong. In many cases, it is a philosophical difference, a case of polls measuring slightly different things – a YouGov poll is showing how people say they would vote tomorrow, an ICM poll is showing how ICM think those people would vote tomorrow. In other cases, some weighting targets probably are better than others! My sad conclusion after writing about polling for 6 years, however, is that most people tend to believe the poll they like the results from is the one to trust, and subconsciously interpret arguments about methodology to back up that preconception (not, I should add, very different from how we come to our opinions about anything else in life!)


122 Responses to “Guardian/ICM – CON 37, LAB 36, LDEM 17”

1 2 3
  1. I find this quite amusing. Jacqui Smith allegedly used convicts on day release to decorate a room in her house. Restorative justice ?

    http://www.dailymail.co.uk/news/article-2029437/Jacqui-Smith-used-convicts-day-release-redecorate-house-community-payback.html

  2. @John Fletcher

    “Am I the only one who thinks the BBC’s coverage if Libya has been very poor?”

    You may well be, although I think Colin of this parish might share your disapproval. That said, I would think that the BBC reporter and his TV crew who were ambushed by some of Gadaffi’s troops in Tripoli might take exception to your implication that they were “in the safety of the studio” and that they were indulging in activities that were “a substitute for on the spot reporting”.

    As for the divergence in recent polling results, and their susceptibility to partisan interpretation, I must say that I’m particularly intrigued by the argument relating to methodology. Surely the key requirement of any opinion poll is that it should, within the accepted statistical margin of error, accurately reflect opinion at any given time on the particular issue being polled. Widely different results justified by differing methodologies suggest some voodoo elements to me. They can’t all be right, therefore by implication some are just plain wrong, irrespective of how they may justify their methodology.

    My rule of thumb? Who called May 2010 fairly accurately (i.e 37/30/24)? Stick with them and their methodology, I’d suggest.

  3. “My rule of thumb? Who called May 2010 fairly accurately (i.e 37/30/24)? Stick with them and their methodology, I’d suggest.”

    Who was that, then?

  4. “My rule of thumb? Who called May 2010 fairly accurately (i.e 37/30/24)? Stick with them and their methodology, I’d suggest.”
    This is a bit of a problem – they all over-estimated LibDems because of Cleggmania.

    h.ttp://today.yougov.co.uk/sites/today.yougov.co.uk/files/YouGovsRecord27092010.pdf
    Yougov – 35/28/28, but yougov was still distorted by Cleggmania. Average error 2.25.
    ICM – 36/28/26. Average error 1.25.
    Populus – 37/28/27. 1.75.
    Ipsos – 36/29/27. 1.75.
    Comres – 37/28/28. 2.25.
    So ICM was the most accurate, but all pollsters overestimated the LibDems.

    But, if we look back to 2005 – when we didn’t have the Cleggmania effect. 33/36/23.
    Yougov – 32/37/24. 1.
    Populous – 32/38/21. 1.5
    ICM – 32/38/22. 1
    Mori – 33/38/23. 1

    Yougov have been good at predicting mayoral and leadership elections as well.

  5. @Tinged Fringe

    Averaging error is the wrong way to do it. You want to use cumulative error.

  6. @Nick Poole/TingedFringe

    “My rule of thumb? Who called May 2010 fairly accurately (i.e 37/30/24)? Stick with them and their methodology, I’d suggest.”

    “Who was that, then?”

    There were some outlandish pre May 10 polls, Angus Reid leading the way, but, at the risk of being accused of blatant ingratiation, YouGov appeared as accurate as any polling organisation. In fact their exit poll in terms of both vote share and seats gained was uncannily accurate. I know exit polls tend to be more precise than VI predictions made before any votes are actually cast, but I would contend that YG’s credibility is as good as anyone’s.

    That said, and just to queer the pitch a tad, ICM’s latest poll is very close to the way people actually voted in the May 2011 Local Elections, only three months ago. Food for thought, maybe.

  7. @JayBlanc
    I was just pasting the figures from the yougov pdf..

  8. @Crossbat11 – “..YouGov appeared as accurate as any polling organisation. In fact their exit poll in terms of both vote share and seats gained was uncannily accurate.”

    Indeed. In fact I vividly recall the period of around three hours or s after the polls close when journalists were reporting Tory HQ roundly dismissing the exit polls as wildly wrong. William Hague even went live on air stating that the exit polls were clearly not accurate.
    A few short hours later we found that the seat prediction was almost 100% spot on.

    This is something that isn’t widely recognised now, and I think does still carry with it a political impact similar to Labour’s defeat in 1992. Tories ferventlt believed they were going to romp home in 2010, even after the polls closed. Hague himself publically predicted a three figure majority three months before the election, and the numbing shock of their failure to win outright was largely masked by the excitement of the hung parliament and coalition talks.

    For the Tories this has perpetuated their loss of confidence, which still pervades the party. It has an impact, and will continue to do so, as greater fear means a less capable approach to crises and periods of unpopularity.

  9. The exit poll at the 2010 election was MORI/NOP (as it always is!), not YouGov.

  10. I think one potential problem with ICM’s Don’t Know reallocation is that it models solely on the last election. Which ignores that some of the new Conservative voters had not shifted ideologically but were ‘giving the conservatives their turn’, and are just as likely to shift back to their 2005 vote as their 2010 vote.

    I’d be very interested to see how different the result is when you reallocated don’t knows to their 2005 vote.

  11. “I’d be very interested to see how different the result is when you reallocated don’t knows to their 2005 vote.”
    But even that could distort the LibDem vote too much, as there was little change between 2005 and 2010 for the LDs, but their coalition position may have essentially killed their vote potential.

    There does seem to be that much difference, between the pollsters, as far as the Tory vote goes – but the difference is largely Labour/LibDem polling figures which would be distorted by the 2010 vote-weighting if LibDem VI figures have collapsed.

    So I think it’d make little difference – but I’m expecting posts from more intelligent people for why I’m wrong. ;)

  12. Looking at the last time the 3rd party was in coalition and what happened to their vote as a consequence shows it declined from 18.3% in 1974 to 13.8% in 1979, a loss of 4.5% or 25% of their total vote.

    Of course this is only one example of such a situation and things were very different back then. But now, dropping from 23% to 17% does seem broadly comparable to that, as opposed to a drop from 23% to 9%. I suppose we’ll know the truth eventually, anyway.

  13. @Anthony W

    “The exit poll at the 2010 election was MORI/NOP (as it always is!), not YouGov.”

    My mistake. Maybe it was the memory of your man Peter Kellner picking over the entrails of the exit poll data in the BBC studio on election night that made me think it was YouGov’s poll. I’ll look ar MORI/NOP in a different light now!

    @Alec

    “Indeed. In fact I vividly recall the period of around three hours or s after the polls close when journalists were reporting Tory HQ roundly dismissing the exit polls as wildly wrong. William Hague even went live on air stating that the exit polls were clearly not accurate.
    A few short hours later we found that the seat prediction was almost 100% spot on. ”

    The best one for me was dear old Paddy Ashdown pompously declaring to Jeremy Clarkson in the BBC studio that “your exit poll is clearly rubbish” when the first few results came in. He then mysteriously disappeared from the panel when it became blindingly obvious that, contrary to his earlier protestations, the poll was uncannily accurate!

    Your wider point is well made, though. As Tim Montgomerie, one of the more interesting Tory thinkers and commentators around at the moment, has observed, the Tory performance in the May 2010 election was a lamentable one in the context of the extraordinarily propitious political circumstances that existed for them. They gained a record number of seats, probably due to their Ashdown-financed targetting of marginals combined with FTTP distortions, but their vote share and the swing they gained suggested a chronic and endemic electoral weakness; one that still haunts them now, in my view.

  14. Keith P,

    I suspect that Jeremy Thorpe’s naughty business had more of an impact on the Liberal vote in 1979 than the Lib-Lab pact, which if anything would tend to win over more moderates from the Labour party than right-wing Liberals to the Conservatives.

    VI tends to be more a comment on the government than a declaration of intent regarding voting, so SOME swing Lib Dem voters might come back at the next election.

    We saw in the last parliament that the leading opposition party can have large but soft support during tough times. Labour won back a fair few voters in the last six months or so in the run-up to the election. It does look like it is governments, rather than just the Conservatives, who get a pre-election boost.

  15. @Crossbat11 – I agree. I’ve said many times that I doubt either of the two main parties could form a majority without winning some seats in their ‘no go’ regions, and while Labour can reasonably expect to make some minor gains in the SE, the Tories have almost no chance still in Scotland and the northern cities.

    I just can’t see, and haven’t for a very long time, where the seats will come from for a Tory majority. In this context I think it was a mix of delusion and arrogance from many Tories that led them to reject out of hand any electoral reform. Coalitions are their only chance of holding power, other than in highly exceptional circumstances.

  16. http://www.bbc.co.uk/news/world-europe-14646975

    More calls from the wealthy for them to pay more tax.

  17. I must say I find the LD figure in the ICM surprising, and don’t get the sense that the Tories can really be ahead at the moment. That being said, it is only a feeling & I’d not want to overemphasize it. There may be some evidence that the full bloom of the LDs’ unpopularity may have faded somewhat, and that they may be slightly up, but 15-17% seems a little high. It’s also true that, while in the past opinion polls during election campaigns have tended to underestimate their vote, in the last one the opposite was the case in general. Nonetheless, they sometimes do get the I-can’t-really-decide-between-Labour-and-Conservative-so-I’ll-put-my-cross-in-between-them kind of vote at the last minute, so I wouldn’t dismiss the poll out of hand by any means. The local elections this year, after all, showed only a very slight Labour lead in terms of vote share and while recent developments haven’t generally been good for the Government it’s clear that Labour are far from sealing the deal at the moment.

  18. Its always the same with polls, the more accurate they are the weaker the Labour showing.

  19. Allan Christie – “Its always the same with polls, the more accurate they are the weaker the Labour showing.”

    :roll:

  20. alec @ Crossbat11

    ” ….the Tories have almost no chance still in Scotland and the northern cities.”

    I can’t speak for Northern cities, but the Cons are nearly certain to make negative progress in Scotland.

    Not because of the SNP advance, not even because of unpopular cuts in services, but simply because of the long term underlying downward trend, sometimes exacerbated by current events, is inexorably downwards. The key factor now is the grim reaper for only the most loyal remain and with the LibDem debacle, we can look forward to the grudge fight in the final round between the “tired and failing” old champion not fit enough to survive many more fights and the more nimble upstart challenger.

    The Greens are waiting on the periphery like carrion to pick over the corpse of the loser hoping that the Socialists won’t come back and edge them out. LibDems and Cons have been knocked out of the contest.

    What matters is the SNP:Lab balance, and the absolute number of SNP MP’s.

    If a significant increase in SNP MP’s, with support from PC should hold the balance at Westminster, the main UK parties will be taken by surprise and unprepared for the consquenses.

    In a new situation, many will seek comfort in adhering to familiar old strategies inappropriate to changed conditions.

    Mistakes will be made and there will internal dispute about the options and recriminations especially within Labour. The disarray resulting from a failure to plan for every eventuality will be greatly to the advantage of the SNP.

    Nor should this surprise us.

    The SNP would still be a handful of hopeless romantics with at best as much political power as the Greens, were it not for the lack of interest in Scotland of the two main parties and their awsome ignorance of Scottish values and sensibilities, combined with gross incompetence in party management over decades.

    I expect to see independence in my lifetime and will vote for it, but it is not the ideal method of governance in these Islands.

    The SNP will have done little to bring it about. Christian Schmidt summed them up on these pages four years ago: “Bog standard competent government and a few minor gimmicks”.

    What a USP! Competence. It’s so unfair. How dare they. It’s not cricket. In cricket each side gets their turn. It’s almost cheating.

    Frustrate their politics
    Confound their knavish tricks

    May the best team win.

  21. I expect to see independence in my lifetime and will vote for it, but it is not the ideal method of governance in these Islands.

    Well these remarks certainly inspire confidence in us Unionists – even some of those that would support the UK being hacked apart admit that it isn’t the best way of governing Britain.

    ‘The SNP would still be a handful of hopeless romantics with at best as much political power as the Greens, were it not for the lack of interest in Scotland of the two main parties and their awsome ignorance of Scottish values and sensibilities, combined with gross incompetence in party management over decades.’

    Certainly true to an extent but I don’t think the SNP would/have been any better- half of their ‘achievements’, council tax freezing, reduced business rates, more polis on the beat and town centre improvements were in fact Tory ideas originally I’m afraid. And the SNP are gravely mistaken if they believe that anymore than half of all Scots give a toss about independence – only at most a third of Scots will PERMANENTLY support it – core support if you like- the rest will have to be brainwashed in my opinion at least. Nevertheless, even though independence comes only 25th on a list of priorities for Scots, we must be prepared to defend the union at all costs, I for one am emotionally and intellectually devoted to it.
    C

  22. Excellent analysis, thank you for explaining why there are such big differences between the polls. I have been perplexed for some time. Your explanation was clear and informative!!!

1 2 3