ICM’s Crewe and Nantwich poll – which I look at here – also heralds two new changes to ICM’s methodology, which will probably be rolling out across their other political polls for the Guardian and Sunday Telegraph.

The first change is to to adjust their targets for past vote weighting to be slightly closer to the actual result of the 2005 election – specifically, the target is now based 80% on the 2005 election results, and 20% on the average recalled vote in ICM’s polls. In theory this will produce results that are slightly better for the Conservatives and Lib Dems and slightly worse for Labour, but in practice it is a very, very minor change. Taking the Crewe and Nantwich poll, the change was not large enough to change the results by a percentage point.

The second change is more interesting. As I discussed before the London election, turnout is actually very challenging to predict and is probably one of the reasons MORI seem to have overestimated Labour’s lead. One of the reasons it is so tricky is that people aren’t very good at predicting their own likelihood to vote. In the UK pollsters normally rely on asking people to rate their likelihood to vote on a scale of 1-10, but this still often produces more people who are 10/10 certain to vote than actually do. ICM’s new approach seems to draw some lessons from the more complex approaches taken in the USA where pollsters take into account not just people’s own estimates of their likelihood to vote, but also attitudinal factors like their interest in politics, whether they see voting as a duty, and so on.

In this case ICM asked a question on attitudes to voting, asking people whether it was their duty to vote, or if people should only vote if they cared who won, or whether it was really not worth doing at all. This was then cross referenced with the 1-10 likelihood to vote scale to produce a 30 cell matrix and people were weighted by the result. This appears to be a slightly harsher likelihood to vote filter – in the case of the Crewe and Nantwich poll it increased Conservative support by 1 point and reduced Labour by 1 point – but we won’t really be sure of the effect until we’ve seen it in action over a couple of polls.

20 Responses to “Changes to ICM’s methodology”

  1. a bit like mori icm seem to be out of line with other polling such as yougov or comres

  2. Second change sounds interesting, think they’ll fiddle with it as time goes on or are they taking an American method as writ?

  3. The second change is very interesting. Even though one may think that they are certain to vote (10 out of 10) in fact, on the day something can easily get in the way.

    These changes will have been forced on them by YouGov’s success.

  4. Simon Orr – lots of American pollsters have more complications methods of identifying likely voters than the normal UK method of just asking, but I don’t think ICM have just copied it wholesale. This is their own method, that just takes a similar approach to many companies in the USA.

  5. Does anyone know the [current] odds for a June election…?

  6. Perhaps they should just adjust by how much they got it wrong last time they predicted – this could be a rolling change – or maybe just make it up and therefore not waste time and money asking people

  7. Out of curiosity: I’d say I’m 10/10 certain to vote, I have never missed a vote and never would. I almost think its a duty to vote, but I would answer that I think you should only vote if you care (which I do and always will) – I don’t think those totally disinterested in politics and who don’t know anything (and don’t vote) should be deciding who runs our government and compelled to vote in ignorance like they are in Australia or due to a maligned sense of duty to vote, with no duty to get informed too.

    If I were polled and thus answered 10/10 certain but you should only vote if you care, would my vote be weighted below 1?

  8. Fluffy: A June election this year? About 0.00% give or take 0% and with a Margin of Error of 0.00%

  9. For all the criticism on this site about YouGov and the so called internet intellectuals who contribute to the YouGov surveys – they still seem to get it right on a lot more occasions than the competition.

    YouGov have set the trend once again for a big rethink amongst the old timer POLLING companies – and about time – EVEN I was predicting better than these companies (and i was getting paid)

  10. was’nt getting paid

  11. Fluffy – why do you ask about the possibility of an election? Any specific reason?

  12. I thought for a second you’d blown your cover there Mike!

  13. So differental turnout is still a problem. As ever. In the June 1970 election only one polling organisation-MORI- took into account the fact that Tory voters were more likely to vote than Labour voters and correctly predicted the election result whilst the other pollsters were famously miles out. NOP actually forecast a 11.4 Labour lead. Bar MORI the rest were not much better.
    As I recall pollsters went back and reinterviewed those who said they were likly to vote Labour and asked them why they did not . I was told by a researcher that some simply said that on the day they could not be bothered, some were secret Enoch Powell supporters, others changed their minds at the last moment and voted for another party but an extraordinary number denied that they had ever said that they were going to vote Labour in the first place!
    Perhaps it might be better if the Oracle did take over…

  14. I was polled by ICM earlier this evening and I wasn’t asked about my attitude towards voting apart from the standard likelihood to vote question. They could of course cross reference that with other answers but there was nothing as explicit as in the Crewe and Nantwich poll.

  15. Nick Sparrow told me they were planning to, so they probably haven’t rolled it out in time for this month.

  16. Slightly off topic, but I’m wondering what people would say if they were asked whether appearances of GB on the media made them more or less likely to vote Labour?

  17. I’m sceptical about all this, if only because i think that to some extent they’ve misdiagnosed the problem.

    Some pollsters will argue that the “turnout filter” is not even designed specifically to predict turnout, merely to establish a sample population which will not be significantly different in political complexion between those of the sample who ultimately decide to vote and those who don’t. So it doesn’t matter if you have a 45% turnout on a filter implying 60% as long as there is no difference (beyond statistical variation) between the 15% who ultimately don’t vote and the 45% who do.

    The problem regardless is that turnout is almost always “differential” dependent on political circumstances. Motivation among groups of political supporters will rise and fall with political events, even though throughout their theoretical attitude to voting may not change much at all. Perhaps the turnout question should be replaced by a “motivation” question?

  18. Anthony. I may have missed it- When approx. is the C&N result to be announced? Is anyone likely to do an exit poll?

  19. Jon H

    Fluffy – why do you ask about the possibility of an election? Any specific reason?

    I don’t know, just an inkling. What with:

    forty-two days detention,
    Gordon Brown’s demeanor on TV,
    likely outcome in Crewe and Nantwich by-election,
    sub-standard ministers who show their briefs,
    Irish EU-referendum and British Judicial Review,
    Mervyn King’s optimism, and
    the comments of some Labour supporters here.

    Why would any self-respecting human put themselves through such agony…?

  20. collin – probably the early hours of next Friday morning. An exit poll is very unlikely, they certainly haven’t been done for by-election for many, many years, if indeed they ever were (I can’t remember it ever happening, but people with longer memories than me might).