ICM have again conducted two parallel polls for the Guardian, one online, one by telephone (tabs). The pattern is the same as last month, on Westminster voting intention the two ICM polls show the same two point lead, although the ICM online poll has a higher level of UKIP support:

ICM Online – CON 34%, LAB 32%, LDEM 7%, UKIP 17%, GRN 4%
ICM Phone – CON 36%, LAB 34%, LDEM 7%, UKIP 13%, GRN 4%

For the EU referendum ICM have the typical phone vs online contrast. They have a eight point lead for Remain by phone, a four point lead for leave online… a twelve point gap (the average gap between online and telephone polls since the start of April is about 10 points, so ICM is a little larger, but nothing to write home about).

ICM Online – Remain 43%, Leave 47%, Don’t know 10%
ICM Phone – Remain 47%, Leave 39%, Don’t know 14%

Martin Boon’s own take over on the ICM website is, as usual, both honest and somewhat bemused: “The narrative that phone polls are more likely to be right ignores some fundamental flaws in phone methods. Labour supporters are continually oversampled by phone, and that may matter more than those same phone polls missing out on supposedly pro-Remain types, who are disproportionately less likely to turn out to vote. Similarly, what’s lurking under online covers could be equally nasty, and we should not ignore that the fact the UKIP voters are again, as they have long since been, higher in online polls than phone (or indeed at recent elections).”

Incidentally, it’s probably worth flagging up that there are house effects beyond just the phone/online difference. There are differences between different online pollsters too. This is ICM’s sixth online poll in a row to show Leave ahead, and they are clearly showing a small Leave lead. In contrast the majority of online polls conducted by YouGov and TNS over the last six weeks have had Remain very narrowly ahead, it’s not a big gap, but it’s starting to look consistent. When it actually comes to learning lessons from the EU referendum, these smaller differences may end up being the more valuable: without much fuss, pollsters are taking quite different approaches to correcting their methods after last year and the referendum may teach us something useful about what corrections are (or are not) working for online; what corrections are (or are not) working for telephone.

Methodological concerns aside, what does ICM tell us about the state of public opinion? Well both their phone and online polls have the gap between Tory and Labour narrowing, down from five point leads a month ago. In the referendum race the four point leave lead in the online poll is ICM’s largest this year… but that trend isn’t echoed in the phone poll. We shall see if other EU polling this week shows any coherent trend.

There was also a new ComRes online poll at the weekend for the Indy and Sunday Mirror. This had topline figures of CON 36%(+1), LAB 30%(nc), LDEM 8%(nc), UKIP 17%(+1), GRN 4%(nc). On the face of it this is a stronger poll for the Tories, but this is largely methodological – ComRes’s online polls tend to produce the most positive results for the Tories of any company because of their demographic based turnout model. Full tabs are here.


109 Responses to “ICM parallel phone and online polls”

1 2 3
  1. Hmm. If the onlines are right in detecting more ‘kippers/leaves, this means there were a lot of ‘kippers who didn’t bother turning out for the GE but who will for the referendum. (‘Kipper count was a big online/phone difference at the GE, and their poll rating wasn’t delivered) I find this a complicated story that has to happen for them to be right. Selectively Shy Kippers.

  2. Meanwhile, inside No.10 Downing St, they think there is a 58% vote for Remain.

  3. The interesting thing about the GE Voting Intention polls is UKIP’s 17% online and 13% by phone. Given their performance in last week’s local elections, and the clear trend that online polls have overstated them in a variety of different UK elections, surely the phone poll’s 13% is probably the more accurate.

    But this of course doesn’t come near to explaining the 12% net difference between the two modes in the EURef poll results.

  4. Alex – UKIP at the last election was even more complicated. There was an obvious phone/online divide, but UKIP’s final score was somewhere in between those figures… and there was variation between the modes, so there were both phone and online polls who got UKIP correct. It was only really Survation and Panelbase who were out on UKIP by the end, YouGov, Opinium, BMG, TNS & Populus all had them right.

  5. James E –

    don’t do that! Compare UKIP score in the polls and their results in the London, Scottish and Welsh elections by all means (it does look like the polls had them too high), but don’t compare them to local results. People vote differently in local elections and there were no local elections polls to compare to (I remember having to repeat this point ad infinitum to Lib Dem supporters convinced local election results meant the polls were underestimating their support).

  6. I had to write this eventually http://ukpollingreport.co.uk/blog/archives/5439.

    The R&T and BBC vote share projections are what they think the shares would be if everywhere was voting in a local election, not a general election, and people *really* do vote differently.

  7. @ Anthony Wells

    Sorry!!
    I had just read James Kanagasooriam make the same comparison on twitter (“UKIP @ 1.4X their equivalent vote share”) but I’d happily amend my last post to say ‘last week’s elections’ rather than ‘last week’s local elections’.

    If you could point me to the post-mortem regarding UKIP and the online/phone dichotomy at last year’s GE, I’d be very grateful.

  8. Anthony,

    The best comparable elections to the EU referendum must be the EP elections in 2009 and 2014.

    Do you have any figures to show the difference between final polls and actual results in those two elections ?

    I can see the ladt polls in the link on the right, but not the results.

    IIRC, the polls did not overstate UKIP’s actual result.

  9. Paul H-J

    “The best comparable elections to the EU referendum must be the EP elections in 2009 and 2014.”

    Why?

  10. @ Paul H-J

    The post-mortem AW did in May 2014 showed that UKIP was on average overstated by 2.5% across 6 pollsters’ final polls, achieving 27.5% against the pollsters’ average of 30%.

    http://ukpollingreport.co.uk/blog/archives/8845#comments

    Note that while ICM did understate UKIP, their final poll was done the longest time from the election day.

  11. Another EU Phone Poll:

    A 15 point lead for Remain in the Telegraph.

  12. ORB/Telegraph (#EURef):

    REMAIN 55 (+4)
    LEAVE 40 (-3)

    The largest Remain lead we’ve seen for nearly 3 months.

  13. @Alun

    Have replied on previous thread. Much of what you wrote involved projecting views on me I dont hold and haven’t argued for, so it was a bit weird to read your post, like it was aimed at someone else entirely, but at least it makes for an easy, straightforward response.

  14. “Meanwhile, inside No.10 Downing St, they think there is a 58% vote for Remain.”

    ———–

    I dunno, they seem pretty desperate to whoever they can out in support. They haven’t summoned the ghosts of Macmillan or Heath yet, but there’s still time…

  15. I apologise if this point has already been considered or addressed (I admit I do not follow polling as keenly as some here) but I wonder how the differences between a general election and a referendum will effect the accuracy of the polling.

    What I mean by this is that in an online poll, someone might say they would vote UKIP, and vote Leave, but in an actual general election they think “It’s Labour vs. Conservative in my constituency so I will vote Conservative” yet would still vote to Leave in a referendum. This might be a possible explanation of why online polls tend to over-state UKIP support, and if this were the case would suggest Leave were doing better than people think.

  16. @Steve
    ‘What I mean by this is that in an online poll, someone might say they would vote UKIP, and vote Leave, but in an actual general election they think “It’s Labour vs. Conservative in my constituency so I will vote Conservative” yet would still vote to Leave in a referendum. This might be a possible explanation of why online polls tend to over-state UKIP support, and if this were the case would suggest Leave were doing better than people think.’

    Can see your reasoning for the first part of your post but struggling with the second part. People are asked in opinion polls if they would vote stay or remain, a straight forward binary choice, why would the over representation of UKIP. or any other Political Party for that matter, in polls make a difference to that?

  17. There is clearly an issue with polling methods and this latest brace show just how unreliable they have become. Are there any old fashioned polls left which involve knocking on doors? It would be instructive to see if that produces a third result which in the end was more reliable. What does seem undeniable is that the polling industry is not producing reliable results, even if the question is a simple yes or no. Somewhere, in the race to cut costs, the baby was thrown out with the bathwater.

  18. I think I explained myself badly. What I meant is, some people say that ‘online polls are over-representing UKIP voters, and as UKIP voters are (almost) certainly Leave voters, online polls are also over-representing Leave’ . And that’s exactly what I was trying to say, that what if the online polls are broadly correct and the problem with previous over-representation of UKIP was down to constituency effects.

    I was just thinking about this after reading James Forsyth in the Spectator state ” Phone polls are generally regarded as slightly superior to online ones,” and was thinking to what extent that is actually true.

  19. I have to admit to being less than confident that the polls, either phone or online, are at all well placed to predict an accurate result in the referendum. Weighting according to political persuasion may not work in this instance. The turnout demographics may not apply here. The only sort of weighting that might work is weighting to people’s degree of political engagement, which is probably impossible to do. I would not really blame the polls if they are miles out this time.

  20. Could the weather affect the outcome? That latest Telegraph poll shows the Remain lead at only 6% when accounting for only those who are certain to vote. So a nice sunny day on 23rd June may well bring a higher turnout and help the Remain cause. If it pees down with rain be prepared for Brexit.

  21. CARPFREW:
    You misrepresent my views and fail to read my posts properly, so quit whining.

  22. @STEVE
    ‘I think I explained myself badly. What I meant is, some people say that ‘online polls are over-representing UKIP voters, and as UKIP voters are (almost) certainly Leave voters, online polls are also over-representing Leave’ . And that’s exactly what I was trying to say, that what if the online polls are broadly correct and the problem with previous over-representation of UKIP was down to constituency effects.’

    Understand now cheers

  23. The online v phone gap re: the referendum is easily enough explained.

    Online is well known to be a bastion of the ‘grumpy old man’. And it is that same group who are most anti-EU.

    Phones tend to be used by younger people who are, as they still believe in life and hope and have a career, more pro-EU.

    When it comes to the actual vote, the former group are by and away the more likey to go out and vote. So the UK WILL vote to leave.

    In other words, the online polls – in respect of the referndum – are the more accurate.

  24. Does anyone know how or whether expat voters entitled to vote in the referendum are included in polls? Presumably they could be included in online polls but less likely in telephone polls? In a close result could their turnout and votes be important, almost like an additional region?

  25. I’m not expecting too much accuracy in the polls either and that doesn’t help when they’re fairly tight as they seem to be here.

    As for local elections, they just tend to provide a sort of supporting impression of where a particular party is.

  26. @DAVID IN FRANCE

    Online is well known to be a bastion of the ‘grumpy old man’. And it is that same group who are most anti-EU.

    Phones tend to be used by younger people who are, as they still believe in life and hope and have a career, more pro-EU.

    Do you have any evidence for this?

  27. Anthony Wells
    Can you point me to anywhere that gives information about how the various pollsters are modelling referendum turnout? Seems that this may be critical to the outcome so it would be nice to have a bit more detail. Is ‘passion’ about the issue a good predictor of voting behaviour or are traditional demographic factors better? Your article also makes tantalising reference to divergence in how they’ve gone about trying to correct for the errors at GE2015 – anything further on this? One of your polling post mortem pieces also mentioned that MORI was experimenting with regularity of voting in its turnout model – do you know any more about that? Are they using it in their referendum modelling?

    Hireton
    Don’t know about the ex-pats. It’s a good point. I strongly suspect they’re not being polled and are probably under-represented in online samples, but I’d like to think that the pollsters have at least gestured towards recognising them as a separate bloc and come up with some sort of model for ‘predicting’ how much more/less pro-Remain they are than domestic voters and perhaps how likely to turn out (although I suspect that’ll amount to an educated guess because for ex-pats the referendum really isn’t like the GE – I’m assuming they’ll be more likely to vote than they are in GEs, but I’ve no idea what typical ex-pat turnout in a GE is).

  28. Squeeky bum time all round until that exit poll in June I reckon.

  29. As others have said in previous threads, the great unknown is turnout. Prior to the local elections our local Labour party had dozens of volunteers working daily for some weeks. Previous to that there was a big focus on student voter registration because of potential boundary changes.

    In comparison there is zero canvassing and activity related to the EU vote. When I asked why this was the case the response was that the CLP had different views so the decision was made not to back one or other campaign. Whether this makes a huge impact on voter turnout remains to be seen.

  30. @ Rich

    There isn’t going to be a published exit poll on 23rd June, as there isn’t any past data on which the pollsters could base their sampled polling stations. I think that the best clue on the day as to the result will be the direction of movement of Sterling: it will be rising sharply if we’re staying, and falling if the markets believe we’re leaving.

    @ David In France
    “Phones tend to be used by younger people…”

    A surprisingly large number of older people use phones too!
    :-)

  31. In the meantime the YouGov poll of Labour Party members (it’s not yet up, and today’s Times reports on it) suggests that Corbyn’s popularity in the LP increased (fully paid members), and improved all indicators, including the likelihood of winning the next GE.
    (Sorry read it only at the news agent, so this is all I remember).

    Found it surprising.

    Would his vaguely more participation damage his standing in the LP, help the Remain?

    I don’t think either is likely, although a Brexit outcome might effect his standing.

  32. @ David In France etc
    “Phones tend to be used by younger people…”

    I thought polling companies had techniques for selection and adjustment to ensure their samples matched the population? Applied to both methods, presumably?
    Thus either this discussion is irrelevant, or these adjustments don’t work.
    As the results of online and phone are so different, the latter seems to be the case.

  33. @ Colin

    Thank you.

  34. JAMES E
    “the direction of movement of Sterling”

    Genuine question: why do you think currency traders will have a better idea of the coming result?

  35. David in France

    “younger people who are, as they still believe in life and hope ”

    I may be 76 but i still believe in life and hope and i will be voting to leave the EU with real hope in my heart.

  36. TOH

    @”I may be 76 but i still believe in life and hope”

    :-) :-) :-)

  37. Good afternoon all from a warm but cloudy central London.

    DAVID IN FRANCE…

    Rubbish..
    ………
    RICH
    “Squeeky bum time all round until that exit poll in June I reckon”
    ___

    Talking of squeaky bum time, has your bum come off the fence yet? ;-)

  38. THE OTHER HOWARD

    “I may be 76 but i still believe in life and hope and i will be voting to leave the EU with real hope in my heart”
    ______

    Hear hear and may I commend your post to the house my fellow Brexiter. :-)

  39. The fact that there isn’t going to be an exit poll (thanks for that info, James E) tells us that the pollsters really aren’t confident that they understand how all the standard demographic factors relate to referendum voting behaviour, which means that they can’t confident that they’ve got their sample weighting right…

    Of course the exit poll has a pretty good track record, so they might be being cautious because they don’t want to risk its reputation.

    And I’m assuming that potential clients think the outcome is important enough and/or uncertain enough to warrant the expenditure.

    I’d like to think that a bunch of academics will give it a go, just to see what can be learned about the differences between voting in an explicitly party political context and voting in a slightly different political context (establishment vs. rebels?).

  40. HIRETON

    Does anyone know how or whether expat voters entitled to vote in the referendum are included in polls? Presumably they could be included in online polls but less likely in telephone polls? In a close result could their turnout and votes be important, almost like an additional region?

    It’s more possible they could be included in online polls by registering for them, though most pollsters like to check postcode and so on to make sure people are UK-resident for the purposes of commercial polling. Phone polls are random and ringing up the entire population of the world in the hope of finding a miniscule proportion of expats would be an even more thankless task than they already have.

    In truth there aren’t that many expats who are registered to vote. There were only 106,000 at the last general election:

    https://www.gov.uk/government/world-location-news/too-many-uk-expats-are-still-not-referendumready

    which as a proportion of an electorate in 2015 of 46.4 million is only 0.2%. So it’s hardly a separate region (unless the region is the Channel Islands). Because they are by definition postal voters, turnout might be higher than for other groups (86% of postals voted in 2015 compared to 63.5% of those without), though practicalities might reduce that.

  41. I was trying to think what big powerful gun the remain side would shuffle out this week and thought of Ed Milliband.

    “This time last year Ed Miliband thought he was about to become the most powerful person in Britain – now he’s only the 40th most powerful person in Doncaster”

    “The former Labour leader has tumbled down the ‘Doncaster Power List’, behind such civic giants as a director at a chartered accountants Allots, and two providers of compliance training materials”
    ___

    Hmm….. perhaps not.. )-:

  42. @Alun

    “You misrepresent my views and fail to read my posts properly….”

    —————

    Lol, that really ain’t gonna work….

  43. “Talking of squeaky bum time, has your bum come off the fence yet?”

    ————

    Dunno about Rich, but mine hasn’t. I dunno how peeps can be so definite on the matter, the whole thing seems shrouded in so many interacting indeterminate things…

  44. @Allan C

    Well they shuffled out Ed Balls AND Vince Cable yesterday, flanking the Chancellor. I was like, what’s next? Nick Clegg?…

  45. @Roger Mexico

    Thanks for the info.

  46. @ Syzygy

    Yes, I hadn’t thought of it in terms of rushing to get it done before the next Crunch. Yes, the agenda is very hidden, hence all the secrecy over the trade deals.

    I think yes, they do want a trade block to rival the big emerging economies, and indeed there’s a bit of a battle going on down the line, where we are engaged in a race to grow our population and hence economy in order to try and hold our position as much as possible.

    Your average Libertarian doesn’t like States having power, and they think by hammering the State, the market will take over and – consumer Power!! – will ensure fair play.

    They don’t seem to realise that as you hand more power to the “market”, the corporates Hoover up and fill the power vacuum, and keep a state going anyway, but bent more to their will, whether it’s stopping regulations to ensure safety, or ensuring there is more immigration, and much else besides…

  47. CARFREW

    “Dunno about Rich, but mine hasn’t. I dunno how peeps can be so definite on the matter, the whole thing seems shrouded in so many interacting indeterminate things”
    _____

    I made my mine up quite a while back and it’s mostly based on accountability and sovereignty. I can’t see why a county such as the UK has to be part of some grand French and German project to fully integrate us all into one big super state and hand more and more powers over to a European capital named after a tasteless vegetable.

    I know there is a lot of good the EU has done but it’s not for me and I’m very uncertain to what a remain vote means for the UK in the future. Projects like the EU grow tentacles and get out of hand.

  48. CARFREW
    @Allan C
    Well they shuffled out Ed Balls AND Vince Cable yesterday, flanking the Chancellor. I was like, what’s next? Nick Clegg?
    _____

    Nick who? I laughed when I saw Balls and ol Yoda flank Ozzie because the both of them spent 5 years rubbishing the chancellor and now they are standing shoulder to shoulder with him. I have to admit, this whole EU referendum is throwing up some very obscure alliances.

  49. Well, Cameron and Sturgean at least share one thing in common – they both (temporarily and dishonestly, in Sturgeon’s case) want us to think that there is only one chance at the big vote.

    In Scotland today I was amused to see a Yes2 shop, so clearly the campaign for a second referendum is well underway, but in a rare divergence from the format of the two referendums, today we have Farage on the Leave side talking about the possibility of a second vote, with Cameron firmly trying to squash this. The ‘once in a lifetime chance’ line tends to be the preserve of the change candidates, so it’s an interesting flip in the dynamic.

    As with Scotland, it’s nonsense. For the Indyref, everyone and their dog knew that the SNP would keep trying if they lost, with the only metric being if they thought they might win the next vote. Likewise, a narrow Yes vote would lead to pressures for a second confirming vote as sure as night follows day.

    For the EU vote, I think we have something similar. Leave won’t go away after a remain vote, unless it’s a huge margin, and if there is a Leave vote, remain will be hoping for instant dislocation impacts and a second bite once the future starts to take shape.

1 2 3