ICM put out their weekly EU online poll today. Topline figures were REMAIN 45%, LEAVE 45%, DON’T KNOW 10% and tabs are here. ICM have tended to produce some of the most leave figures and the neck-and-neck result actually follows on from a series of polls showing a small leave lead, but this is due to a change in methodology.

As is often the case, ICM’s poll is actually less interesting than Martin Boon’s commentary that accompanies it – Martin’s response to the polling errors of last year has been one of the most candid and interesting of the pollsters, if occasionally one of the most pessimistic. In his article today he writes about what he considers to be the bleak future for phone polling given how people use the telephone these days, but also writes about the problems ICM have encountered moving most of their EU referendum polling to online.

Specifically Martin writes about how when ICM set online surveys live on Friday nights they get a rush of fast respondents that are skewed towards Leave. These entirely fill some of the demographic quotas set for the poll, meaning there is no room for the slower responding Remain respondents. To tackle this ICM have made two changes – one is to their sampling (they will spread it across the whole weekend, rather than opening it fully on Friday), the other is to weight respondents by how quickly they respond. According to Martin’s commentary the overall effect of this is to improve Remain’s relative position by four points.

I should point out (as Martin does in his article) that this is very much an issue to do with the way ICM carry out their online polls. Other online companies won’t necessarily do things the same way or face the same issues. I can only speak confidently about YouGov’s systems, but I know YouGov’s don’t invite respondents to specific surveys, they have a system that sends invites to respondents automatically, ensuring a constant flow of respondents into YouGov’s system. This means when people click on their invite (be it immediately, after a few hours, or days later) they are sent into whatever YouGov surveys are open at the time and need someone matching their demographics – hence when YouGov surveys go live they get a mixture of both fast respondents and slower respondents, who may have actually been invited before the survey was even written.

There was also a new contribution to the ongoing mystery of the gap between online and telephone polls on the EU referendum, this time from Prof Pat Sturgis (the Chair of the recent BPC/MRS inquiry into the polling error) and Prof Will Jennings (who served on the inquiry). It can be read in full here, but Pat and Will conclude that “While there are of course many caveats required here, this comparison suggests that the true picture may lie somewhere between the two modes, possibly somewhat closer to online. At the very least it suggests a good deal of caution is needed before concluding that one method is right and the other wrong. That will only be known for sure on June 24.”


ComRes had a new EU telephone poll in this morning’s Daily Mail. Topline figures are REMAIN 52%(-1), LEAVE 41%(+3), Don’t know 7%(-2). Tabs are here.

Note that this poll is now adjusted for likelihood to vote, using ComRes’s turnout model based on socio-economic factors, like age and class (the changes are adjusted to reflect this). Note that adjusting turnout based on ComRes’s model has marginally increased support for Remain (before the adjustment the figures would have been 51 and 41).

There’s a broad assumption that differental turnout is more likely to favour Leave in the EU referendum campaign, largely based on the fact that polls normally show Leave voters claiming they are more likely to be 10/10 certain to vote, and that Leave voters are older. I’m not so sure. Self-reported likelihood is a blunt tool (people who say they are 10/10 certain to vote are not really much more likely than 8/10 or 9/10 people), and the age skew that should favour Leave in terms of turnout (older people are more likely to vote, and more Leave) will to some degree be cancelled out by the social class and educational skews that favour Remain (middle class people and graduates are more likely to vote, and more Remain).

On the subject of education, YouGov also had an interesting article up today. Like Populus and ICM they have carried out parallel telephone and online surveys, but unlike other such tests which have found a big gulf between phone and online results YouGov found results that were very similar to each other: both phone and online polls found a small lead for Leave.

This result wasn’t just the weighting (even before weighting the raw sample was a lot more “leave” than the raw samples from other phone polls) suggesting it is something to do with the sampling. Obviously we can’t tell for certain what the reason is – the most obvious difference is that the YouGov poll was conducted over the period of a fortnight, so was slower than most telephone polls and there was more opportunity to ring back people who were unavailable on the first call – but there could be other differences to do with quotas or the proportion of mobile calls (the YouGov poll was about a third mobile, two-thirds landline. My understanding is most phone polls are about 50/50 now, though MORI is about 20/80).

Looking at the actual demographics of the sample YouGov highlight the difference between their landline sample and the samples for the Populus paper looking at phone/online differences – specifically on education. In the Populus telephone samples between 44-46% of people had degrees, whereas the actual figure in the Census and Annual Population Survey is around 30%. The YouGov phone sample had a lower proportion of people with degrees to begin with, and weighted it to the national figure.

There is a clear correlation between education and attitudes to the EU referendum (in the YouGov polls there was a Leave lead of about 30 points among people who left school at 16 and a Remain lead of 33 points among those who were in educated beyond the age of twenty. This is partially to do with age, but it remains true even within people of the same age) so samples are too educated or not educated enough it could easily make a difference. As it is we’ve only got education data for the Populus polling – we don’t know if there’s the same skew in other phone polls, or how much of a difference it would make if corrected, but different levels of education within achieved samples is a further hypothesis that could explain that ongoing difference between phone and telephone samples for the EU referendum.


-->

Earlier on today Ipsos MORI published their monthly poll for the Standard. Topline referendum figures were REMAIN 55%, LEAVE 37%, Don’t know or won’t vote 8%. This is the largest lead we’ve seen for Remain yet, but normal caveats apply – never get too excited about a single poll showing an unusual result. The ORB poll in the week also showed movement towards Remain, as did YouGov this morning… but the last TNS and ICM polls showed movement in the opposite direction, so there is no consistent trend.

Two interesting things about the MORI data. One is that the sheer size of the lead is down to the “squeeze question”, the question asked to those people who initially said don’t know asking which way they are most inclined to vote. Those people split 50%-14% in favour of remain, turning a 13 point lead for Remain into an 18 point lead.

The other thing is the Tories. Conservative voters in the sample split almost 2-to-1 in favour of Remain. Conservatives splitting in favour of remaining is not necessarily remarkable (the ORB and ICM telephone polls this week had the same), but the size of the Tory lead among Tory voters seems remarkable.

Meanwhile general election voting intention stands at CON 36%, LAB 34%, LDEM 8%, UKIP 10%. Full details are here.


YouGov have new EU voting intention figures in today’s Times – REMAIN 44%, LEAVE 40% (tabs). This is YouGov’s biggest lead for remain since March. Only one of YouGov’s last eight polls has shown Leave ahead, so while the regular ICM online polls have a consistent small lead for Leave, YouGov’s online polls are on average showing a small lead for Remain.

Part of today’s poll is due to a methodology change, reintroducing party ID as a weight in addition to past vote. While YouGov’s polls for Scottish, Welsh and London elections earlier this month generally performed very well, there was a consistent overstatement of UKIP support compared to what actually happened. Adding party ID weighting back in addresses that (essentially comparing the YouGov data to the British Election Study data suggested that even with past vote weighting, there were two few people who said they normally identified with “mainstream” parties like Lab & Con, and too many who normally identified with “challenger” parties like UKIP & Grn. This puts that right). The change does does favour Remain a little – in this poll it changed the topline figures by a point, moving it from a three point lead for Remain to a four point lead for Remain.

There was also a new ORB telephone poll yesterday for the Telegraph, which had topline figures of REMAIN 55%, LEAVE 40%, Don’t know 5% (details are here), and there are new Ipsos MORI figures to come in the Evening Standard later, which Ben Page says are “interesting”…


ICM have again conducted two parallel polls for the Guardian, one online, one by telephone (tabs). The pattern is the same as last month, on Westminster voting intention the two ICM polls show the same two point lead, although the ICM online poll has a higher level of UKIP support:

ICM Online – CON 34%, LAB 32%, LDEM 7%, UKIP 17%, GRN 4%
ICM Phone – CON 36%, LAB 34%, LDEM 7%, UKIP 13%, GRN 4%

For the EU referendum ICM have the typical phone vs online contrast. They have a eight point lead for Remain by phone, a four point lead for leave online… a twelve point gap (the average gap between online and telephone polls since the start of April is about 10 points, so ICM is a little larger, but nothing to write home about).

ICM Online – Remain 43%, Leave 47%, Don’t know 10%
ICM Phone – Remain 47%, Leave 39%, Don’t know 14%

Martin Boon’s own take over on the ICM website is, as usual, both honest and somewhat bemused: “The narrative that phone polls are more likely to be right ignores some fundamental flaws in phone methods. Labour supporters are continually oversampled by phone, and that may matter more than those same phone polls missing out on supposedly pro-Remain types, who are disproportionately less likely to turn out to vote. Similarly, what’s lurking under online covers could be equally nasty, and we should not ignore that the fact the UKIP voters are again, as they have long since been, higher in online polls than phone (or indeed at recent elections).”

Incidentally, it’s probably worth flagging up that there are house effects beyond just the phone/online difference. There are differences between different online pollsters too. This is ICM’s sixth online poll in a row to show Leave ahead, and they are clearly showing a small Leave lead. In contrast the majority of online polls conducted by YouGov and TNS over the last six weeks have had Remain very narrowly ahead, it’s not a big gap, but it’s starting to look consistent. When it actually comes to learning lessons from the EU referendum, these smaller differences may end up being the more valuable: without much fuss, pollsters are taking quite different approaches to correcting their methods after last year and the referendum may teach us something useful about what corrections are (or are not) working for online; what corrections are (or are not) working for telephone.

Methodological concerns aside, what does ICM tell us about the state of public opinion? Well both their phone and online polls have the gap between Tory and Labour narrowing, down from five point leads a month ago. In the referendum race the four point leave lead in the online poll is ICM’s largest this year… but that trend isn’t echoed in the phone poll. We shall see if other EU polling this week shows any coherent trend.

There was also a new ComRes online poll at the weekend for the Indy and Sunday Mirror. This had topline figures of CON 36%(+1), LAB 30%(nc), LDEM 8%(nc), UKIP 17%(+1), GRN 4%(nc). On the face of it this is a stronger poll for the Tories, but this is largely methodological – ComRes’s online polls tend to produce the most positive results for the Tories of any company because of their demographic based turnout model. Full tabs are here.