Today we got three more EU referendum polls.

A new YouGov poll for Good Morning Britain, conducted in the middle of last week, echoed the trend we’ve seen towards Leave. Their topline figures are REMAIN 41%, LEAVE 45%, Don’t know/Won’t vote 15%. Full tabs are here.

ICM’s weekly online poll has topline figures of REMAIN 43%(-1), LEAVE 48%(+1), Don’t knows 9%(nc). It’s no significant change from last week, but it consolidates last week’s leave lead. There’s no parallel ICM telephone poll this week. Full tabs for the online poll are here.

Finally there was a “new” TNS online online poll. The topline figures were REMAIN 41%, LEAVE 43%, Undecided 16%. This one is a little harder to interpret than the other two – TNS have made some changes to their methodology, including changing their past vote weighting and introducing turnout weighting and it’s not clear what impact the methodology change had, so we can’t be sure whether the polls suggests any movement in either direction – either way, the fieldwork was completed back in mid-May (full tabs are here).

All three polls show leave ahead, but all three polls were conducted online and most online polls show a close race anyway. What will be interesting is if either online polls do consistently start showing a clear lead for Leave rather than just movement around neck-and-neck, or if other telephone polls echo that ICM phone poll showing Leave ahead.


There are three new EU referendum polls today, a telephone poll from ORB and parallel phone and online polls from ICM.

The ORB poll for the Telegraph has topline figures of REMAIN 51%(-4), LEAVE 46%(+4), Don’t know 3%(nc) – full tables are here. The Telegraph wrote it up as immigration producing a significant swing towards Leave. I would normally have been ready to dismiss that as reading far too much into a poll that was probably just reversion to the mean: if you look at ORB’s previous figures their poll last week may have shown a thirteen point lead… but the two before that were almost identical to today’s figures. Newspapers may want to write that up as “big swing to Remain” followed by “big swing to Leave”, but a more parsimonious interpretation is just “slightly wonky sample returns to normal”.

However, today’s other polls from ICM raise more interesting questions.

ICM once again carried out two parallel polls, one conducted online, one conducted by telephone. So far all but one of these experiments have found a big gulf between phone and online polls (typically online polls show a race that’s neck-and-neck, telephone polls show a lead averaging around 8-10 points). ICM’s online poll found the sort of close race we’ve come to expect, with topline figures of REMAIN 44%(-1), LEAVE 47%(+2), Don’t know 9%(-1). ICM’s telephone poll found wholly unexpected results of REMAIN 42%(-5), LEAVE 45%(+6), Don’t know 13%(-1). Full tables for both are here.

There is nothing unusual about how the ICM poll was carried out that might explain the unusual result, it was done the same way as their previous telephone polls that showed Remain leads of seven or eight points. This leaves us with two obvious possibilities:

Either there has been a genuine movement towards Leave in recent days and the movement in the ICM poll reflects that (suggesting also that the ORB poll is showing more than just reversion to the mean), or…

ICM just got a really wacky sample by polling over the bank holiday weekend, and future telephone polls will revert to the normal pattern of solid remain leads.

Right now we can’t really tell. I will only urge my normal caveats about not reading too much into an individual poll, especially one conducted over a bank holiday weekend (whether or not that makes an actual difference or is “pollster folklore” is unclear. It’s certainly possible to point to clear examples of weird results from polls with bank holiday fieldwork, but it’s possible to point to weird results from polls conducted at other times and bank holiday results that are perfectly normal. While I’d always try to avoid polling over Easter or Christmas, I suspect in reality the Spring bank holiday doesn’t make that much difference to fieldwork). It will be interesting to see if forthcoming polls show a wider trend towards Leave, and if forthcoming telephone polls give any further suggestion that the online-phone gap may be disappearing.


-->

ICM put out their weekly EU online poll today. Topline figures were REMAIN 45%, LEAVE 45%, DON’T KNOW 10% and tabs are here. ICM have tended to produce some of the most leave figures and the neck-and-neck result actually follows on from a series of polls showing a small leave lead, but this is due to a change in methodology.

As is often the case, ICM’s poll is actually less interesting than Martin Boon’s commentary that accompanies it – Martin’s response to the polling errors of last year has been one of the most candid and interesting of the pollsters, if occasionally one of the most pessimistic. In his article today he writes about what he considers to be the bleak future for phone polling given how people use the telephone these days, but also writes about the problems ICM have encountered moving most of their EU referendum polling to online.

Specifically Martin writes about how when ICM set online surveys live on Friday nights they get a rush of fast respondents that are skewed towards Leave. These entirely fill some of the demographic quotas set for the poll, meaning there is no room for the slower responding Remain respondents. To tackle this ICM have made two changes – one is to their sampling (they will spread it across the whole weekend, rather than opening it fully on Friday), the other is to weight respondents by how quickly they respond. According to Martin’s commentary the overall effect of this is to improve Remain’s relative position by four points.

I should point out (as Martin does in his article) that this is very much an issue to do with the way ICM carry out their online polls. Other online companies won’t necessarily do things the same way or face the same issues. I can only speak confidently about YouGov’s systems, but I know YouGov’s don’t invite respondents to specific surveys, they have a system that sends invites to respondents automatically, ensuring a constant flow of respondents into YouGov’s system. This means when people click on their invite (be it immediately, after a few hours, or days later) they are sent into whatever YouGov surveys are open at the time and need someone matching their demographics – hence when YouGov surveys go live they get a mixture of both fast respondents and slower respondents, who may have actually been invited before the survey was even written.

There was also a new contribution to the ongoing mystery of the gap between online and telephone polls on the EU referendum, this time from Prof Pat Sturgis (the Chair of the recent BPC/MRS inquiry into the polling error) and Prof Will Jennings (who served on the inquiry). It can be read in full here, but Pat and Will conclude that “While there are of course many caveats required here, this comparison suggests that the true picture may lie somewhere between the two modes, possibly somewhat closer to online. At the very least it suggests a good deal of caution is needed before concluding that one method is right and the other wrong. That will only be known for sure on June 24.”


ICM have again conducted two parallel polls for the Guardian, one online, one by telephone (tabs). The pattern is the same as last month, on Westminster voting intention the two ICM polls show the same two point lead, although the ICM online poll has a higher level of UKIP support:

ICM Online – CON 34%, LAB 32%, LDEM 7%, UKIP 17%, GRN 4%
ICM Phone – CON 36%, LAB 34%, LDEM 7%, UKIP 13%, GRN 4%

For the EU referendum ICM have the typical phone vs online contrast. They have a eight point lead for Remain by phone, a four point lead for leave online… a twelve point gap (the average gap between online and telephone polls since the start of April is about 10 points, so ICM is a little larger, but nothing to write home about).

ICM Online – Remain 43%, Leave 47%, Don’t know 10%
ICM Phone – Remain 47%, Leave 39%, Don’t know 14%

Martin Boon’s own take over on the ICM website is, as usual, both honest and somewhat bemused: “The narrative that phone polls are more likely to be right ignores some fundamental flaws in phone methods. Labour supporters are continually oversampled by phone, and that may matter more than those same phone polls missing out on supposedly pro-Remain types, who are disproportionately less likely to turn out to vote. Similarly, what’s lurking under online covers could be equally nasty, and we should not ignore that the fact the UKIP voters are again, as they have long since been, higher in online polls than phone (or indeed at recent elections).”

Incidentally, it’s probably worth flagging up that there are house effects beyond just the phone/online difference. There are differences between different online pollsters too. This is ICM’s sixth online poll in a row to show Leave ahead, and they are clearly showing a small Leave lead. In contrast the majority of online polls conducted by YouGov and TNS over the last six weeks have had Remain very narrowly ahead, it’s not a big gap, but it’s starting to look consistent. When it actually comes to learning lessons from the EU referendum, these smaller differences may end up being the more valuable: without much fuss, pollsters are taking quite different approaches to correcting their methods after last year and the referendum may teach us something useful about what corrections are (or are not) working for online; what corrections are (or are not) working for telephone.

Methodological concerns aside, what does ICM tell us about the state of public opinion? Well both their phone and online polls have the gap between Tory and Labour narrowing, down from five point leads a month ago. In the referendum race the four point leave lead in the online poll is ICM’s largest this year… but that trend isn’t echoed in the phone poll. We shall see if other EU polling this week shows any coherent trend.

There was also a new ComRes online poll at the weekend for the Indy and Sunday Mirror. This had topline figures of CON 36%(+1), LAB 30%(nc), LDEM 8%(nc), UKIP 17%(+1), GRN 4%(nc). On the face of it this is a stronger poll for the Tories, but this is largely methodological – ComRes’s online polls tend to produce the most positive results for the Tories of any company because of their demographic based turnout model. Full tabs are here.


There are three EU polls in the Sunday papers.

  • An online Opinium poll for the Observer had topline figures of REMAIN 42%, LEAVE 41%, DON’T KNOW 14%. The one point lead for remain compares to a four point leave lead a month ago (tabs).
  • An online ORB poll for the Independent had topline figures of 50% REMAIN, 50% LEAVE without turnout, REMAIN 49%, LEAVE 51% once weighted for turnout (the previous ORB online poll a month ago had a break of Remain 51%, Leave 49%, but didn’t account for turnout) (tabs)
  • An online ICM poll in the Sun on Sunday had toplines of 43% REMAIN, 46% LEAVE, DON’T KNOW 11%. These are almost unchanged from the ICM poll in the week, which had figures of 44% remain and 46% leave.

Three online polls, all showing the extremely close referendum race that online polling has been consistently showing. The Opinium poll also had some intriguing Westminster voting intention figures: CON 38%, LAB 30%, LDEM 5%, UKIP 15%, GRN 5%. An eight point lead for the Conservatives is the largest any poll has shown since before the budget, and is an increase of seven points since Opinium’s last poll. The Tues-Fri fieldwork period overlapped with Labour’s anti-Semitism row, so it could be that it has dented Labour’s support… but it is only one poll, so wait to see if other polling echoes it. (Interestingly the tables for the Opinium poll have the voting intention question at the end, after a question about who people would trust on the economy. If that actually is the order the questions were asked it that could have potentially affected responses as well.)

UPDATE: Ignore the strange question ordering in the Opinium tables – the questions were actually asked in the normal order, with voting intention at the start