There are three new EU referendum polls today, a telephone poll from ORB and parallel phone and online polls from ICM.

The ORB poll for the Telegraph has topline figures of REMAIN 51%(-4), LEAVE 46%(+4), Don’t know 3%(nc) – full tables are here. The Telegraph wrote it up as immigration producing a significant swing towards Leave. I would normally have been ready to dismiss that as reading far too much into a poll that was probably just reversion to the mean: if you look at ORB’s previous figures their poll last week may have shown a thirteen point lead… but the two before that were almost identical to today’s figures. Newspapers may want to write that up as “big swing to Remain” followed by “big swing to Leave”, but a more parsimonious interpretation is just “slightly wonky sample returns to normal”.

However, today’s other polls from ICM raise more interesting questions.

ICM once again carried out two parallel polls, one conducted online, one conducted by telephone. So far all but one of these experiments have found a big gulf between phone and online polls (typically online polls show a race that’s neck-and-neck, telephone polls show a lead averaging around 8-10 points). ICM’s online poll found the sort of close race we’ve come to expect, with topline figures of REMAIN 44%(-1), LEAVE 47%(+2), Don’t know 9%(-1). ICM’s telephone poll found wholly unexpected results of REMAIN 42%(-5), LEAVE 45%(+6), Don’t know 13%(-1). Full tables for both are here.

There is nothing unusual about how the ICM poll was carried out that might explain the unusual result, it was done the same way as their previous telephone polls that showed Remain leads of seven or eight points. This leaves us with two obvious possibilities:

Either there has been a genuine movement towards Leave in recent days and the movement in the ICM poll reflects that (suggesting also that the ORB poll is showing more than just reversion to the mean), or…

ICM just got a really wacky sample by polling over the bank holiday weekend, and future telephone polls will revert to the normal pattern of solid remain leads.

Right now we can’t really tell. I will only urge my normal caveats about not reading too much into an individual poll, especially one conducted over a bank holiday weekend (whether or not that makes an actual difference or is “pollster folklore” is unclear. It’s certainly possible to point to clear examples of weird results from polls with bank holiday fieldwork, but it’s possible to point to weird results from polls conducted at other times and bank holiday results that are perfectly normal. While I’d always try to avoid polling over Easter or Christmas, I suspect in reality the Spring bank holiday doesn’t make that much difference to fieldwork). It will be interesting to see if forthcoming polls show a wider trend towards Leave, and if forthcoming telephone polls give any further suggestion that the online-phone gap may be disappearing.


ICM put out their weekly EU online poll today. Topline figures were REMAIN 45%, LEAVE 45%, DON’T KNOW 10% and tabs are here. ICM have tended to produce some of the most leave figures and the neck-and-neck result actually follows on from a series of polls showing a small leave lead, but this is due to a change in methodology.

As is often the case, ICM’s poll is actually less interesting than Martin Boon’s commentary that accompanies it – Martin’s response to the polling errors of last year has been one of the most candid and interesting of the pollsters, if occasionally one of the most pessimistic. In his article today he writes about what he considers to be the bleak future for phone polling given how people use the telephone these days, but also writes about the problems ICM have encountered moving most of their EU referendum polling to online.

Specifically Martin writes about how when ICM set online surveys live on Friday nights they get a rush of fast respondents that are skewed towards Leave. These entirely fill some of the demographic quotas set for the poll, meaning there is no room for the slower responding Remain respondents. To tackle this ICM have made two changes – one is to their sampling (they will spread it across the whole weekend, rather than opening it fully on Friday), the other is to weight respondents by how quickly they respond. According to Martin’s commentary the overall effect of this is to improve Remain’s relative position by four points.

I should point out (as Martin does in his article) that this is very much an issue to do with the way ICM carry out their online polls. Other online companies won’t necessarily do things the same way or face the same issues. I can only speak confidently about YouGov’s systems, but I know YouGov’s don’t invite respondents to specific surveys, they have a system that sends invites to respondents automatically, ensuring a constant flow of respondents into YouGov’s system. This means when people click on their invite (be it immediately, after a few hours, or days later) they are sent into whatever YouGov surveys are open at the time and need someone matching their demographics – hence when YouGov surveys go live they get a mixture of both fast respondents and slower respondents, who may have actually been invited before the survey was even written.

There was also a new contribution to the ongoing mystery of the gap between online and telephone polls on the EU referendum, this time from Prof Pat Sturgis (the Chair of the recent BPC/MRS inquiry into the polling error) and Prof Will Jennings (who served on the inquiry). It can be read in full here, but Pat and Will conclude that “While there are of course many caveats required here, this comparison suggests that the true picture may lie somewhere between the two modes, possibly somewhat closer to online. At the very least it suggests a good deal of caution is needed before concluding that one method is right and the other wrong. That will only be known for sure on June 24.”


-->

ComRes had a new EU telephone poll in this morning’s Daily Mail. Topline figures are REMAIN 52%(-1), LEAVE 41%(+3), Don’t know 7%(-2). Tabs are here.

Note that this poll is now adjusted for likelihood to vote, using ComRes’s turnout model based on socio-economic factors, like age and class (the changes are adjusted to reflect this). Note that adjusting turnout based on ComRes’s model has marginally increased support for Remain (before the adjustment the figures would have been 51 and 41).

There’s a broad assumption that differental turnout is more likely to favour Leave in the EU referendum campaign, largely based on the fact that polls normally show Leave voters claiming they are more likely to be 10/10 certain to vote, and that Leave voters are older. I’m not so sure. Self-reported likelihood is a blunt tool (people who say they are 10/10 certain to vote are not really much more likely than 8/10 or 9/10 people), and the age skew that should favour Leave in terms of turnout (older people are more likely to vote, and more Leave) will to some degree be cancelled out by the social class and educational skews that favour Remain (middle class people and graduates are more likely to vote, and more Remain).

On the subject of education, YouGov also had an interesting article up today. Like Populus and ICM they have carried out parallel telephone and online surveys, but unlike other such tests which have found a big gulf between phone and online results YouGov found results that were very similar to each other: both phone and online polls found a small lead for Leave.

This result wasn’t just the weighting (even before weighting the raw sample was a lot more “leave” than the raw samples from other phone polls) suggesting it is something to do with the sampling. Obviously we can’t tell for certain what the reason is – the most obvious difference is that the YouGov poll was conducted over the period of a fortnight, so was slower than most telephone polls and there was more opportunity to ring back people who were unavailable on the first call – but there could be other differences to do with quotas or the proportion of mobile calls (the YouGov poll was about a third mobile, two-thirds landline. My understanding is most phone polls are about 50/50 now, though MORI is about 20/80).

Looking at the actual demographics of the sample YouGov highlight the difference between their landline sample and the samples for the Populus paper looking at phone/online differences – specifically on education. In the Populus telephone samples between 44-46% of people had degrees, whereas the actual figure in the Census and Annual Population Survey is around 30%. The YouGov phone sample had a lower proportion of people with degrees to begin with, and weighted it to the national figure.

There is a clear correlation between education and attitudes to the EU referendum (in the YouGov polls there was a Leave lead of about 30 points among people who left school at 16 and a Remain lead of 33 points among those who were in educated beyond the age of twenty. This is partially to do with age, but it remains true even within people of the same age) so samples are too educated or not educated enough it could easily make a difference. As it is we’ve only got education data for the Populus polling – we don’t know if there’s the same skew in other phone polls, or how much of a difference it would make if corrected, but different levels of education within achieved samples is a further hypothesis that could explain that ongoing difference between phone and telephone samples for the EU referendum.


Earlier on today Ipsos MORI published their monthly poll for the Standard. Topline referendum figures were REMAIN 55%, LEAVE 37%, Don’t know or won’t vote 8%. This is the largest lead we’ve seen for Remain yet, but normal caveats apply – never get too excited about a single poll showing an unusual result. The ORB poll in the week also showed movement towards Remain, as did YouGov this morning… but the last TNS and ICM polls showed movement in the opposite direction, so there is no consistent trend.

Two interesting things about the MORI data. One is that the sheer size of the lead is down to the “squeeze question”, the question asked to those people who initially said don’t know asking which way they are most inclined to vote. Those people split 50%-14% in favour of remain, turning a 13 point lead for Remain into an 18 point lead.

The other thing is the Tories. Conservative voters in the sample split almost 2-to-1 in favour of Remain. Conservatives splitting in favour of remaining is not necessarily remarkable (the ORB and ICM telephone polls this week had the same), but the size of the Tory lead among Tory voters seems remarkable.

Meanwhile general election voting intention stands at CON 36%, LAB 34%, LDEM 8%, UKIP 10%. Full details are here.


YouGov have new EU voting intention figures in today’s Times – REMAIN 44%, LEAVE 40% (tabs). This is YouGov’s biggest lead for remain since March. Only one of YouGov’s last eight polls has shown Leave ahead, so while the regular ICM online polls have a consistent small lead for Leave, YouGov’s online polls are on average showing a small lead for Remain.

Part of today’s poll is due to a methodology change, reintroducing party ID as a weight in addition to past vote. While YouGov’s polls for Scottish, Welsh and London elections earlier this month generally performed very well, there was a consistent overstatement of UKIP support compared to what actually happened. Adding party ID weighting back in addresses that (essentially comparing the YouGov data to the British Election Study data suggested that even with past vote weighting, there were two few people who said they normally identified with “mainstream” parties like Lab & Con, and too many who normally identified with “challenger” parties like UKIP & Grn. This puts that right). The change does does favour Remain a little – in this poll it changed the topline figures by a point, moving it from a three point lead for Remain to a four point lead for Remain.

There was also a new ORB telephone poll yesterday for the Telegraph, which had topline figures of REMAIN 55%, LEAVE 40%, Don’t know 5% (details are here), and there are new Ipsos MORI figures to come in the Evening Standard later, which Ben Page says are “interesting”…