ORB have a new poll out tonight for the Independent showing a ten point lead for leave: REMAIN 45%(-4), LEAVE 55%(+4). Changes are since their last comparable poll, all the way back in April. Unlike the weekly ORB telephone polls for the Telegraph, their more infrequent polls for the Indy are done online – hence the results that are far more pro-Brexit than their poll in the week. Even accounting for that, it shows a shift towards leave that we’ve seen in many recent polls.

The ten point lead is large, but as ever, it is only one poll. Don’t read too much into it unless we see it echoed in other polling. As things stand most other online polls are still tending to show a relatively close race between Remain and Leave.

Also out today was a statement on some methodology changes from Ipsos MORI. As well as following their normal pre-election practice of filtering out people who aren’t registered to vote now the deadline for registration has passed, from their poll next week they are also going to start quotaing and weighting by education, aimed at reducing an over-representation of graduates. MORI suggest that in their last poll the change would have reduced the Remain lead by 3 or 4 points.

While they haven’t yet decided how they’ll do it, in their article they also discuss possible approaches they might take on turnout. MORI have included examples of modelling turnout based on people who say they are certain to vote and voted last time, or say the referendum is important, or who say they usually vote and so on. Exactly which one MORI end up opting for probably doesn’t make that much difference, they all have a very similar impact, reducing the Remain share by a couple of point, increasing the Leave share by a couple of points.

The combined effect of these changes is that the MORI poll in the week is going to be better for Leave due to methodological reasons anyway. If it does show another shift towards Leave, take care to work out how much of that is because of the methodology change and how much of it is due to actual movement before getting too excited/distraught.


Today we got three more EU referendum polls.

A new YouGov poll for Good Morning Britain, conducted in the middle of last week, echoed the trend we’ve seen towards Leave. Their topline figures are REMAIN 41%, LEAVE 45%, Don’t know/Won’t vote 15%. Full tabs are here.

ICM’s weekly online poll has topline figures of REMAIN 43%(-1), LEAVE 48%(+1), Don’t knows 9%(nc). It’s no significant change from last week, but it consolidates last week’s leave lead. There’s no parallel ICM telephone poll this week. Full tabs for the online poll are here.

Finally there was a “new” TNS online online poll. The topline figures were REMAIN 41%, LEAVE 43%, Undecided 16%. This one is a little harder to interpret than the other two – TNS have made some changes to their methodology, including changing their past vote weighting and introducing turnout weighting and it’s not clear what impact the methodology change had, so we can’t be sure whether the polls suggests any movement in either direction – either way, the fieldwork was completed back in mid-May (full tabs are here).

All three polls show leave ahead, but all three polls were conducted online and most online polls show a close race anyway. What will be interesting is if either online polls do consistently start showing a clear lead for Leave rather than just movement around neck-and-neck, or if other telephone polls echo that ICM phone poll showing Leave ahead.


-->

Opinium have a new EU referendum poll in the Observer. The topline figures are REMAIN 43%, LEAVE 41%, Don’t know 14%… if you get the data from Opinium’s own site (the full tabs are here). If you read the reports of the poll on the Observer website however the topline figures have Leave three points ahead. What gives?

I’m not quite sure how the Observer ended up reporting the poll as it did, but the Opinium website is clear. Opinium have introduced a methodology change (incorporating some attitudinal weights) but have included what the figures would have been on their old methodology to allow people to see the change in the last fortnight. So their proper headline figures show a two point lead for Remain. However the methodology change improved Remain’s relative position by five points, so the poll actually reflects a significant move to leave since their poll a fortnight ago showing a four point lead for Remain. If the method had remained unchanged we’d be talking about a move from a four point remain lead to a three point leave lead; on top of the ICM and ORB polls last week that’s starting to look as if something may be afoot.

Looking in more detail at the methodology change, Opinium have added weights by people’s attitudes towards race and whether people identify as English, British or neither. These both correlate with how people will vote in the referendum and clearly do make a difference to the result. The difficulty comes with knowing what to weight them to – while there is reliable data from the British Electoral Study face to face poll, race in particular is an area where there is almost certain to be an interviewer effect (i.e. if there is a difference between answers in an online poll and a poll with an interviewer, you can’t be at all confident how much of the difference is sample and how much is interviewer efffect). That doesn’t mean you cannot or should not weight by it, most potential weights face obstacles of one sort or another, but it will be interesting to see how Opinium have dealt with the issue when they write more about it on Monday.

It also leaves us with an ever more varied picture in terms of polling. In the longer term this will be to the benefit of the industry – hopefully some polls of both modes will end up getting things about right, and other companies can learn from and adapt whatever works. Different companies will pioneer different innovations, the ones that fail will be abandoned and the ones that work copied. That said, in the shorter term it doesn’t really help us work out what the true picture is. That is, alas, the almost inevitable result of getting it wrong last year. The alternative (all the polls showing the same thing) would only be giving us false clarity, the picture would appear to be “clearer”… but that wouldn’t mean it wasn’t wrong.


Following the FT story about hedge funds and exit polls, it’s probably worth setting out some facts about exit polling and the referendum. I have not a clue whether the FT story is correct, but for those interested here’s what we can say about exit polls at the referendum.

There will not be an official exit poll for the referendum. At general elections the BBC, ITN and Sky normally jointly fund an exit poll. The fieldwork is normally conducted by Gfk and Ipsos MORI, and then John Curtice, Steve Fisher and the rest of their team use the data to project seat numbers. This did not happen for the Scottish referendum or the AV referendum, and it won’t be happening for the EU referendum either.

The way exit polls are done at general elections can’t be done for a referendum. I’m not privy to the BBC’s discussions, but my guess is that the reason they are not doing an exit poll is for technical reasons: the method the exit poll team use for general elections would not and cannot work for a referendum. Here’s why. At general elections the team try to revisit the same polling stations at each election (obviously some are added as electoral battlegrounds change, but the core remains the same) the projection is then done by looking at the changes in support in those polling stations since the exit poll five years before. Curtice, Fisher and colleagues will look at patterns of change (e.g, are there bigger or smaller changes in different regions, or where there are different parties in contention) and use that to project the swing across different types of seat. While the overall swing can be used to come up with national shares of the vote, that’s very much a by-product, at its heart the exit poll is all about change since the last election.

For obvious reasons this is not an option at a referendum: there was not a previous EU referendum a few years back that we can draw changes from. This means the exit poll method that has been so successful at the last three general elections cannot be used for referendums, and presumably as a result of this, the BBC, ITN and Sky have chosen not to have an exit poll at all.

Someone could still do an exit poll in theory, but who knows whether it would be accurate or not. It is possible to do exit polls in other ways. Instead of looking at swing, one could try and sample from a random selection of polling stations and work out national shares of the vote. This used to be how exit polls were done in this country before the current method was developed. Don’t fall into the trap of thinking it would necessarily be as accurate as the recent BBC exit polls though: back in 1992 the exit poll was conducted this way, and was almost as inaccurate as that year’s pre-election polls.

It is illegal to publish an exit poll until the polls have closed. Exit polls are the only type of poll that has specific laws around their publication. Under the Representation of the People Act it is illegal to publish any poll based on the responses of people who have already voted until the polls close at 10pm. There is no legal restraint on carrying them out (after all, Gfk and MORI do it at every general election), the legal bar is on publishing them while people are still voting.

Any results from much before 10pm would be of questionable use anyway. Anyone with any experience of elections knows that voting is not uniform throughout the day, there are ebbs and flows and different types of people vote at different times. Exit poll data from only the morning or only the early afternoon could be wildly misleading.

So when will we know the result? In the absence of any exit polls, we will have to wait for actual counts to take place. The Electoral Commission already has estimated result times up here. The earliest results are expected to be Sunderland, Wandsworth and Foyle, all around half twelve. Swindon, Oldham, Newcastle and Hartlepool are expected at about one. Lots of results are expected about two-ish, the bulk around three or four in the morning. Obviously how soon those results actually allow us to be confident of the overall picture depends how close it is – if all the early results show a heavy lead one way or the other we will know quite early, if they are extremely close we won’t be certain till most places have counted.

On other matters YouGov had their regular EU poll for the Times this morning. Topline figures were REMAIN 41%, LEAVE 41%, Don’t know/Won’t vote 17%… exactly the same as a week ago. There is no obvious sign of any movement towards Leave there. Full tabs are here.


There are three new EU referendum polls today, a telephone poll from ORB and parallel phone and online polls from ICM.

The ORB poll for the Telegraph has topline figures of REMAIN 51%(-4), LEAVE 46%(+4), Don’t know 3%(nc) – full tables are here. The Telegraph wrote it up as immigration producing a significant swing towards Leave. I would normally have been ready to dismiss that as reading far too much into a poll that was probably just reversion to the mean: if you look at ORB’s previous figures their poll last week may have shown a thirteen point lead… but the two before that were almost identical to today’s figures. Newspapers may want to write that up as “big swing to Remain” followed by “big swing to Leave”, but a more parsimonious interpretation is just “slightly wonky sample returns to normal”.

However, today’s other polls from ICM raise more interesting questions.

ICM once again carried out two parallel polls, one conducted online, one conducted by telephone. So far all but one of these experiments have found a big gulf between phone and online polls (typically online polls show a race that’s neck-and-neck, telephone polls show a lead averaging around 8-10 points). ICM’s online poll found the sort of close race we’ve come to expect, with topline figures of REMAIN 44%(-1), LEAVE 47%(+2), Don’t know 9%(-1). ICM’s telephone poll found wholly unexpected results of REMAIN 42%(-5), LEAVE 45%(+6), Don’t know 13%(-1). Full tables for both are here.

There is nothing unusual about how the ICM poll was carried out that might explain the unusual result, it was done the same way as their previous telephone polls that showed Remain leads of seven or eight points. This leaves us with two obvious possibilities:

Either there has been a genuine movement towards Leave in recent days and the movement in the ICM poll reflects that (suggesting also that the ORB poll is showing more than just reversion to the mean), or…

ICM just got a really wacky sample by polling over the bank holiday weekend, and future telephone polls will revert to the normal pattern of solid remain leads.

Right now we can’t really tell. I will only urge my normal caveats about not reading too much into an individual poll, especially one conducted over a bank holiday weekend (whether or not that makes an actual difference or is “pollster folklore” is unclear. It’s certainly possible to point to clear examples of weird results from polls with bank holiday fieldwork, but it’s possible to point to weird results from polls conducted at other times and bank holiday results that are perfectly normal. While I’d always try to avoid polling over Easter or Christmas, I suspect in reality the Spring bank holiday doesn’t make that much difference to fieldwork). It will be interesting to see if forthcoming polls show a wider trend towards Leave, and if forthcoming telephone polls give any further suggestion that the online-phone gap may be disappearing.