If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so

finalpolls

The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.


It’s the eve of the referendum, so we have a flurry of late polls. Later on this evening we will have figures from ComRes and YouGov (Ipsos MORI’s final poll is normally in the Standard, so will probably be out tomorrow morning), already we have final figures from TNS and Opinium.

Opinium have topline figures of REMAIN 44%, LEAVE 45%, Undecided 9%. Leave are ahead by the tiniest of margins, but clearly the two sides are within the margin of error of each other. Full tabs are here.

TNS‘s final referendum poll also has Leave ahead, this time by two points. Topline figures here are REMAIN 41%, LEAVE 43%, Undecided or won’t vote 16%. Note that unlike TNS’s last few polls their headline figures here are NOT weighted for turnout – with their turnout model they would have been Remain 42%, Leave 49%. Full tabs are here.

I will update later once ComRes and YouGov publish. In the meantime both of the non-British Polling Council companies who produced more unorthodox polls last week have produced updated figures – SurveyMonkey have final figures of REMAIN 50%, LEAVE 47%; Qriously (the company sampling via smartphone ads) has final figures of REMAIN 37%, LEAVE 51, Don’t know 12%. Again, make of that what you will.

UPDATE: The ComRes and YouGov eve-of-referendum polls are now also out. Whereas TNS and Opinium both had Leave leads, ComRes and YouGov both show Remain ahead (albeit, by different margins):

ComRes for the Daily Mail have topline figures of REMAIN 54%, LEAVE 46%, a widening of the Remain lead after their last poll showing Remain and Leave within a point of each other. ComRes have reallocated don’t knows based on respondents’ views of the impact of Brexit on the economy, which looks like it boosted Remain by a point or so. Full tabs are here.

YouGov for the Times have topline figures of REMAIN 51%, LEAVE 49% – so considerably closer. The YouGov poll now includes a turnout weight (though it made no difference at all to the topline) and a squeeze question, which also bumped Remain up by a point. Full tables are here. On YouGov’s website they’ve also updated the multilevel regression and post-stratification (MRP) model of referendum voting using all their data, which they first posted earlier in the week, that is now also pointing towards a small lead for Remain.

Note that all four of the polls here include Northern Ireland. Most general election polls don’t, and so polls during the EU campaign have varied on whether they do or do not include NI – all these four do.

UPDATE2: Two more polls published on the day itself. Note that these polls were conducted before polls opened, they are only published today. It’s illegal to publish polls conducted on the day until polls close, but perfectly fine to publish polls conducted before polls opened.

Ipsos MORI‘s final poll has topline figures of REMAIN 52%, LEAVE 48%, putting Remain back ahead after a leave lead in MORI’s penultimate poll. MORI have slightly changed their turnout filter for their final poll, basing it on how likely people say they are to vote and how important they say the result is to them. Full tabs are here.

Finally, and a little surprisingly, Populus have produced a final call poll. Populus’s Andrew Cooper has been working with the StrongerIn campaign so the company haven’t been putting out regular polls during the campaign, but they have produced final topline figures of REMAIN 55%, LEAVE 45%. Unexpectedly given the topline results the poll was conducted online (completely messing up that “phone & YouGov saying in, other online saying out” pattern). Populus haven’t released tables yet, so I’ve no details of the weightings or adjustments used.


-->

Tuesday polls

Two more polls have been released during today, both showing the race essentially neck and neck.

Survation have released their final EU telephone poll for IG Group (not sure if that’s their final poll for the referendum itself, or just the final one for IG). Topline figures with changes from their weekend poll are are REMAIN 45%(nc), LEAVE 44%(+2), Undecided 11%(-2). Full tabs are here.

Surveymonkey also released new online figures this morning (for those unfamiliar with Surveymonkey as pollsters, I wrote about them here). Their topline figures in the new poll, conducted Friday-Monday are REMAIN 48%, LEAVE 49%. Changes are from their poll last week.

I don’t think any polls are due in tomorrow morning’s papers, most of the remaining final calls will presumably be showing up tomorrow afternoon or evening.

Finally a note about the ORB poll this morning. As regular readers will know, ORB figures have been a little confusing over the campaign – they have published two sets of figures, one for those 10/10 certain to vote, one for all voters. ORB have regarded the latter as their main figure, but the Telegraph have focused on the former. For their final call though ORB have been much clearer and put up an explanation on their site, with final projections of REMAIN 54%, LEAVE 46% – based on those certain to vote, and an assumption that the remaining don’t knows will split 3 to 1 in favour of Remain.


There are three polls in tomorrow morning’s papers – ORB in the telephone, YouGov in the Times and a NatCen poll in the Financial Times.

YouGov for the Times has topline figures of REMAIN 42%(-2), LEAVE 44%(+1), Don’t know or Won’t vote 13%, conducted between Friday and Sunday. While Leave nudge ahead of Remain again, YouGov continue to show an extremely close race (and it confirms the narrowing of the race from the seven point Leave lead they had a week ago).

ORB’s poll is reported in the Telegraph as showing Remain “surging back into the lead” with figures of Remain 53%(+5), Leave 46%(-3). These figures are based on only those certain to vote however, and ORB have previously suggested that they regard their figures for all voters as their primary measure. On those figures the movement is in the other direction – REMAIN 49%(nc), LEAVE 47%(+3).

Thirdly there is a NatCen poll. Full details of the NatCen poll were embargoed until midnight, but Reuters have the topline figures here. Headline voting intention is REMAIN 53%, LEAVE 47% – but it’s important to note that the fieldwork is very old, conducted between May 16th and June 12th, with two thirds of the fieldwork done before May 26th.

This means the NatCen poll is of limited use in measuring current support, but is an interesting methodological experiment. The poll was conducted online by recontacting people who took the randomly sampled British Social Attitudes Survey, making it effectively a small randomly recruited online panel (people who couldn’t be contacted online were interviewed by phone instead, taking several weeks over the fieldwork to maximise response rate). Random recruitment of online panels is often suggested as a potential way forward for polling, though it’s not necessarily a panacea (in the States Pew already have a randomly recruited online panel called the American Trends Panel, but when they benchmarked it on how representative it was compared to commercial online panels recruited from volunteers and it ended up mid table).

Looking back at other polling at about the time the NatCen poll was conducted, online polls were showing an average Remain lead of about two points, telephone polls were showing an average lead of about twelve points, so the six point Remain lead is somewhere inbetween the two.

The Natcen fieldwork took place between the significant shift towards Leave we saw at the start of June, and obviously before the possible movement back towards Remain in recent days. In the Reuters article NatCen are quoted as saying that responses moved towards Leave over the fieldwork period, though it’s not possible to tell if that was changing opinions or harder to reach people being more Leave. Slightly counter-intuitively it also says that people who answered the survey online were more Remain than people who answered by phone – though that could easily be because people who couldn’t take the survey online were older or poorer.


Like the Survation poll in the Mail on Sunday, YouGov in the Sunday Times show Remain ahead again following the pro-Leave polls a week ago. Topline figures are REMAIN 44%, LEAVE 43%, with fieldwork conducted on Thursday and Friday (full tabs are here). Almost inevitably people are going to look at these polls and assume that the murder of Jo Cox on Thursday has caused the move back towards remain.

My own view is that Jo Cox’s death probably isn’t the cause of the reverse. YouGov also conducted a poll on Wednesday-Thursday for ITV, and that already showed Leave’s lead falling (and indeed, a third of the fieldwork for this poll was conducted before Jo Cox’s death was announced). Looking at the rest of the questions, there is also a marked shift in people’s views on how they think leaving the EU would impact their finances – 33% of people now think that they would be worse off outside the EU, compared to 23% a fortnight ago.

The historic trend in referendums is for people to move towards the status quo. In Scotland a couple of years ago a couple of polls a fortnight out were neck-and-neck, but moved back to a clear NO lead by the final polls (and there was a further swing on the day itself). In the EU referendum polls have consistently shown that people think leave is the riskier choice and that people think it will damage the economy. While it was never inevitable, this has always suggested that late movement towards Remain was quite likely. If people are increasingly worried about Brexit’s impact on their own personal finances, then even more so.

Of course, we will never know for sure. The reality is that we can see changes in headline voting intention in polls, but we can never be certain what causes them: all we can do is look at what events happened at the same time and at what changes there have been in other questions in the poll that might have driven a shift. What we do know is that, whatever the reason, we’ve got four new polls tonight – some before Jo Cox’s death, some after – with three of them showing a shift back towards Remain.