What sort of Brexit?

I’ll be taking a break from the blog over the next week while I have a summer rest (I may pop in if something interesting happens, but I’m going to try not to), but before I go a quick pointer to something I wrote over on the YouGov website on what the public think about Brexit.

The type of Brexit the public want is a tricky subject to poll. It will obviously be one of the dominant issues in British politics over the next few years, yet we also know so little of it. We don’t yet know with any confidence what the government’s aims or negotiating position will be, nor what other European countries will be willing to offer (or what they will want in return). Public opinion will be one of the limitations upon the government’s negotiations so it’s certainly important, but it’s hard to measure it at this stage when people have so little information about what’s on offer.

We tried to explore the issue in two ways. The first was to ask whether people thought various things would be acceptable trade-offs in exchange for continued British free trade with the EU. That suggests that the public would accept having to follow some EU trade rules, could be persuaded on immigration (33% think freedom of movement is desirable anyway, 19% a price worth paying, 33% a deal-breaker), but would object to Britain making a financial contribution to the EU (41% think it would be fine or a price worth paying, 44% think it would be a deal-breaker).

However, taking things individually risks being a little misleading. When it comes to it a deal will be a package of measures and will be judged as a whole. On that basis, I think the questions that present people with various scenarios and ask them to judge them as a whole are more enlightening.

By 44% to 32% people thought it would be bad for Britain if we simply left and had no trade deal with the remainder of the EU. A Norway-type deal, with Britain joining EFTA and maintaining free trade with EU in exchange for free-movement, a financial contribution and following trade rules is seen a little less negatively (35% good, 38% bad)… but perhaps more importantly, by 42% to 32% people would see it as not seen as honouring the result of the referendum. Finally, we asked about a Canada-type deal, where there is no freedom of movement or financial contribution, but only a limited free trade deal that excludes services. That was seen as both honouring the result of the referendum, and as positive for Britain.

Of course negotiations haven’t yet started and the actual deals that end upon on the table may very well differ from these examples. I suspect views are not very deeply held yet, and people may very well change their minds when deals start to take shape and politicians and the media start to debate them. The public’s starting point, however, seems to be that a limited trade deal is both the best solution and a solution that respects the referendum result. We shall see how that changes once the negotiations actually begin.

The full tabs are on the website here.


If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so

finalpolls

The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.


-->

It’s the eve of the referendum, so we have a flurry of late polls. Later on this evening we will have figures from ComRes and YouGov (Ipsos MORI’s final poll is normally in the Standard, so will probably be out tomorrow morning), already we have final figures from TNS and Opinium.

Opinium have topline figures of REMAIN 44%, LEAVE 45%, Undecided 9%. Leave are ahead by the tiniest of margins, but clearly the two sides are within the margin of error of each other. Full tabs are here.

TNS‘s final referendum poll also has Leave ahead, this time by two points. Topline figures here are REMAIN 41%, LEAVE 43%, Undecided or won’t vote 16%. Note that unlike TNS’s last few polls their headline figures here are NOT weighted for turnout – with their turnout model they would have been Remain 42%, Leave 49%. Full tabs are here.

I will update later once ComRes and YouGov publish. In the meantime both of the non-British Polling Council companies who produced more unorthodox polls last week have produced updated figures – SurveyMonkey have final figures of REMAIN 50%, LEAVE 47%; Qriously (the company sampling via smartphone ads) has final figures of REMAIN 37%, LEAVE 51, Don’t know 12%. Again, make of that what you will.

UPDATE: The ComRes and YouGov eve-of-referendum polls are now also out. Whereas TNS and Opinium both had Leave leads, ComRes and YouGov both show Remain ahead (albeit, by different margins):

ComRes for the Daily Mail have topline figures of REMAIN 54%, LEAVE 46%, a widening of the Remain lead after their last poll showing Remain and Leave within a point of each other. ComRes have reallocated don’t knows based on respondents’ views of the impact of Brexit on the economy, which looks like it boosted Remain by a point or so. Full tabs are here.

YouGov for the Times have topline figures of REMAIN 51%, LEAVE 49% – so considerably closer. The YouGov poll now includes a turnout weight (though it made no difference at all to the topline) and a squeeze question, which also bumped Remain up by a point. Full tables are here. On YouGov’s website they’ve also updated the multilevel regression and post-stratification (MRP) model of referendum voting using all their data, which they first posted earlier in the week, that is now also pointing towards a small lead for Remain.

Note that all four of the polls here include Northern Ireland. Most general election polls don’t, and so polls during the EU campaign have varied on whether they do or do not include NI – all these four do.

UPDATE2: Two more polls published on the day itself. Note that these polls were conducted before polls opened, they are only published today. It’s illegal to publish polls conducted on the day until polls close, but perfectly fine to publish polls conducted before polls opened.

Ipsos MORI‘s final poll has topline figures of REMAIN 52%, LEAVE 48%, putting Remain back ahead after a leave lead in MORI’s penultimate poll. MORI have slightly changed their turnout filter for their final poll, basing it on how likely people say they are to vote and how important they say the result is to them. Full tabs are here.

Finally, and a little surprisingly, Populus have produced a final call poll. Populus’s Andrew Cooper has been working with the StrongerIn campaign so the company haven’t been putting out regular polls during the campaign, but they have produced final topline figures of REMAIN 55%, LEAVE 45%. Unexpectedly given the topline results the poll was conducted online (completely messing up that “phone & YouGov saying in, other online saying out” pattern). Populus haven’t released tables yet, so I’ve no details of the weightings or adjustments used.


Tuesday polls

Two more polls have been released during today, both showing the race essentially neck and neck.

Survation have released their final EU telephone poll for IG Group (not sure if that’s their final poll for the referendum itself, or just the final one for IG). Topline figures with changes from their weekend poll are are REMAIN 45%(nc), LEAVE 44%(+2), Undecided 11%(-2). Full tabs are here.

Surveymonkey also released new online figures this morning (for those unfamiliar with Surveymonkey as pollsters, I wrote about them here). Their topline figures in the new poll, conducted Friday-Monday are REMAIN 48%, LEAVE 49%. Changes are from their poll last week.

I don’t think any polls are due in tomorrow morning’s papers, most of the remaining final calls will presumably be showing up tomorrow afternoon or evening.

Finally a note about the ORB poll this morning. As regular readers will know, ORB figures have been a little confusing over the campaign – they have published two sets of figures, one for those 10/10 certain to vote, one for all voters. ORB have regarded the latter as their main figure, but the Telegraph have focused on the former. For their final call though ORB have been much clearer and put up an explanation on their site, with final projections of REMAIN 54%, LEAVE 46% – based on those certain to vote, and an assumption that the remaining don’t knows will split 3 to 1 in favour of Remain.


There are three polls in tomorrow morning’s papers – ORB in the telephone, YouGov in the Times and a NatCen poll in the Financial Times.

YouGov for the Times has topline figures of REMAIN 42%(-2), LEAVE 44%(+1), Don’t know or Won’t vote 13%, conducted between Friday and Sunday. While Leave nudge ahead of Remain again, YouGov continue to show an extremely close race (and it confirms the narrowing of the race from the seven point Leave lead they had a week ago).

ORB’s poll is reported in the Telegraph as showing Remain “surging back into the lead” with figures of Remain 53%(+5), Leave 46%(-3). These figures are based on only those certain to vote however, and ORB have previously suggested that they regard their figures for all voters as their primary measure. On those figures the movement is in the other direction – REMAIN 49%(nc), LEAVE 47%(+3).

Thirdly there is a NatCen poll. Full details of the NatCen poll were embargoed until midnight, but Reuters have the topline figures here. Headline voting intention is REMAIN 53%, LEAVE 47% – but it’s important to note that the fieldwork is very old, conducted between May 16th and June 12th, with two thirds of the fieldwork done before May 26th.

This means the NatCen poll is of limited use in measuring current support, but is an interesting methodological experiment. The poll was conducted online by recontacting people who took the randomly sampled British Social Attitudes Survey, making it effectively a small randomly recruited online panel (people who couldn’t be contacted online were interviewed by phone instead, taking several weeks over the fieldwork to maximise response rate). Random recruitment of online panels is often suggested as a potential way forward for polling, though it’s not necessarily a panacea (in the States Pew already have a randomly recruited online panel called the American Trends Panel, but when they benchmarked it on how representative it was compared to commercial online panels recruited from volunteers and it ended up mid table).

Looking back at other polling at about the time the NatCen poll was conducted, online polls were showing an average Remain lead of about two points, telephone polls were showing an average lead of about twelve points, so the six point Remain lead is somewhere inbetween the two.

The Natcen fieldwork took place between the significant shift towards Leave we saw at the start of June, and obviously before the possible movement back towards Remain in recent days. In the Reuters article NatCen are quoted as saying that responses moved towards Leave over the fieldwork period, though it’s not possible to tell if that was changing opinions or harder to reach people being more Leave. Slightly counter-intuitively it also says that people who answered the survey online were more Remain than people who answered by phone – though that could easily be because people who couldn’t take the survey online were older or poorer.