If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so


The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.

2,711 Responses to “EU referendum post-mortem”

1 2 3 4 55
  1. Don’t know how I got a K in nuclear! K next to L I suppose

  2. Carfrew

    Yes off to watch now. Last match was great, like the referendum I never thought they would do it.

    For me more great Rugby tomorrow. England v Australia (can we make it a whitewash?) and England Under 20 v Ireland Under 20 World Cup final.

  3. Alec – “Instant market judgements are poor, so we’ll have to wait and see, but I suspect the headline falls have as much to do with markets misreading the polls and artificially rising in the last ten days of the campaign as with the perceived impacts of the actual results.”

    Basically the markets surged when poor Jo Cox died, a distasteful and wrongheaded move if ever there was. And now it’s come to bit the backsides of market players who cheered so much. Hope their losses sober them up.

  4. Anthony, we need some early polls on Sco ref. In the nightmare scenario, Westminster negotiates with the EU and Scotland simultaneously, both a trade/people movement relationship and a carve-up of the UK at the same time. Saving 10bn from Brussels won’t mean much when England and Wales lose 30bn of oil tax.

  5. Roger Mexico

    Youre right here but still a bit odd if the final poll you released after 10 pm shows another point swing in favour of Remain.

    ComRes, ORB and Populus produced another 12 to 14 percentage points miss and also Yougov has been off by three or four points in both ways repeating its errors of the last general election overestimating young turnout in particular and underestimated the shy Leaver effect.

  6. Do English Premier League need to get work permits for all those Euro players they want to sign? What about the youth players?

    Gonna be interesting.

  7. @Laszlo: It would be the same money really, except it doesn’t have to go to Brussels first, but I guess Wales will have to compete for the subsidies more directly with other UK claimants. OOTH the Welsh Barnett settlement is seen in Wales as unfair compared to Scotland’s.

  8. There seems to be an instant buyers remorse effect occurring, with the press taking comments from many Leave voters saying they regret their vote, because they thought it was a safe protest vote that wouldn’t result in actually leaving.

    Meanwhile I’m going to get my papers in order, so my dual Irish Citizenship will be properly recognised.

  9. I have no idea at all why Scotland should prefer to throw in its lot with New Little England and its Empire nostalgists when it could choose instead to snuggle up to the likes of Sweden, Ireland, France, Germany etc.

    The referendum has been won by what Professor Michael Dougan called last week “dishonesty on an industrial scale”, and now reality is going to bite. Already Farage has been on TV saying that that £350 million isn’t going to go to the NHS after all – surprise, surprise…

    To quote an internet meme, “project fear is dead – now it’s called ‘the news'”.

  10. @Jayblanc:

    I’ve similarly heard reports of people that changed their mind at the last second and voted remain but are happy it’s a leave vote.

    We need polls, not anecdotes.

  11. Devastated by the result – I can not believe we have voted also to break up the United Kingdom – Corbyn should be hung out to dry for the awful Labour performance although as a Tory I would prefer he stayed.

    I look at the Brexit leadership and cringe at the thought of Cash,Redwood,Mogg,Jenkins and Duncan Smith taking the party I have supported all my life to the right which is somewhere I can not go – I feel disenfranchised at the moment – hopefully I will see the light!!!!

    Will not be back for a while but I have enjoyed all your comments.

  12. @AC – “I watched part of Sturgeon’s speech on BBC24 and she did not mince her words and looked and sounded very angry at Scotland being dragged out of the EU and you have to acknowledge the remain vote in Scotland was overwhelming.”

    Yes, agree entirely, especially the later bit about federalism.

    I said previously many times that a coherent UK constitutional settlement was desperately needed after the indyref. My referred solution was to have some mechanism for major issues (I think I quoted declaring war) had to have the approval of all four parliaments. I would have put major constitutional change (like leaving the EU) into the same bracket.

    Most countries have some requirement for a super majority or a confirming vote etc when it comes to major changes, and given the UK make up we should have had something in place.

  13. Morgan Stanley looks to move 2,000 London staff

    Posted at

    BBC business reporter Joe Lynam reports…

    Sources within Morgan Stanley say it has already begun the process of moving about 2,000 of its London-based investment banking staff to Dublin or Frankfurt. And it has a taskforce in place.

    The jobs which would be moved from the UK would be in euro clearing but also other investment banking functions and senior management.

    The American investment bank needs to avail of the passporting system which allows banks to offer financial services in all countries in the EU without having to establish a permanent base in that member state.

    The president of Morgan Stanley, Colm Kelleher, told Bloomberg two days ago that Brexit would be “the most consequential thing that we’ve ever seen since the war”.

  14. @Bobinnorfolk – “….Corbyn should be hung out to dry for the awful Labour performance although as a Tory I would prefer he stayed.”

    Would it not be more appropriate to call for Boris to be thrown out? He’s the one that did it.

  15. @ Dave

    Yes, I know. But there will be quite a competition for any money left in the kitty.

  16. These are jobs that once moved, will not come back. London is losing it’s place as a centre for European market trades, and even a reversal of the referendum won’t bring those jobs back because we’ll be seen as too big a risk. Even with the tobin taxes on the EU mainland, the UK is no longer the right place for an international bank trading in Euros to be.

  17. @Jay, Frankfurt and Paris will benefit. France and Germany won’t tolerate a special financial trading relationship with a country outside the EU.

  18. Morgan Stanley announce they are to move 2,000 staff from London to Dublin and Frankfurt. Process will start in days.

    We might have discussions about re-balancing the economy, the social value of investment bankers etc. But these are (very) well paid jobs departing so quickly, with all the lost consequential spending power.

    Hopefully this is one bank taking a position; however, the BBC report placed this in a context of contingency planning for Brexit across the sector. We wait to see if others follow.

  19. I do find it odd that Jeremy is being blamed for Labour’s “poor performance”.

    The part of Labour’s vote that voted Leave, in my view, was based on a deep and long term disenchantment with the establishment, probably with good cause.

    No amount of words speeches could undue that. I suspect the bigger issue is that the overwhelming support from the Labour hierarchy for Remain was out of step with a fair chunk of it’s voters.

  20. This is the problem, the mkts and industry will react before the Cons have a chance to even elect a new leader. The politicians can muck around for 2yrs, Nissan will have a new production line in the EU running by then. These companies are big and have their own planners/economists. They won’t delay, not when profits depend on it.

  21. @ AD/AS why would the EU want a little Scotland that has very little to offer them join the EU. It would be just another country they would have to bail out in a couple of years like Greece.

    On the other hand what you will now see is the Germans wanting to strike a trade deal with the all powerful English.

  22. Morgan Stanley makes a knee-jerk reaction and decides to just pay less tax to the UK.

    Not that they paid much at all.

  23. Thanks AW for the analysis – I agree that given the over all context this was far from a disaster for the pollsters, who also accurately identified the demographic splits between Remain and Leave.

    What is mind blowing is the wishful thinking we are still getting from the Leavers. NS pretty much explicitly stated that this would trigger a second ref, and if I feel alienated from the rest of England atm I am sure that is currently ten times greater in Scotland. I don’t think questions of the economy or currency will matter one iota now and NS knows this. What happens over NI is now anybody’s guess.

    As far as the economy goes it will take more than a ‘we are English we will be ok’ attitude. There was a reason why those maligned experts advised against this.

    I have know idea which party will come out of the with improved VI in the short – but I think we are going to see a fundamental lurch to the right over the next couple of years.

  24. After the shedding of industry in the early eighties, the deregulation in finance allowed us to snaffle a load of banking etc. to compensate, albeit more in the SE.

    More recently, economic policy being what it was, immigration rather helped to compensate for lack of growth.

    If these two factors are now to be history, be interesting to see what Boris and Gove et al can come up with instead…

  25. Labour leaders are egalitarians, they have never been connected with half of their voters who are knuckle dragging racists. Polls have proved that 25% of Britain’s are racist and they’re mostly in classes C D E; Labours natural territory.

  26. When do we get this emergency budget then? Quite soon presumably, what with it being an emergency an’ all…

  27. Morgan Stanley; their employees pay a lot of 40% tax.

  28. For Christ sake save us the forecasts of doom because we will no longer be governed by unelected bureaucrats from Europe. The people have spoken. What they have said is they don’t want Labour
    (last year) and they don’t want Europe last night.
    Get over it some of you. And if you have a republic of Ireland passport go and live there.

  29. DSL – “Saving 10bn from Brussels won’t mean much when England and Wales lose 30bn of oil tax.”

    Now you’re inventing numbers! The oil tax revenue came to a paltry £130 million in 2015:


    Scotland is being subsidised by England. You could argue that England in voting for Brexit knowing it would trigger another Scottish referendum, was trying to shuck off the costs of Scotland at the same time as the costs of the EU.

    Also love the comments about the loss of banker jobs – since when is the man in the street concerned with jobs at Morgan Stanley? They’re more concerned about jobs at Ford leaving the UK because the EU gave Ford a subsidy to open up in Turkey as part of the migrant deal.

    If you were campaigning saying “Vote remain to save Morgan Stanley jobs”, then that right there is where you went wrong!

  30. Osborne quit yet?

  31. One interesting snippet from Ashcroft:

    Seven voters in ten expected a victory for remain, including a majority (54%) of those who voted to leave. Leave voters who voted UKIP at the 2015 election were the only group who (by just 52% to 48%) expected a leave victory.

    So for an awful lot of Leave voters, they may have felt safe in voting that way, ‘knowing’ that they could express their anger about whatever and yet still there would not be any consequences. As I said before, be careful of what you wish for.

    Meanwhile here’s a Mirror article from a month ago:


    where Nigel Farage “speaks to the Mirror’s Associate Editor Kevin Maguire and warns that a ’52-48 result would be unfinished business'”

  32. CANDY.
    Right, I for one, am all broken up about Morgan Stanley jobs.

  33. haha, see how you feel about a recession this year. It will affect all of us. Companies and jobs will leave; fact. They aren’t going to hang around to see what happens in the next two years. Sturgeon ain’t waiting, business certainly won’t. Business is about survival and prospering. Sitting around doesn’t achieve that. House prices will fall, sterling and the markets will fall. Were 9/10 economists right or UKIP right. I’d say the former based on what’s already happened on day 1.

  34. Roger, are you saying a lot of leavers were just blustering (their hearts), hoping that remain won (their heads)?

  35. The Guardian is reporting that Morgan Stanley is denying the reports. It may be a leaked contingency plan that hasn’t yet been agreed on (or maybe won’t be). Still they seem better prepared than the UK government was.

  36. Adam Harvey

    “Why would the EU want Scotland?”
    Or, for that matter, Finland, or Portugal or Lithuania, or ……

    One of the greatest problems of this whole debate is the inability of the Brexiters to see beyond France and Germany. Sure, they are major players; but Scotland fits easily into the category of smaller nations which make up the majority of participants (at least at the level of nations/states) within the EU.

    Scots are used to riding two horses: to be a small nation within a wider context (e.g. within the British Empire), and enjoying the benefits of both, is how we have lived for centuries. Given the choice between being part of the EU and being part of the much smaller UK many of us see no contest – but some opt for the UK and others for the EU.
    Personally I see no universally acceptable option available.

  37. One of the largest internet marketing companies in the world is moving their 9,000 international employees from the UK to Lyon and Munich – the staff was told this morning. They are all taxpayers.

  38. @DSL it is people like you that call people racists at the drop of a hat to try and win an argument that have actually helped the leave campaign. People are fed up all the same verbal diarrhoea and being spoken down to. So thanks for all your help mate in the referendum.

  39. Laslo, whose that bud?

  40. @RedRich

    Have you considered that the English who voted Leave .wanted to force another Scottish referendum while they were at it? After all nobody consulted them on the Barnett formula or on the other goodies offered to Scotland after their referendum.

    The whole thing was couched in terms of “the people of Scotland want this and that but want to keep all the subsidies”, and the people who had to do the giving were not consulted at all.

    It was all couched in the “will of the people of Scotland” but the will of the people of England not to be taken advantage was not considered.

    So English voters have simply forced the issue. Sturgeon will go to the Scots and say “Joining the EU is so important, you need to make the following financial sacrifices” – and their “will” to join the EU will be paid for by their own money – which is as it should be.

  41. @Adam Harvey “what you will now see is the Germans wanting to strike a trade deal with the all powerful English.”

    @Red Rich “What is mind blowing is the wishful thinking we are still getting from the Leavers”

    Well, precisely.

  42. @Roland Haines

    Get over it some of you. And if you have a republic of Ireland passport go and live there

    Given that my wife has already suffered from racial abuse in the past 24 hours by a man wearing a British bulldog badge, we are seriously considering that actually.

    If the boot was on the other foot we would already be talking about another ref. given the closeness of the vote. Whatever you may think there are lot of very disillusioned people in this country today and that’s not going to disappear over night.

  43. I’m not immune to downturns in the economy but I am unlikely to be left in the hard cart whatever happens. The people who will suffer when the city jobs disappear are not the rich but the poor. The rich will move with the jobs, the poor will stay and get poorer. It was always thus.

    On the other hand, Ireland looks like a buy. Perhaps a couple of well appointed buy to let flats in Dublin, before the market goes ballistic.

  44. @Adam, not all leavers were racist, but all racists did vote leave. 1 in 4 of Britains are racist and did vote leave. Facts from polling orgs. Which means half the Leave vote are racists.

  45. The Blairites are just lashing out because they can’t recognise their own uselessness.

  46. @Roland

    Lol, it’s ok for you guys to talk of doom due to “unelected bureaucrats of Europe” etc., but peeps aren’t allowed to consider other aspects.

    You can imagine a Brexit utopia if you like, There’s some doom both ways, frankly…

  47. Roland Haines

    We have never been governed by unelected bureaucrats in Europe. There has never been any rule or regulation from the EU which was not approved by the elected Commissioners (elected by the sovereign governments) or by the Council of Ministers (elected by the national parliaments) or by the EU Parliament (elected by us). You might as well say that we are governed by unelected writers of election manifestos.


    Scotland is not subsidised by England outside the City of London. If anything, Scotland subsidises those areas of England which votes Leave.

    I couldn’t care less about Morgan Stanley; but I see no sign that the Leave people have any ‘pro-manufacturing-business’ policy which will help bring jobs to those areas devasted in the 80s and left to rot by Blair and Brown’s love in with the City.

  48. @Roland Haines

    Yesterday when it looked like Remain was going to win, this site was full of Leavers banging on about how the poll had been rigged.

    Today, when Leave has won, all the talk of rigging has magically stopped. Instead we have Leavers banging on about how Remain voters should quit the country.

    It’s the paranoia and viciousness of so much of Leave that has poisoned the country.

  49. @ DSL

    I can’t disclose it, but it is owned by a company from another continent and it is not NA.

    I wonder how many companies had this preparation and if Khan’s message reassuring the EU citizens is related to it (this company has outlets in many places in England (and one in Scotland)).

1 2 3 4 55