If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so


The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.

2,711 Responses to “EU referendum post-mortem”

1 2 3 4 5 55
  1. @Candy

    “Also love the comments about the loss of banker jobs – since when is the man in the street concerned with jobs at Morgan Stanley?”


    Well they may not be, but that doesn’t mean it can’t possibly be an issue. It may well be, if it’s part of a bigger exodus, and hits growth and tax revenues. And it may differentially affect SE and Tory heartlands…

  2. @ Candy & Roland Haines

    “Also love the comments about the loss of banker jobs – since when is the man in the street concerned with jobs at Morgan Stanley?”

    When the man is the street is the brickie, sparkie or flush that builds or renovates his/her home, works in the shops, bars, hairdressers, gyms or restaurants that the banker buys goods and services from. When they build the cars the bankers buy on the Jaguar Land Rover lines.

    The economy is connected. Your heart doesn’t need to bleed for a banker being relocated, but you might acknowledge that their disposable earnings go into the economy to support a lot of other people’s livelihoods.

    In my experience many people in construction and service sector are acutely aware of these connections.

  3. @ JOHN B. Hi my friend sorry there was no serious disrespect meant for Scottish people on hear I just can’t stand people calling us little Englanders all the time. I am proud to be English and British. Britain had much more influence outside of the EU than it ever did within it and I think people will realise this in the next couple of years. I know Scotland could become a member of the EU and the EURO and that is the big problem. There is a huge mismatch between economies such as Germany, France and the smaller countries that has led to so many problems. Germany uses smaller weaker countries to keep the euro weak to help with their own exports. I hope Scotland stays and realises that we can be really strong now we have been set free.

  4. Great Britain votes to leave the EU. Great Britain votes to end Great Britain. Scotland will be gone, maybe NI. And what does Khan have planned for London?

  5. Perhaps some leavers need to look how statutory instruments work in the UK. If they ever had a look in the HoC library, they would see whole loads of laws created by unelected British bureaucrats.

  6. Candy – 2.44

    The English have always been consulted about Barnet: at each election they ahve voted for pro-Barnet parties.

    Also, there is a constitutional difference between England and Scotland. In
    Engish constitutional understanding sovereignty lies with the crown in Parliament (from the 17th century onwards at least). In Scotland, on the other hand, since the Declaration of Arbroath i 1322, sovereignty lies with the people. That’s why ‘The Scots want this’ or ‘The Scots want that’ is the equivalent in England of a vote in Parliament. Chalk and cheese.

  7. The parliamentary petitions webpage is getting hammered by people trying to sign the petition calling for a second referendum.

  8. @DSL

    Sharia law?

    We aren’t out of Europe yet, in fact the first stages of the process aren’t even begun.
    In my opinion based on past EU performance they will already have a revised deal offering those things they refused Cameron – emergency brake on immigration, repatriation of foreign criminals, lower levels of benefit for migrants etc etc.
    Then there will be a second referendum, and they only need to persuade 2% of the public to change their minds and hey presto we’re back in the EU again !

  9. CANDY

    I do, however, agree entirely with people taking financial responsibility for their decisions. That is why I disagree with Thatcher’s attack on local government: instead of hobbling them, she ought to have told them to find their own money. It is also why I cannot understand the Scottish Tory position: why continue to support subsidies (i.e. support the Union)?

  10. @Assiduousity

    “The economy is connected. Your heart doesn’t need to bleed for a banker being relocated, but you might acknowledge that their disposable earnings go into the economy to support a lot of other people’s livelihoods.”


    Well apparently, once we are no longer plagued by the unelected bureaucrats none of this economics stuff about circulation of money applies any more. We just have to get over ourselves and growth happens magically…

  11. @John B

    Scotland has a deficit of £15billion – that is it spends £15bn more than it raises in tax.

    We’ve heard a lot about “the will of Scotland” with this and that power being demanded, but funnily enough Sturgeon desperately wanted the Barnett to stay.

    So people have forced the issue. They knew full well another Scottish referendum would be triggered – Blair and other warned about it, and English voters thought “what a good idea!”.

    If Sturgeon and co want to have another referendum to become independent and join the EU, they’ll get one. But that £15bn deficit won’t go away – she needs to narrow it. One way is to go the Irish route and charge people to see the GP. Another is to raise tax, another is to cut spending or a combination of all of these things.

    If Scots are serious about paying the price to join the EU nothing will stop them and they’ll willingly swallow all the austerity they need to do to achieve it.

    But what Scots can’t do is say “it’s our will to have this that and the other but it is our will to have someone else pay for it”. That’s not how it works.

    What you are seeing is a very fed-up England finally drawing the line. Scottish demands, EU demands, begging bowls here, there and everywhere. People have just finally said No. That is their will.

  12. @Alec @wolf

    During the first Indy ref the NATs were at great pains to say that independence would be about gaining control, they did not want to concede there would be any damaging changes, so they had to make a point they would keep sterling (otherwise they would have been attacked on the grounds of becoming another Greece)

    If we are going to be in Europe but not in Britain and mini Britain is not going to be in Europe then the Euro makes perfect sense – it works ok for Republic of Ireland

  13. @candy

    So you are saying that people voted leave to get rid of the Scots? Seems a bit far fetched to me.

  14. Can I claim a small prize? I reproduce below a summary I posted last week and the week before on this website (so it’s there to read).

    Pretty accurate. I said turnout 72 per cent (it turned out to be 71.8). I said Remain would poll Gibraltar 95 per cent (it turned out to be 95.9), Scotland 70 (62), NI 65 (55), Wales 50 (52.5), London 58 (60), Rest of England 45 (44.7).

    I also recommended the best punt – William Hill had Wales Leave 5/2, which you would have won.


    Remain’s problem is they do best in territories with the smallest electorates. Approximate eligible electorates: England 37.4 million (of which London 5.2 million), Wales 2.2 million, Scotland 3.9 million, NI 1.2 million, Gibraltar 23,000. Total 44.7 million.
    Scottish referendum turnout 85 per cent, GE 66 per cent. Assume 72 per cent, so England 26.9 million (of which London 3.74 million), Wales 1.58 million, Scotland 2.8 million, NI 860,000, Gibraltar 20,000 (assumes 90 per cent).
    Remain polls, say, Gibraltar 95 per cent, Scotland 70 per cent, NI 65 per cent, Wales 50 per cent (William Hill has Wales Leave on 5/2, which is about the best punt), London 58 per cent, Rest of England 45 per cent.
    So Remain/Leave = Gibraltar 19,000/1,000, Scotland 1,960,000/840,000, NI 559,000/301,000, Wales 790,000/790,000, London 2,169,200/1,570,800, Rest of England 10,422,000/12,738,000. Total 15,919,200/16,240,800. 49.5/50.5 per cent
    Of the 5.6 million ex-pats, three million can vote (15 year rule). Two years ago only 23,366 were registered, but of course many more could now do so. About a million live in the EU, so possibly 700,000 are eligible. Let’s assume 350,000 register and that 90 per cent vote to Remain. This adds 315,000/35,000. (Others outside the EU will also register, but assume they’re evenly split).
    So the new Remain/Leave total is 16,234,200/16,275,800. Leave majority 41,600.

  15. @ Hawthorn

    “The Blairites are just lashing out because they can’t recognise their own uselessness.”

    There may be some truth in this position. However, if there was ever a time to painlessly replace a leader – with so much else going on elsewhere – this is it.

    I suspect this is opportunism combined with a growing feeling that – whatever Corbyn’s other qualities – he is not a natural leader. Or may not grow into one quickly enough to keep pace with the developments that are likely to take place in the next 2-3 years.

    Of course, that’s not to say there’s anyone else out there who would be better!

  16. @DSL as you have already seen the polls are not to be relied apon by the remain camp. I wonder have they conducted a poll to see how many black and Asian racist there are in the country. Or how homothobic Muslims are? White people are continually being called racist but the truth is other ethnic groups get away with being racist and homothobic all the time. Racist is a word that is used far too easily and is disrespectful to the real sufferers of racism. Your time for winning argument ts by calling people this has come to an end as people will no longer stand for such rudeness. All you try to do is ban free speach with fear and win your point.

  17. Adam Harvey – 2.55

    The Britian to which you refer was still coming out of its three centuries of Empire and the victory (helped, to some extent, by the Russians and Americans) in the Second War. Don’t forget, 1973 was only twenty-eight years after the end of WWII. The full effects of the end of Empire were only just beginning to be felt at a cultural and financial level. Are you seriously suggesting that the UK in 2016 is anything like as powerful on the world stage as it was in, say, 1970?

  18. @thoughtful, if companies move to the continent, the EU won’t want us back. This is an opportunity for the EU as well, they will vest their own interests against us. And that’s right, they represent 500m people – minus us now. we are just 3% of the EU exports. They are 45% of ours. Who’s got the cards?

  19. No Adam, 25% of Britains don’t like or hate people of colour. You can try and say it’s rude all you like, but the survey’s prove it. You can hear it in every pub across the land. It’s a fact that Britains are intolerant and small minded. If they weren’t they wouldn’t have voted to leave.

  20. Bardin1 – “So you are saying that people voted leave to get rid of the Scots? Seems a bit far fetched to me.”

    They heard all the warnings from Blair, Cameron and others that Brexit would trigger another Scottish referendum and thought “I don’t care, I’m voting Leave anyway”.

    Attitudes to Scotland have changed over the last two years, there is anger about being short-changed, and also a feeling that Scots wouldn’t lift a finger to help the English so why should the English bother to consider them?

    So I think if Sturgeon wants to go and the Scots are prepared to pay the price for their EU dream, they’ll be waved on their way.

  21. @ Candy

    “Have you considered that the English who voted Leave .wanted to force another Scottish referendum while they were at it? ”

    I don’t totally discount the possibility that ‘getting rid of the Scots’ was on the list of priorities for voters in Hartlepool and Blaenau Gwent, Clacton and South Shields when casting their votes, but I don’t record seeing it on all those polls of ‘issues that matter to you’!

    Give it a break…

  22. Assiduousity

    I would be quite happy to see Corbyn go, because I don’t think he is up to it. The problem is that the Blairite right are even worse.

    A pragmatic soft-left leader who did not act like a robot would be quite good I think.

  23. Anyone willing to explain the currnecy market. For one Pound how many euro’s would i get today compared to yesterday? I am very confused by it all in that regard ( I am a law student so written stuff is my strong point)

  24. The parliament petition for a second referendum is getting two signatures *per second*.

    Not only that but the heat map of where “second referendum” calls are coming from must be chilling to CCHQ.


  25. Candy

    Scotland spent thirty years supplying oil revenues to help England, Thatcher in particular squandered billions. If the English now complain that the boot is now on the other foot, then so be it.

  26. The EU will….at best offer EEA, Norway style deal and the public will not be here for it at all. They shouldn’t even think of putting that question to the electorate.

  27. @Carfrew

    “Well apparently, once we are no longer plagued by the unelected bureaucrats none of this economics stuff about circulation of money applies any more. We just have to get over ourselves and growth happens magically…”

    And money grows on trees, though apparently none is any longer available for the NHS.

  28. I think the Scotland point is correct – the repetition over the years that we should bend over backwards (thismax or thatmax) to stop Scotland leaving has got a bit annoying to be honest – just split.

    They’ll be about the same size as Denmark and Denmark often tops the list of best countries so I’m sure they’ll be fine.

  29. By correct I don’t think people voted for that to happen but I think people in England are less and less inclined to do any bending.

  30. @ Hawthorn

    “A pragmatic soft-left leader who did not act like a robot would be quite good I think.”

    Looking at the likely way the conservatives are going to be positioned in terms of their leader that may well be a wise move.

    The question is – where should the leader be on Europe and immigration – I’d suggest that these two issues aren’t going away.

  31. @candy

    “They heard all the warnings from Blair, Cameron and others that Brexit would trigger another Scottish referendum and thought “I don’t care, I’m voting Leave anyway”.
    Attitudes to Scotland have changed over the last two years, there is anger about being short-changed, and also a feeling that Scots wouldn’t lift a finger to help the English so why should the English bother to consider them?”

    Any polling to support those assertions? Otherwise the Daily Mail comment boards can be found by googling them.

  32. Fortune:

    In all, the city that has long been considered the financial capital of Europe could lose as many as 40,000 workers in the wake of Brexit.

  33. Don’t suppose Denmark wanna join the UK to join their Viking mates? We could do a swap with Scotland, everybody wins…

    Would quite like Scots to stay though, not least because it’s a good place for a Spaceport…

  34. @ Jayblanc

    It is about ten a second now.

    Unless millions sign up, nothing will happen, but if millions do … ?

  35. @DSL you have just proved my point that you are one of the racist that I was talking about. You are clearly racist against white British people. You are disgusting and should be removed from this site. There is no room for hard core racists and racism in here. It works both ways and you clearly have a problem with white British people. Take a long hard look at yourself and the Hate you are spreading. You have more in common with the far right than you know.

  36. I think the worry that the Scotch will vote Yes in a referendum is premature.

    Why would they vote to leave a democratic Union that has lasted 300 years and achieved so much, only to join an undemocratic union with a basket case of a currency – which they would have to adopt to be accepted as members?

    And that is on top of there £15 billion deficit of spending vs. taxation.

  37. The EU cause is lost, as is free movement.

    What they would need to do is improve worker’s rights etc.

    The working class were just (if not more) anti-immigrant in the 1960s, but Labour were able to give them stuff that stopped that being the reason to vote.

  38. DSL

    Please no sore loser talk. How about the far left desire to destroy traditional Britain proposing childish white guilt feelings for colonialism and using mass immigration as a tool for that.
    That notion remains a bigger threat for our future than alleged racism in pubs or on the poltical right.

    And dont you think that Blacks might be racist too?Or Arabs hating black and white christian people? Or Polish or Muslim immigrants hating/dislike Jews?

    On a different topic most people I know including myself treat women who are thin and good looking with more caution and respect than less attractive or ugly ones. We are a bit sexist here that may be morally wrong but is part of human nature.

  39. @ Roger Mexico

    “The Guardian is reporting that Morgan Stanley is denying the reports. It may be a leaked contingency plan that hasn’t yet been agreed on (or maybe won’t be). Still they seem better prepared than the UK government was.”

    BBC now clarifying that Morgan Stanley ‘haven’t started the process yet’ but will not wait for Article 50 to start making these strategic decisions.

    Notice the The Graun also had a report that JP Morgan, were doing something similar.

    I suppose just something to keep a watching eye on – often finance jobs get shifted and shunted about without any of the attention that accompanies those in manufacturing.

  40. Jamie, you’re right and I’d be relieved if they did offer us Norway mk.2 and something worse than that. And can we refuse? Not really because they’ve got the better bargaining position; we won’t be bargaining with an equal, this will very much be flyweight takes on heavyweight. And if we do get a Norway mk2 we still have movement of EU nationals into this country; nothing changes,except we’ve lost influence. But we’ll have to pay to have access to that mkt like Norway does. So overall, we lose, lose and lose. And all because of our knuckle-dragging brethren. DC making a smart move for his legacy, he ain’t pushing the article 50 button. It will be on bojo’s head!

  41. @Assiduousity

    “And money grows on trees, though apparently none is any longer available for the NHS”


    Ah well, that’s just Farage doom-mongerimg. The emergency budget must be doom mongering too… Anything bad that actually happens as a result of Brexit is mere doom mongering.

    Criticisms of immigration weren’t doom mongering, obviously…

  42. Adam

    Why the anger when you have won?


  43. Carfrew

    Sure, I’ve always been a unionist but i think the demos was broken in the 1980s and left unfixed too long so now it’ll be endless arguments.

  44. @Mcclane, forget the poor hard-up white guy act. I said 25% of Britains are racist, that includes all colours. These are facts but they’re mostly white surprisingly . Pretty deplorable. To me, religious extremists, racists, bigots and knuckle-draggers all fall into the same fruit basket.

  45. @ Hawthorn

    I does seem as though members from certain wings of the Labour Party are queuing up to give interviews to the BBC, the content being highly critical of Corbyn.

    I’m not sure if it amounts to a hill of beans, someone more familiar with Labour party procedure would have to explain how they could mount an effective coup within the current rules.

  46. @hawthorn, because we know they have an even worse agenda. armbands and all.

  47. John B

    What Candy has expressed rather eloquently in my opinion is that many English people, myself included are bored to tears with the Scots demanding this and that and the other. Your are not more important that the collective view of the UK as expressed yesterday.

    Back to the cricket, more interesting.

  48. For Adam who doesn’t like reality:

    Here is the link to the British Social Attitudes survey in which 30% admit prejudice:


  49. Morgan Stanley and JPM are by no means the only banks to have plans in place to move highly paid (and therefore highly taxed) finance jobs out of the City. The £10 billion EU contribution seems fairly insignificant when we have to find a way to replace the £65bn in tax receipts that the City provided last year (11% of HMRC’s total tax receipts, from just 3% of the total UK workforce).

  50. @Mr Jones

    (Assuming you’re on about the UK Union) Sure, what happened in the 80s economically, and then add in stuff like piloting the poll tax up in Scotland is understandably going to cause issues.

    That said, the SNP did vote to reject old Labour’s approach of trying to preserve industry, and ushered in the Thatcherite approach instead, and in more recent times they’ve had devolution and Barnett, which are of some significance surely.

    Being as they voted to stay in the Union. I suppose the question is, to what extent are Scots looking at things as they really are and deciding they want no more of it, and how much is it affected by koolaid. One can ask the same question of the EU ref.

    Since there has clearly been some imbibing of koolaid, e.g. Oil prices.

1 2 3 4 5 55