If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so


The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.

2,711 Responses to “EU referendum post-mortem”

1 50 51 52 53 54 55
  1. One should always have a specious reason for supporting one team in a contest in which one is not involved (the normal condition for a Scotland fan).

    Tonight, I will be supporting the existing EEA team against the one which will find itself there very soon! :-)

  2. @ToH

    Indeed, but he won’t go on forever, gonna have to start planning for the future without him. (Once we’ve settled the burning question of who succeeds Cameron and Corbyn of course…)

  3. “who would vote red as my grandad used to say, even if they ‘pinned a red rosette on a donkey’.”


    Has anyone actually tried that? It’s about time we knew the truth of the matter, one way or the other. Right after we’re clear precisely when Osborne should announce his support for,someone…

  4. @Alec

    Hmm, I demur slightly. From what I’ve seen the e-mail stuff, whilst not brilliant, is being hideously overplayed (seems more of just poor planning resulting from wings of party not talking to one another) and smacks of more than a whiff of Johnson and Benn trying to deflect from their own p*ss-poor showings (I didn’t even know that Benn was joint manager of the Labour In campaign until today).

    Allied to this, nobody has yet coherently explained how the alternate strategy of ‘full-throated support for the EU, coupled with promises on free movement everyone and their dog knows we couldn’t possibly keep’ would have fared any better.

    As an unrelated aside, cocking up a coup (where there is literally only one thing you have to do) is not a great advert for political effectiveness.

  5. @carefrew,

    Lol, Could do. I know I was slightly rude on Liverpool, but they are just so tribal and represent the perfect take for granted city labour love. I Knew Liverpool would be remain, as leave was too associated with the Tories, no other reason at all.

  6. @Catman

    Ideally Thorium energy would be so cheap wouldn’t need to cut many deals. But I’m in principle all for peeps helping each other out on the board, and AW could start negotiating some discounts and vouchers for us. Cut price cricket tickets, City breaks for boomers etc.

  7. ALEC & PETE B
    Lol. We’ll soon be able to have 10mW vacuums if we want!

    Very drole.


    I’m sorry what you are suggesting makes no sense to me.I believe the reality is that whole of the UK is leaving the EU, what parts of the UK do after that event would depend on many things. In the case of Scotland for example I am not sure that they would vote to leave the UK in the event of a second referendum. Why would they when Scottish exports to England represent 67 % of total Scottish Exports?

  9. @Rich

    Well, Liverpool being a port, one might expect them to be more Outward looking and internationalist beyond tribal reasons…

  10. Yep, can unwind the 173 pieces of legislation on toasters too.

  11. Carfrew

    ““who would vote red as my grandad used to say, even if they ‘pinned a red rosette on a donkey’.”
    Has anyone actually tried that? It’s about time we knew the truth of the matter, one way or the other.…”

    Well they hanged a Monkey for spying, captured from a French ship, a couple of hundred years ago.. In Hartlepool



    Woakes is the obvious candidate.I wasn’t a fan but he does seem to have come on a bit this year, bowling faster and with more control and skill.

  13. Jonathan Stuart-Brown – “Meanwhile pre-EU Ref we were on course to be £2 trillion in debt at interest by 2020. Not even £1 trillion in debt in 2010.”

    Brexit will help here because we no longer have to put up with double irish dutch sandwiches and other EU tax wheezes, and can just legislate that profits earned in the UK must be taxed in the UK.

    It”s not like Starbucks will react by moving out – they can’t serve coffee to a Brit from Spain, so they’ll take it on the chin.

    I expect once the Treasury is at last be able to collect tax on these companies, the deficit will start to fall.

  14. How come peeps know so much about toaster legislation?

  15. @theotherhoward: ‘Why don’t you do the research yourself? I can’t be bothered. If you think I’m wrong say why preferably with a reference or just say you disagree.’

    My point is that (as far as I know) there is no relevant legislation that defines a threshold for victory in the referendum, so your claim that the threshold is 50%+1 can really be nothing more than a guess. The national counting officer made the same guess, but it’s still a guess.

    If you want a reference, I can list you a few sources that support my point by _not_ saying anything on the subject:

    The European Union Referendum Act 2015
    The Treaty of Lisbon
    The European Union (Amendment) Act 2008
    The European Communities Act 1972

    and, in the interests of balance, one source that I haven’t checked thoroughly yet:

    The Political Parties, Elections and Referendums Act 2000

  16. @ToH

    Indeed, the ability to improve oneself is for me a key skill, or metaskill when it comes to sports peeps. Jimmy has that ability too, of course.

    Not so sure about some of our political representatives…


    Didn’t you do toaster legislation at school?

  18. @Robert

    Ah, but was the monkey wearing a red rosette?…

  19. @ToH

    It’s possible I may have skipped those lessons, prolly to practice Eric Clapton guitar solos…

  20. Laszlo

    I never understood why anyone rated Alan Johnson. Did they not see his non-performance as Shadow Chancellor?

  21. HDAN

    Sorry you have wasted your time, I was talking about my general understanding of what constitutes a win in a referendum. I was not talking about the EU and as you say it’s just my general understanding of the matter (guess is the wrong word).

    In any event it isn’t of any real importance since as far as our politicians are concerned ithe EU referendum has been decided in favour of Brexit which will go ahead as outlined today by DC. Please don’t waste any more time on research for me.

  22. I visited my local BMW dealer today, he told me not to worry, as long as I show genuine remorse, he’ll still be generous enough to sell me a car, after Brexit. ;-)

  23. KEN

    Nice one Ken :-)

  24. If we beat Iceland do we get a trophy for being the top non-Eu football team in Europe?

  25. Jim Jam

    Don’t forget the Welsh.

    I’m sorry what you are suggesting makes no sense to me.I believe the reality is that whole of the UK is leaving the EU

    The future is always uncertain, but there is a fair probability that E&W will leave the EU. The probability of Scotland leaving the EU is much lower now Germany’s support is clear*, and the FM has announced she will prepare for an indyref2 if Brexit goes ahead, which was a manifesto commitment last month. What happens to NI is less clear but they have options of remaining in the UK [probably with a “hard” border with the RoI], joining the RoI via a border poll or conceivably allying with Scotland in the EU. That’s in addition to the possibility of E&W doing a Greenland while Scotland & NI take over the EU seat.

    Today’s line of the Cons in the HoC seemed to want to ignore the Greenland option, but we shall see.

    * See Reuter’s Scotland welcome to join EU, Merkel ally says of yesterday.

  27. Seeing the messy rabble outside parliament really inspires confidence in the direction of labour doesn’t it…

  28. David Cameron = Joe Hart

  29. Oops! :-)


  30. Rich

    I doubt many of them are actually labour members. Got to say that it’s fairly heartbreaking (to me, at least) to see a Labour leader appealing to the SWP to save him from his own Parliamentary Party.

    This is heading for a split.

  31. No tactics, no shape. Out of Europe twice in 4 days at this rate. Shameful fans, no friends in Europe. What a total embarrassment. I bet Juncker’s cheering on Iceland along with the rest of the wretched commission.

  32. THE OTHER HOWARD……I’m sure that with its normal efficiency, German industry will be printing the thousands of redundancy notices triggered by the results of politically naive statements emanating from frustrated European politicians and bureaucrats. Apparently, they want to commit commercial hari kiri by threatening to price themselves out of a key market, to teach it a lesson. It’s called, cutting your nose off to spite your face. ;-)

  33. S&P have downgraded the UK ‘s credit rating from AAA to AA and warn of possible further downgrades.

  34. Rich

    “Seeing the messy rabble outside parliament really inspires confidence in the direction of labour doesn’t it…”

    And what’s more, I bet hardly any of them play golf.

  35. @Barbazenzero
    I am always puzzled as to why people give links to sites which do not actually say what they suggest
    “The probability of Scotland leaving the EU is much lower now Germany’s support is clear*”
    The article actually said:
    “An independent Scotland would be welcome to join the European Union, a senior German lawmaker and ally of Chancellor Angela Merkel has said”

    1. The UK, (not England + Scotland +Wales + N. Ireland + Gibraltar), is the EU member state which is required to give notice to leave under Article 50. UK exit carries Scotland with it.
    2. Scotland is not at the moment an independent state. (If it were, a Scottish independence referendum would not be needed.)
    3. Whether such a referendum could be held is in Westminster’s hands, not Scotland’s. Ms Sturgeon does not seem in any real hurry, and the voting could certainly be delayed until after the UK left the EU.
    Scotland would then (assuming voting to leave UK) would then be an independent state seeking EU membership from outside.
    4. Herr Krichbaum is one German opinion (albeit a relevant and important one) but Scotland’s membership application would need the approval of all 27 states (or more if others join before her). Scotland’s books are balanced at the moment with funds from England. An independent Scotland might not be a positive contributor to the EU budget.
    5. I have read other reports by Reuters which clearly selected the material and the wording to suit Reuters general stance – as do all newspapers.
    Reuters has a reputation for objectivity, which has sometimes been challenged

  36. OLDNAT
    Seems SLab are also investigating the possibilities of a “reverse Greenland”.

    Interesting, thanks. Looks like Falconer knows how to use the web, and may have spotted some of the conversation here or on sites like WoS. If the SG haven’t already scheduled it in, I guess that will be proposed at Holyrood tomorrow after the FM’s EU Ref statement. At least they are trying to do something sensible for a change.

    I note that the link you provided includes: Diplomatic sources in Brussels said the only realistic route for Scotland to stay in the EU after Brexit would be to apply as an independent country.

    Now that could just be sloppy code for “this is what Brussels said in 2014”, but it might also be a coded warning that yesterday’s Scotland welcome to join EU, Merkel ally says could have been an offer to Scotland only which would make sense if they had a quick look at the Treaty of Union.

    That would indicate that Greenland won’t be on the table [perhaps unless the new UK PM grovels and says “pretty please”] and possibly that they would prefer NI to be in or out via the RoI [which may be unrealistic for some time to come].

    Time will tell, but if SLab will vote for indyref2 then the FM may be prepared [or already have] for the Greenland option also to be in the plans.

  37. @norbold,

    You can’t have it both ways mate. You were telling me after the vote they were all mindless and had been duped by Farage.

  38. At least our football team is better than our government.

    The naivete of some of the “leave” supporters on this list is so stunning, for me. Do you not understand global “real politic” at all.
    The argument in Europe between the political elites has been about how to best accommodate capital and who should do it.

    One of the best commentary i have read so far


  40. And what’s going on with Harry Kane? He looks to have the pace and manoeuvrability of a Dalek, and not one of those better modern ones.

  41. Barbazenzero

    It’s been standard practice in the SNP to game all possible options from events.

    Maybe it’s because small countries know that they need to be able to respond quickly to circumstances which will never be under their control.

    The lumbering pace of the UK is so similar to England knowing that Iceland use long throw-ins, but seemingly not bothering to plan for that, because they “know they’re bound to win”.

  42. @ Pete B

    I do not disagree with you Pete. That’s why Canada refused to participate in the invasion of Iraq under Chretien, as did New Zealand also so refuse, because the leaders of both countries understood the blowback that would come from destabilizing the Middle East.

    If you want to stop people migrating out of the “hell holes” their countries have become you need to stop stirring things up militarily and creating more “failed states”.

    We also refused to de-regulate our banks like you and the Americans did, and under Carney, who is now your Bank Governor, we weathered the global economic crisis much better than most. I saw the impacts of the 2008 recession first hand when I visited the UK in 2012, and I listened very carfully to what people were telling me on the doorstep in the 2015 UK GE>

    Post 2008, in Canada, we are now having a serious conversation about how to tackle poverty, discrimination (mysoginist and racist), and confront the atrocious relationship with our First Nations.

    Why are you not having that conversation in England and Wales?

    @ The Other Howard

    You do not seem to understand that the US is inching ever closer and closer to an all out civil war. Yesterday, for example, 40 White Supremacists demonstrating on the steps of the California legislature in Sacremento, when an angry mob of 400 people attacked them with knives and sticks.

    I am sitting here in Canada in utter bewilderment at what is unfolding in the UK, my birth home, and in the country to the south, the country with which Canada does most of our trade.

    If I thought my views were fringe I would probably shut up and get off this list, but my Canadian and other Commonwealth friends are looking at the UK and shaking our heads in great sorrow.

  43. I came across this comment on another site which I thought was very amusing.

    “But there is a sort of genius to the way Jezzer and co can match someone else’s disaster and raise them a catastrophe. It’s like Game of Thrones written and acted by the Keystone Kops.”

  44. Andy Shadrack

    I wish I could find a convincing argument against what you say.

  45. @ TOH

    “.. A trade deal with the world’s fifth-largest economy—and one of Europe’s healthiest—is in America’s interests for its own sake ”

    That will be a deal with France then .. but what about the UK ?

  46. KEN

    So much twaddle from EU people at times. Still they are panicking about the rest of the EU. The Czech’s are furious with Junker, the Czech foreign minister says “he was a negative symbol of Europe and the British rejected him”. Want’s him to resign. Too right IMO! History tells us that when people try to bully us we have a tendency to become very difficult.


    “…the Greenland option…. Cannot see it happening myself. But I accept that the EU would probably accept Scotland into the EU if Scotland eventually leaves the UK after the UK has exited the EU.

    Depending on the terms the UK has agreed with Europe of course there could mean formal borders and tariffs which would have significant effect on exports from Scotland to England and visa versa. Scotland would of course have to join the Euro as on leaving the UK it would losethe £ sterling as was explained during the last Scottish referendum.


    No great surprise but the rating will go up again in time. Having no effect on the coast of borrowing.


    Not a very good joke, just try reading the piece in the Wall Street Journal.

  48. ‘re S&P that’s the first time they have cut a triple A sovereign rating by two notches.

  49. @ The Other Howard

    “If I thought my views were fringe I would probably shut up and get off this list, but my Canadian and other Commonwealth friends are looking at the UK and shaking our heads in great sorrow.2

    Not the response I’m getting from US, New Zealand, and Australian ex business friends I’ve been in contact with. They are very supportive and wonder why we didn’t do it years ago.

1 50 51 52 53 54 55