If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so


The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.

2,711 Responses to “EU referendum post-mortem”

1 51 52 53 54 55
  1. Andy S
    “and confront the atrocious relationship with our First Nations.
    Why are you not having that conversation in England and Wales?”

    Because the English and Welsh ARE the First Nations, and have been consistently ignored, belittled and taken for granted by their representatives for at least 40 years.

    Even on this site we have had outbursts calling Leave voters ‘chavs’ ‘knuckle-draggers’ and other choice insults. People have had enough.

    I agree with TOH
    “History tells us that when people try to bully us we have a tendency to become very difficult.”

    I was just pleasantly surprised that there is still a majority of people with good old British bl**dy-mindedness.

  2. DAVE re Reuters

    I suggest the next para is the important one:
    “The EU will still consist of 28 member states, as I expect a new independence referendum in Scotland, which will then be successful,” said Gunther Krichbaum, a member of Merkel’s conservatives and chairman of the European affairs committee in parliament.

    Whether the text was directed at Scotland, the UK or the other 26 members you’ll have to guess. My guess would be all of them unless the other 26 members had already discussed and agreed it.

    The same English text, from Welt am Sonntag was on most of the international news services yesterday morning. There’s at least one other link on this thread from about lunchtime yesterday from a different service if you care to look at earlier pages.

    Welt am Sonntag says:
    Der Vorsitzende des Ausschusses für die Angelegenheiten der EU im Bundestag, Gunther Krichbaum (CDU), spricht sich für eine Mitgliedschaft Schottlands im Staatenverbund aus. “Die EU wird weiter aus 28 Mitgliedstaaten bestehen, denn ich rechne mit einem neuerlichen Unabhängigkeitsreferendum in Schottland, das dann Erfolg haben wird”, sagte Krichbaum der “Welt am Sonntag”.

    My German’s a bit rusty but that looks about right.

    See http://www.welt.de/politik/deutschland/article156567422/Und-zurueck-bleiben-die-verwirrten-Staaten-von-Europa.html

    The rest of your post is about future possibilities. Some of them may happen and some may not, but at least the FM is busy readying contingency plans, which seem likely to be needed given the PM’s response to Angus Robertson in the HoC today.


    Lots of jokes around. , like Die Zeit “£350 mil p/w are now free to be invested in goalkeeper training.”

  4. Blimey, even the ITV commentators have given up talking up England.


    Agreed until

    The lumbering pace of the UK is so similar to England knowing that Iceland use long throw-ins

    You lost me there. Is there a log rolling competition or similar going on somewhere?


    Your post to Dave

    I here what you say about Gunther Krichbaum and Dave and I have both outlined to you what we think will happen. If you refer to my earlier post re the “Greenland option not happening IMO” I was clear that of course Scotland could apply and then be welcomed back into the EU. The EU would again be 28 member states again as Herr Krichbaum states. What then follows in the relationship between England and Scotland then depends on the relationship the UK has negotiated with the EU. See the second part of my earlier post.

    It will be possible to rejoin the EU IMO after leaving if the UK if that is what happens but it will have consequences for both Scotland and the rUK.

  7. @ TOH

    if only it were funny ..

    if you have a link i’d be interested to read.. at 2599 comments and growing havent got time scroll right through to catch up

  8. Fitch downgrade UK credit rating to AA.

  9. @theotherhoward: ‘Depending on the terms the UK has agreed with Europe of course there could mean formal borders and tariffs which would have significant effect on exports from Scotland to England and visa versa.’

    I’m trying really, really hard not to find a delicious irony in the possibility that the people, in Wales and provincial England, who voted to end free movement of labour might find themselves forbidden to relocate to London to pursue employment opportunities.

    But actually, that’s not a feature only of “reverse Greenland” or “expanded Channel Islands and Isle of Man” scenarios. It could happen even in the event of total Brexit. I can certainly remember, back in the mid-1990s, Tory MPs for constituencies on the South Coast calling for an end to free movement of labour within the UK, because they didn’t want migrants from the North of England and the Midlands turning up on their patch. Outside the EU, there will be nothing to restrain them.

  10. Well perhaps Roy Hodgson has a clear plan for leaving Europe.

  11. I am reminded of the Private Eye front page a few weeks ago.

  12. Barbazenzero

    “Is there a log rolling competition or similar going on somewhere?”

    Ah! So you are watching the footie, after all. :-)

  13. @Pete B

    “I came across this comment on another site which I thought was very amusing.”

    To be fair, this disaster is entirely of the PLP’s making. But the comment was a good one!


    Just for you:

    “…the EU has proved unequal to the urgent tasks of reviving economic growth and resisting security threats on its eastern and southern borders. It’s time for the U.S. to get back in the game because America needs a confident, prosperous Europe as a partner to defend the West against the rise of authoritarian regimes and global disorder.

    An important first signal would be for the U.S. to invite the U.K. to begin bilateral free-trade talks that run alongside current talks with the EU. Mr. Obama may not be able to rise above his pre-Brexit taunt that Britain will move to “the back of the queue” on trade. But this would not be his first strategic mistake.

    A trade deal with the world’s fifth-largest economy—and one of Europe’s healthiest—is in America’s interests for its own sake. A two-track trade negotiation would also help the British in their negotiation over new terms of trade with the European Union by giving Britain the leverage of a U.S. alternative. U.S.-British talks could also prod Brussels to move faster and rebuff the French protectionism that is infecting the EU-U.S. talks.

    Whether or not Mr. Obama leads, Hillary Clinton and Donald Trump should.”

    Wall Street Journal todays leader article.

  15. @assiduosity says that a new referendum on the new negotiated terms is possible, post today 27 June 2.30.
    Yes, that’s seems reasonable and desirable. That’s what I have argued in favour of.

    However, having listened to Tony Blair and Stephen Kinnock people who know the system,Kinnock, particularly knows Europe. They are doubtful that a new referendum will be called. It is the case of politicians wanting to keep the power to themselves maybe.

    @tancred replying to @assiduosity is doubtful that a new referendum will take place, because all 27 EU countries have to agree to stop the article 50 procedure once it has started.

    If we get an EEA settlement it will be similar to staying in EU in two respects – the access to the single market, and secondly, free movement. of EU citizens. For me that is not too bad. Of course staying in the EU would be better.

    @andyshadrack in a long post (27June 6.30p.m.) points out the difficulty Uk on its own has in dealing with entrenched elites and power blocks. Yes, also there is the difficulty in dealing with the business community in Britain who are in large majority are opposed to Brexit, because they lose financially. Therefore, let the people have their say, let the politicians posture, but don’t let it throw money out of the window.

    Therefore, because the referendum is vague and the expectations unrealistic, it is back to practical politics and fighting a general election. That is what Stephen Kinnock thinks should happen.

  16. Michael Gove, Boris Johnson, Nigel Farage, Paul Dacre…your boys took one hell of a beating.

  17. If Scotland really prefers to become a full member of the EU and Eurozone, I wish them well, I enjoy my visits to Boisdale and will continue to give my patronage, shooting and fishing are my country pursuits, and Scotland is a wonderful place to enjoy them, I just hope they don’t confiscate my guns, at the border.
    Incidentally, what percentage of Scottish exports are to England, and can anyone tell me latest price of a barrel of oil ? ;-)

  18. The best team won.

  19. OLD

    Glad I stick to Rugby. My wife watched the match and wished she done the washing up or something more useful.

  20. @TOH

    would that would be a leader comment in the same Wall Street Journal that is owned by Rupert Murdoch by any chance ?

    No great surprises there then

  21. OLDNAT

    “The best team won.”

    She agrees with you.

  22. Hodgson never played a team, he played names. Was always coming this result. The man is an idiot.

  23. England were word rhyming with night – I was moderated.


    You are funny. The Wall Street Journal is a very well respected in the US and elsewhere. You just don’t want to be objective in your comments.

  25. See ya Hodgson. Although he was let down by some spectacularly s*it performances. Kane was woeful beyond belief and Rooneynlooked like he was about 74.

  26. Embarrassed to be English twice in one week – great…. :-(

  27. OLD NAT

    My apologies for OLD, my wife was muttering about the football at the time. :-)

  28. Oh come on Howard

    Murdoch paper supports brexit .. who would have thought it !!

    One of the best commentary i have read so far



    I was very proud to be British on Friday morning and English on Saturday when we beat Australia fr the third time to win 3-0 in the series.

  31. ” football is about managers, not players ” Brian Clough. ;-)

  32. The Other

    No problem. I’m honoured that we’ve now become pals on first name terms. :-)

  33. A lot of rumours Corbyn voted leave.

  34. I know, ToH, but some of us feel differently…


    So what, I repeat that the Wall Street Journal is very well respected, as I say you just cannot be objective.

  36. OLDNAT


  37. RICH

    If that’s right it was very devious of him but at least he got one thing right in his life. :-)

    Night all.

  38. I think the England team have misunderstood what vote Leave was about. Followed it to the letter though.

  39. Looks like the Engalnd team have run away from the interviewers as well.


  41. PETE B.
    Good Evening
    I agree with your point that the BREXIT vote shows that many people feel they have had enough.

  42. THE OTHER HOWARD……Some people simply can’t handle defeat, I put it down to the nanny State, dependency culture, and a lack of vision, plenty of examples in the Remain camp as grist to the psychology mill. ;-)

  43. Woy’s wesigned.

    Unlike Cameron, I do feel some pity.

  44. I am 48% sad to see us out of the Euros and 52% laughing my head off.

  45. I hear that England will be resigning tomorrow.

  46. @Hawthorn

    If Twitter is to be believed, Hodgson has been offered a place on Cameron’s cabinet, due to him having Europe exit experience.

  47. England seems to undergoing some kind of national mental crisis, we could all do with a holiday perhaps.

  48. Statgeek

    He’d be better than Oliver Letwin.

  49. Ken,

    Approximately two thirds of both Scotland’s exports and Imports are with RoUK!

    In 1960’s before joining the EU the UK accounted for three quarters of Ireland’s imports and exports. Now and particularly since joining the EU we only account for a quarter.

    Not because Ireland has suffered but because it has flourished!

    One of the ironic consequences of Brexit could be that Project Fear is put to the test. If the UK economy suffers then the dire warnings of the consequences of voting the wrong way will be a warning to Scots.

    If the Uk pulls through this with limited pain will Scots be. As fearful of going it alone?

    The Chinese Curse; May You Live in Intersting Times!



    Scotland would of course have to join the Euro as on leaving the UK it would lose the £ sterling as was explained during the last Scottish referendum.

    Perhaps the fact that you [and many others] keep repeating this misinformation is that you keep reading it in the Unionist press. It was even put to the previous FM in a TV interview yesterday [can’t remember which channel – possibly Sky News]. It simply is not true.

    First, as the GBP is an internally traded currency, nobody can be stopped from using it. If the UK government don’t want to share the Bank of England then so be it, but that will be at the cost of giving something else. Re the Euro, the Swedish Embassy has probably set up a premium phone line with the information that they have neither an opt-out nor any intention of joining [the optional] ERMII, which is a necessary first step before a country is allowed to join the Euro.

    Joining the Euro is compulsory, but only when ERMII has been joined and the ‘convergence criteria’ have been met.

    See http://ec.europa.eu/economy_finance/euro/adoption/who_can_join/index_en.htm


    It will be possible to rejoin the EU IMO after leaving if the UK if that is what happens but it will have consequences for both Scotland and the rUK.

    Thanks for acknowledging that.

    We’re not even really at odds here. Can we not agree that there are going to be different options facing the four nations of the UK?

    Whether so or no, the Scottish Government and even the Scottish branch of Lab are reviewing likely possibilities on the path to Brexit and doing as much contingency planning as they can so that if an option is only availavle for a short time it will likely have previously been considered and understood, so that an informed decision can be taken quickly.

    If Leave’s admission that there was no plan is true, and I have no reason to doubt it, then given that the A50 timetable is short the UK government’s contingency planning may prove less than optimal.

1 51 52 53 54 55