If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so


The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.

2,711 Responses to “EU referendum post-mortem”

1 53 54 55
  1. The TV media have thrown away their mask of impartiality completely since the vote – the result of all coming from the same AB Oxbridge background I guess and having no experience of anything else.

  2. Labour are like a naff socialist student union party now. Principles are one thing, but Michael Foot had principles. You have to have some compromise to be electable and get some of the middle class vote. Sorry but that’s just how it is.

  3. @candy: ‘The FT is reporting that the United States has issued a warning to the EU not to be vengeful.’

    Well… the current US Secretary of State is a descendant of Henry II. I think we can all imagine how His Majesty Henry would have reacted to any attempt by England to exit the Angevin Empire. And I think we can be confident that, by comparison with that, the EU _will not_ be vengeful.

  4. @assiduosity
    Thanks for your post of 27 June 10.44 p.m.
    You point out that a new referendum on the terms of new deal with EU is still possible. Yes you are right and that would be the best solution.

    However, as you agree, there are some practical difficulties, which just make things more interesting. @Tancred thinks that article 50 will be triggered, but that UK will seek assurances that they can stop the article process and put a new deal to the EU after another referendum is held.

    What could the new deal be from another referendum? It could be, a return to full EU membership. That is what I and many others,perhaps yourself, would like. However, there are others who do not want free movement who would not like the EEA – Norway model, which gives access to the single market.

    I would say that the EEA Norway model is a compromise between those who on one hand, would like full EU membership, and those on the other hand who would sacrifice access to the single market because of their opposition to free movement of people.

    The EEA model is thus a sort of mid-way position, not pleasing one side, and not pleasing the other. However, perhaps it does enough to gain general acceptance.

    The ideas of some parts of the leave side to cut ourselves off from the single market, and to go boldly with self belief in search of new opportunities, seem to have the haziness of the ideas of Jeremy Corbyn and John Mc Donnell , and cause similar worries to the business and finance sectors.

    You have noted previously that you think that now when attention is on the EU question, it is a good time to replace the Labour leader without attracting too much attention. Yes, during the First World War in 1917, the Russian government was overthrown by the Bolsheviks. a good time to do it, whilst people’s attention was elsewhere. That is the comparison that I thought of.

    The position of Scotland plus Northern Ireland and Gibraltar is also fascinating. Here surely another referendum is essential.

    I wold guess that people who want to stay in EU might have to exit the EU, with EEA terms, and then come back into the EU. The EU 27 might look at that very favourably.


    And the Mail, Express and Sun are all impartial, right? What planet are you from?

  6. @ADGE3

    I don’t think a return to the EU will be an option, as that has already been voted on. I think the choice will be between EEA membership and WTO rules. If people choose the latter then we’ll be well and truly f**ked.

  7. @HDAN

    I don’t think they will be vengeful but they will be firm. If I was in their position I would certainly be firm and not allow any sort of cosy tailored deal.

  8. The German economy is not in a good position therefore any trade deal that inhibits by way of tariffs trade with the UK will undermine its huge UK export market. We must also remember that Germany and the EU do not have the monopoly on tariffs we will be free to apply tariffs to goods from the EU too. Regardless of an irate and bitter Junker I think common sense will overcome the upset. As for a second referendum, well that would really be a gift to UKIP as the electorate would see it as a sell out by the LIBLABCONs to whom the Brexiter’s were sending the message that Westminster is not listening to us.
    Interesting times ahead and who knows where too next

  9. After reading many helpful posts here, I have come to a definite conclusion, which has not been mentioned by another poster – as far as I know.

    Here is the conclusion – comments are welcome.
    You can’t please everybody. However this may come close to pleasing everyone.

    There has been a referendum vote to leave the EU. the next step is deciding the new agreement which the UK will have with the EU.

    At one extreme is the proposal to leave the single market and free movement of labour for EU citizens. This would be the WTO option or the Canada option.

    The midway option is the EEA Norway, type arrangement in which we leave the EU but stay in the single marker and have free movement of EU citizens.

    At the other of the spectrum is the option to stay in the EU as a member as we are. I would say that this option is ruled out for now, following the referendum vote.

    My conclusion is that the mid-way option, the EEA Norway type option is the right option.

    Here is why. If you are an ultra eurosceptic, this is a first step, towards a new era for Britain.You want to go further, but this can be reviewed in a few years time, and Britain can decide to leave the single market. Therefore the extreme eurosceptics have that to look forward to. Why not go further now? The referendum showed a small advantage for the leave side. It was about 3,8 per cent. This is not at all an overwhelming mandate to carry out the most extreme policies which the remain side would find unacceptable. Therefore, take the first step, go gently. Choose the less extreme EEA Norway model with the intention of moving on later to more extensive changes later.

    From the point of view of those in favour of staying in the EU, the midway option is also a good choice. From their point of view, they hope to change back to joining the EU., in a few years time. The public mood may change either way. You keep both possibilities alive. You have n’t burnt your boats. You retain the flexibility to go either way,to the more eurosceptic path, or back towards joining the EU again.

    There was only a small majority in favour of leave at the referendum. This cautious approach, taking the midway option of an EEA agreement seems to be the best approach in these circumstances.

    Comments anyone?

  10. @ADGE3

    Not sure. The EEA offers all the disadvantages of EU membership with few of the advantages, certainly for consumers. And the ardent Brexiteers will also be disappointed with this – they want no more payments to the EU and no more immigration. I don’t see how this can be sold to them.

  11. ADGE3

    That is well reasoned but there is a huge problem with the EEA option. It means the free movement of EU citizens and there seems to be a large percentage of the LEAVE voters that just don’t want that, indeed would be feel betrayed by that. Labour supported REMAIN, admittedly with virtually no help from it’s Leader, but huge numbers of Labour voters voted BREXIT in the NE, NW, EM,WM and Wales and probably immigration was the most important factor for them.

    All of the current candidates for the job of PM accept that there has to be some form of control on immigration with Aussie style points system being favourite.

1 53 54 55