If I were TNS or Opinium I would be rather annoyed today. Looking through social media, twitter and so forth there are lots of comments about the polls all being wrong and it being a terrible night for the pollsters, etc, etc. Both TNS and Opinium had final call figures of REMAIN 49%, LEAVE 51% – within a point of the actual result. Far from being a terrible night, they got it pretty much spot on, and should be getting congratulated.

The last general election was a disaster for all the pollsters. Last night wasn’t the same at all, it was a very bad result for some pollsters, but other companies did very well. Below is a chart of the Leave lead in the final results of all the pollsters who did a poll in the last week or so


The polls in blue were conducted online, the polls in orange were conducted by telephone. Note that ORB and Survation’s fieldwork both finished a few days before the referendum, so one cannot rule out a change in support in the days between their fieldwork and the vote itself. Disappointingly for me personally YouGov’s final poll had Remain ahead, albeit, only by two points. Unlike in May 2015 though I’ve a good idea of what went wrong (the turnout model we used for the poll weighted down people who didn’t vote at the last general election, when in reality turnout ended up being higher than the last general election), which is something that can be worked on.

During much of the campaign discussion of polls focused on the gap between telephone and online polls. The division is, as ever, really not as simple as that – Populus showed the largest Remain lead and it was conducted online, until they stopped polling a few weeks from the referendum ICM’s telephone polls were showing figures as Leave as their online polls. However, the general trend was clear – online polls tended to show a closer race than telephone polls; online polls tended to show it neck-and-neck, telephone polls tended to show Remain clearly ahead.

Many media commentators bought into the view that phone polls were “better” in some way, and should carry more weight than online polls (a debate I sought to avoid as much as possible, as there really wasn’t good evidence either way). I suspect this has played into the perception that the polls as a whole were wrong. If you’ve spent the last few months focusing on the polls showing a solid leave lead, and playing down the polls showing a neck-and-neck race, then you’d have been very surprised by last night.

The gap between online and phone narrowed during the campaign, and that was largely due to changes in online polls. The debate about the gap between phone and online polls has focused largely on potential differences in sampling – studies like that of Matt Singh and Populus found that people gave different answers to questions on things like immigration and national identity in online and telephone polls, that people in online sample seemed to be less socially liberal than people in telephone samples. In response several online pollsters adopted things like attitudinal weights to make their samples more like phone polls… perhaps, in hindsight, it should have been the other way around.

Since the error in the polls in 2015 I’ve said that the problems won’t be solved overnight. Pollsters are experimenting with different methods. Some of those things will work, some will not – it is a learning process. The record of polls conducted online is getting more promising – the performance of the mostly online polls at the May elections was mostly good, and most of the online polls for the EU referendum were either good, or at least only a few points out. While the problems of 2015 are probably not entirely cured yet, online companies are showing clear progress, for some phone polls there is clearly still work to be done.

2,711 Responses to “EU referendum post-mortem”

1 2 3 4 5 6 55
  1. Meanwhile, I wonder how the surveys for the Nats of opinion in Scotland are going…………..

  2. DSL

    Roger, are you saying a lot of leavers were just blustering (their hearts), hoping that remain won (their heads)?

    Well that’s one explanation. Whenever there was one of these ‘Wisdom of the Crowds’ questions before the referendum it got similar results. So most people expected Remain to win (70%). And it was pretty uniform across most groups – even those who voted UKIP last year split 50-50.

    One thing that is related is how serious people expected the outcome to be. Given a choice of:

    The decision we make in the referendum might make us a bit better or worse off as a country, but there probably isn’t much in it either way


    The decision we make in the referendum could have disastrous consequences for us as a country if we get it wrong

    Leave voters agreed 69-31 with the first and Remain 77-23 with the second. Remember both sides were running Project Fears so both sides could equally be scared that bad things would happen if their side lost. So as well as not thinking Leave would win, many of its supporters didn’t think it would matter much if they did.

  3. If we are going down this crazy path, a poll found 100% of Muslims are completely intolerant of Homosexuality, and over 50% wanted it to be made a criminal offence.


    Wow I guess posting the truth makes me a racist Islamophobic Xenophobe!

    You know it’s gone too far when “even reality is racist” (and yes that is a genuine quote.
    And according to Anita Sarkeesian “everything is racist, everthing is homophobic, everything is sexist”.

    The only conclusion is that this invented construct is actually meaningless !

  4. I think the new unelected Tory PM will have a fun job explaining how is new deal (if any) with the EU is any better than what we give up, likely to be just as great contributions, free movement of labour, abiding by the rules…only difference will be the enormous time and money spent staying in the same place … assuming we don’t lose a ton of business in the City and in trade in the meantime that never comes back.

    Complete waste of time and money spent on a minor issue that was the subject of a power struggle in the Tory party. They somehow managed to persuade people in the UK that the EU was responsible for all immigration and immigration was responsible for the cuts and it would go away if we voted out. Ha ha!

  5. @otherhoward

    Candy said that wanting rid of the scots was a factor in the leave vote – I doubt that very much for any significant proportion of the voters.

    I think there is certainly less desperation to keep the Scots than existed before and during Indy ref 1, and I guess that will make Indy Ref 2 easier to negotiate.

    I don’t see any Scots demanding anything at all from the English or other nations so I’m not sure what that bit refers to..

  6. @thoughtful, not at all, it makes you right. I agree with that survey knowing some muslims and it is deplorable. But I’ve also heard a lot of homophobia from the white community as well. Racism is not invented, it happens every day. My black friend was abused on the colour of her skin by leave campaigners. Nope, it wasn’t a construct then either.

    @Roger, I think a lot of leavers thought that even if it did affect the economy badly they’d be ok, because they’re not exporters……………..dun, dun, dun! Global economy, interconnected; what that be then?!

  7. I am not happy about this result to leave the EU. Somebody, BM11 just asked ” could somebody explain the currency market. For one pound, how may euros would I get compared to yesterday?”

    Well, approximately, yesterday, 1 euro was worth about £0.80 i.e. 80 pence. Today it is the other way round £1 is worth 0.80 euros. i.e. 80 euro cents. The pound is now worth less than the euro. That is the first time, as far I know that that has happened.

    It was not an overwhelming vote to leave – less than 52 per cent, but it is a vote to leave. Apparently people over 65 voted to leave and younger people voted to stay.

    I voted to stay. I’m 67. Furthermore a majority of the people in my town (Exeter in Devon) voted to stay. However the people in the rest of Devon voted to leave.

    Why did people vote to leave? The campaign to leave was passionate. The campaign to stay was not. Cameron had previously been complaining about the EU and fighting for better terms. Then he comes back to lead the in campaign. This is a change of personality that did n’t seem convincing enough.

    There is Jeremy Corbyn who was certainly not passionate about the EU. He gave it 7 out of ten. Well there were people on the leave side who were ten out of ten for leave.

    I find the choice of leave difficult to understand. In the life of the man or woman in the street, how has being in the EU affected you? For me, I cannot say that the EU has greatly impacted on my life. I would be interested to hear from people who have been affected personally.

    Take immigration for example – the people in the immigrant camp at Calais are not EU immigrants, neither are refugees from Syria etc. The leave campaign has not made any promise to cut immigration, at all. That is a fact.

    Take businesses for example. You might hear of businesses being put to extra expense to conform to EU regulations. However, business surveys of big and small businesses show that they are in favour of the EU.

    Let us take sovereignty as another example. We want to run our own affairs We don’t want to be told what to do by Brussels. We want to make our own laws. You have heard of Ireland and Scotland who are very keen on independence from Britain, but they don’t mind being in the EU.

    For these reasons, I do not understand why people argue to leave the EU.

    Is there any benefit to an ordinary person from being in EU? There is free movement between countries. You can go and live in another European country. Let’s hope that we can still do that after the vote to leave.

  8. Carfrew

    A bit of cricket commentary sounds an excellent idea…
    Thanks for the reminder.

  9. @NickP, why I’ve never been a fan of universal suff-Rage or refs. Half the population have an IQ of less than 100 and the IQ needed to fully understand all the issues is way above 100. My cat constantly votes more biscuits but it isn’t necessarily a good thing for his health if the cat gets his way all the time.

  10. Anybody sniffing a tactical deliberate weak campaign by Snp to suppress turnout??

  11. BARDIN1

    Just read all of John B’s posts and you will see what I mean. I was not really referring the opening part of Candy’s post but the rest of it. I actually don’t think that a majority of English voters care a fig about whether the Scots go or not anymore. We have been worn down by the years of whinging since Blair opened Pandora’s Box

  12. NickP

    1. Labour reneged on the Lisbon referendum

    2. “They somehow managed to persuade people in the UK that the EU was responsible for all immigration and immigration was responsible for the cuts and it would go away if we voted out.”

    Lab’s ex voters know what the consequences of mass immigration are because they live (or lived) with it.

    Pretending there are no negative consequences is the root of Lab’s problem.

  13. ADGE3

    I cannot imagine why anyone voted to stay.

  14. @DSL

    Some of the black leave campaigners were abused in the same way by remain campaigners too !

    But when the lunacy would allow me to call you out as a ‘waycist’ for failed to use the correct terminology, – person of colour (at the moment correct but subject to change without notice!), you can see that it has gone far too far.

    BTW if you have a spare couple of weeks try writing a definition for it that everyone agrees with and that fits the UK law !

    I can guarantee you will fail ! BTW what race is a packet of sausages ?

  15. Cornwall also wants the Treasury to pay them what the EU would have paid them. They voted leave.

    What a competittion for that 350 million a week (nobody will get it).

  16. I trust that all the companies said to be threatening to move to France and Germany, properly understand the additional higher taxes that they and their employees will pay. And they won’t be able to downsize without massive cost, in the event that they need to let people go in a future downturn.

    Personally it all sounds a bit like people threatening to leave the country in the event of xyz party winning a GE. It rarely happens in reality.

    So the serial petition demanders are at it again. Some sections of society do have a real problem with democracy, don’t they? If they lost again, what would they next demand, the best of five?

    Re your comment this morning about the Belgian PM suggesting a way out, it doesn’t sound like flexibly is the name of the game. Certainly no pragmatism from Merkel or the others.

  17. @Mr Jones, you are right, we do have an immigration numbers problem which is putting a lot of pressure on jobs and the state structure and housing. Doesn’t mean leaving will solve it if we still have to have the free movement of people like Norway does. In fact, it may get worse, we may now have to accept the Schengen area as Norway does even though it’s not in the EU. Imagine a situation where immigration gets worse. Imagine that DC’s deal was actually better than 2019. It could work out that way, we don’t have the bargaining chips to demand closed borders, the EU countries won’t accept that.

  18. It’s a stupid idea to have a referendum on a subject where one side is a passionate yes but probably numbers less than half and the other is a very lukewarm no, but probably a majority (not counting the simply don’t cares)

    So we have a vote where only the YES voters care, unless you attempt to ramp up the NO interest by all sorts of ramping up the importance.

    Should have used that rule where more than half the eligible electorate had to vote leave before any change could happen. At the moment all non voters or unregistered voters are discounted.

    Are we going to have a vote on every yes no issue now? Capital punishment? Fox Hunting?

  19. Yes Laslo, and Wales will chip in with you’ve got to give us the subsidy you promised to replace. Where does that money come from in a shrinking economy? If we get the recession 9/10 economists predicted, and the coffers dwindle, the only solution is expenditure cuts or tax rises. Both will hurt ordinary folk, the leave voters. Turkeys have voted for Xmas. It’s unbelievable and sad.

  20. @Adge3 – “Well, approximately, yesterday, 1 euro was worth about £0.80 i.e. 80 pence. Today it is the other way round £1 is worth 0.80 euros. i.e. 80 euro cents. The pound is now worth less than the euro. That is the first time, as far I know that that has happened.”

    Can you check your figures please. According to the BBC marketwatch (15 minute delay), 24 hours ago £1 bought 1.31 EU, and it currently buys 1.23EU.

    At present, it’s nowhere near parity, so I don’t know where you are getting your figures. The 1 day fall has been 5.2%.

    1EU currently buys £0.81, whereas yesterday it bought around £0.76.

  21. @Thoughful

    When even a glance at the article you cite shows that it found muslims who supported GLBTQ… OR WERE GLBTQ THEMSELVES. Only in the England cross-break were there none, because it was a cross-europe poll that was not weighted for individual countries, and the sub-samples per country were unrepresentative and small.

    Wilful misreporting of a poll’s contents, and gross disregard for statistical practice, on a website about polling, in order to present bigoted views as fact, suggest to me you are a racist.

  22. ADGE3

    “neither are refugees from Syria etc”

    Sure they are. The EU opened its borders to Africa and the middle east

    (4,500 a day by one route i.e. c. 1 1/2 million in a year)


    and then the EU decreed that all member states have to take a stipulated number or face punitive fines with no upper limit.

    The EU has gone completely mad.

  23. DSL

    Still saying that a lot of Blacks, european immigrants and Muslims are foreigners and not british but I get your point. This number could even closer to a third of all people living here.

    Can we both agree that the talk about race seems to be mostly useless? In these days qualifiying is way more important.

  24. And it is my opinion that an unwillingness to call out such behaviour as being racist, has led us to this divisive and toxic discourse.

  25. Alec, you are quite right. I am wrong. Thanks for correcting .

  26. @Lazlo

    We could pay every UK region what it made from the EU and still make ~190mil a week “profit”.

  27. I am pleased to say that the prediction I made some days ago was entirely correct. It was the electorate who got the final vote wrong – clearly Remain being in the lead by twelve points was the more accurate answer.

    I have every confidence that the errors introduced by the British people can be corrected with a few minor tweaks. This might involve waiting ten years for a tranche of Leavers to die off, and of course introducing voting for 16/17 year olds, but with the Cameronian Tories out the way that should be easy. We need to carry out a few other minor adjustments, but I have no doubt that the final position will be the correct one.

  28. @Bardin1

    “So you are saying that people voted leave to get rid of the Scots? Seems a bit far fetched to me.”

    I believe it’s the phenomenon known as ‘trying to find somebody else to blame’.


    “There is Jeremy Corbyn who was certainly not passionate about the EU. He gave it 7 out of ten.”

    In fairness to Corbyn, he’s been a eurosceptic for 30 years – no one would have believed him if he suddenly announced he was 10/10 for enthusiasm.

  29. I don’t want some deluded mom or pop in Burnley, sold a lie, visiting soup kitchens and food banks. But that could actually happen if the economy goes back into recession. They’ll cut welfare first of course, but the NHS and education won’t be immune. Anyone who works in the NHS already knows this to be true in real NHS expenditure during austerity. We’re just recovering and now this. If we’d waited we could have negotiated with Le Pen about immigration to the EU. But oh no, British reserve has been abandoned for jump off the ship. A country famed for not making rash decisions has gone loopy. I hope I and the economists are wrong, I pray we are. One of our big exports is education to EU nationals. They won’t feel like paying 16,000 instead of 9,000 a year to a country that doesn’t like them or welcome them. The university sector will suffer big cuts and job losses and because they do a lot of research for industry, our companies will suffer too.

  30. DSL

    “Doesn’t mean leaving will solve it if we still have to have the free movement of people”

    Given Merkel’s madness *not* having free movement is the key thing.

  31. @ Robert Newark

    Yes, democracy is complex, and probably many people do not understand it. Having said that, politicians have a problem with the concept too.

    However, referenda have no constitutional meaning in the UK, only political. Because of this, I don’t think that much will happen about the petition, unless it reaches 18 million or so.

    It is really an uncharted territory (mainly because of the set up). In theory, not in practice, administrative regions that voted remain could ask a role at the negotiations.

    Also, at least to a degree, our century has redefined at least certain elements of democracy.

    The petition site, by the way, has collapsed – the number of signatures were around 300 a second at that time.

  32. @SORBUS

    “A bit of cricket commentary sounds an excellent idea…
    Thanks for the reminder.”


    Cricket cures many ills. Quite often, it does so by displacing one misery with another, notably with the classic England batting collapse. But happily it doesn’t last forever. (Whereas storage taxes, are still there years later…)

  33. Mr Jones,
    You are not saying that Britain is obliged to take Syrian refugees, surely. Britain decide whether and how many it takes, and has been criticised over it. They are not EU citizens, and do not have a right to come.

  34. Mcclane, it’s a distasteful topic but an important one. THE BAS survery is important and the effect or driver on the vote today is huge. It will be insightful when polling cos do interview Leavers and ask why they voted leave and if prejudice was a reason. At least this link should be explored. If people are voting irrationally, we need to know because it invalidates refs altogether. We live in a representative democracr for many reasons, not least because joe public can’t be trusted to make these decisions and lacks the analysis and data to do so. If he’s also prone to being irrational, that seals it, no more refs!

  35. mrjones

    I agree entirely with you and would solve the problem like Australia did by sending them back to the place theyre coming from.

    But so far 60.000 “boat refugees” entered Italy until June 20 this year that looks so far like 340 to 350 people on average per day. You cant take a daily figure in good weather summer months and project it as the average number for the whole year. I fear increasing numbers as well but they should not reach such an invasion like level you mentioned.

  36. ADGE

    “Well, approximately, yesterday, 1 euro was worth about £0.80 i.e. 80 pence. Today it is the other way round £1 is worth 0.80 euros. i.e. 80 euro cents. The pound is now worth less than the euro.”

    I think you are talking through you rear end.

    The exchange rate closed on 23rd, at 1.30 or just under 77p to a euro and is currently 1.225 or 81.5 pence per euro.

    Source xe.com

    It may have recovered somewhat further as I’m sure I have seen it quoted at 1.24 on the TV.

    So fallen, yes but nothing like the disaster you report.

  37. I see Cornwall, who voted heavily for Leave, have issued a statement saying that they had better not lose a penny of their £60 million EU subsidy. Not sure how this fits into the ‘England are fed up with subsidizing everyone’ argument, but I’m sure it slots in neatly somewhere.

    In other matters, allow me to be the first to wish the United Republics of Scotland and London well for the future.

  38. Mr Jones, as a leaver, how will you feel if we are forced into Schengen and have more immigrants from the EU not less?

  39. @ the other Howard You say I cannot imagine why anybody voted to stay.

    Can you make a bit more effort to reply, after I gave some arguments. I said I would be interested to hear how people have bee personally affected by being in the EU.

    Have you got anything to say?

  40. Alec
    You posted whilst I was writing my post. Apologies for duplication.

  41. @Robert Newark – needs a little time for this to shake out. Belgium and some of the smaller countries (plus Italy) seem to be taking a softer line, while France, Netherlands and the Commission are pushing very hard. Merkel seems a touch in the middle.

    What is interesting are the calls from within Brussels in particular to speed the process. The parliament president is saying they are consulting lawyers to find ways to force Article 50 into effect without waiting for the UK to activate it.

    Para. 2 says – “A Member State which decides to withdraw shall notify the European Council of its intention.”
    I suppose the Commission could take the referendum vote as notification, but that could be a problem as para. 1 says “Any Member State may decide to withdraw from the Union in accordance with its own constitutional requirements.”
    As the referendum isn’t legally binding in UK constitutional terms, I think the EC would be stretching things.

    Overall, what this tells me is the fear of contagion. The EC wants this done fast and hard, before others start to gaze towards the exit door. For the UK, delay may be helpful, as extending the timescale might make the hardliners more twitchy and more prepared to make concessions to get the matter sorted. Set against this, uncertainty is bad for both.

    How the EU 27 now behave will be interesting. If they opt for a punitive approach, it will further enhance their image as an insular, self serving and somewhat selfish body. Ultimately, this could end up being self defeating.

    Also, they are saying that a trade agreement could take 7 years at best. This also betrays much of what is wrong with the EU. A trade deal with a fully integrated member state opting to leave should take no more than a couple of weeks to agree, unless there are fundamental differences. That the EU cannot work these things out except on a timescale of decades is one of the reasons people want to leave.

  42. ADGE3

    The plan was agreed last september


    I haven’t heard much about it since.

  43. Jesus, so we’ve left the EU to get our fish back. Unfortunately, much of that is caught in Scottish waters so it’ll be going to the EU nonetheless Anarchist.

  44. Other flies in the ointment in the coming months are the potential by-elections if it turns out the Tories cheated their expenses in 2015.

  45. Alec

    A few weeks? Ha ha ha. They are a bit more complicated than house conveyancing you know.

    I mean, really.

  46. DSL

    There’s no way we can be forced into it. If the Con negotiators agree to it then it’s back to square one – trying to get out from under a europhile political class that wants to turn itself into a hereditary oligarchy.

  47. I’m surprised there isn’t more discussion here about the huge disparity in voting between London and the rest of England. For us “knuckle-draggers” (@DSL) outside the capital, we have thought for years that the economy and dispersion of resources is designed to benefit London and nowhere else.

    Threats about damage to the economy cut no ice with us because our economy has been struggling for years and is getting steadily worse. When you say damage to the economy, you mean damage to the financial services economy in London. So what? London has been doing well out of the EU and we see no benefits from that, only continuous downward pressure on wages. If we had voted to remain, would the metropolitan elite suddenly have woken up and started creating wealth elsewhere? There was no indication that would happen.

    In London, they now worry about the future, well, join the club. We’ve been worrying for years. Voting Leave may or may not help us, but voting Remain would definitely not have done. If the world of the elite is now falling apart, they only have themselves to blame. They should have taken more notice of us knuckle-draggers before.

    You reap what you sow.

  48. So… I’m trying to hope for the best. Maybe we get a decade or so being a Norway like ‘Associate Member’, still having to follow all the EU’s rules, but without any say. So less apt to cock things up for the rest of Europe with out idiocy. No more of our government forcing through EU regulations, then claiming those regulations were forced on them by Brussels.

  49. McClane

    yes, fair point

  50. @exactprediction,

    Just following up on my promise to correct how wrong your constant and relentless remain trumpeting 56 / 44 was.

    Suggest name change – poorprediction?

1 2 3 4 5 6 55