Donald Trump has been citing Brexit as the model of how he could win the election despite expections, his surrogates of how there might be a shy Trump vote, like Brexit. So what, if any, lessons can we learn about the US election from recent polling experience in Britain?

In 2015 the British polls got the general election wrong. Every company had Labour and Conservative pretty much neck-and-neck, when in reality the Conservatives won by seven points. In contrast, the opinion polls as a whole were not wrong on Brexit, or at least, they were not all that wrong. Throughout the referendum campaign polls conducted by telephone generally showed Remain ahead, but polls conducted online generally showed a very tight race. Most of the online polls towards the end of the campaign showed Leave ahead, and polls by TNS and Opinium showed Leave ahead in their final eve-of-referendum polls.

That’s the first point that the parallel falls down – Brexit wasn’t a surprise because the polls were wrong. The polls were showing a race that was neck-and-neck. It was a surprise because people hadn’t believed or paid attention to that polling evidence. The media expected Remain would win, took polls showing Remain ahead more seriously and a false narrative built up that the telephone polls were more accurately reflecting the race when in the event, those online polls showing leave ahead were right. This is not the case in the US – the media don’t think Trump will lose because they are downplaying inconvenient polling evidence, they think Trump will lose because of the polling evidence consistently shows that.

In the 2015 general election however the British polls really were wrong, and while some of the polls got Brexit right, some did indeed show solid Leave victories. Do either of those have any relevance for Trump?

The first claim is the case of shy voters. Much as 1948 is the famous examples of polling failure in the US, in this country 1992 was the famous mistake, and was put down to “Shy Tories”. That is, people who intended to vote Conservative, but were unwilling to admit it to pollsters. Shy voters are extremely difficult to diagnose. If people lie to pollsters about how they’ll vote before the election but tell the truth afterwards, then it is impossible to distinguish “shy voters” from people changing their minds (in the case of recent British polls, this does not appear to be the case. In both the 2015 election and the 2016 EU referendum recontact surveys found no significant movement towards the Conservatives or towards Leave). Alternatively, if people are consistent in lying to pollsters about their intentions beforehand and lying about how they voted afterwards, it’s impossible to catch them out.

The one indirect way of diagnosing shy voters is to compare the answers given to surveys using live interviewers, and surveys conducted online (or in the US, using robocalls – something that isn’t regularly done in the UK). If people are reluctant to admit to voting a certain way, they should be less embarrassed when it isn’t an actual human being doing the interviewing. In the UK the inquiry used this approach to rule out “shy Tories” as a cause of the 2015 polling error (online polls did not have a higher level of Tory support than phone polls).

In the US election there does appear to be some prima facie evidence of “Shy Trumpers”* – online polls and robopolls have tended to produce better figures for Donald Trump than polls conducted by a human interviewer. However, when this same difference was evident during the primary season the polls without a live interviewer were not consistently more accurate (and besides, even polls conducted without a human interviewer still have Clinton reliably ahead).

The more interesting issue is sample error. It is wrong to read directly across from Brexit to Trump – while there are superficial similarities, these are different countries, very different sorts of elections, in different party systems and traditions. There will be many different drivers of support. To my mind the interesting similarity though is the demographics – the type of people who vote for Trump and voted for Brexit.

Going back to the British general election of 2015, the inquiry afterwards identified sampling error as the cause of the polling error: the sort of people who were able to be contacted by phone and agreed to take part, and the sort of people who joined online panels were unrepresentative in a way that weights and quotas were not then correcting. While the inquiry didn’t specify how the samples were wrong, my own view (and one that is shared by some other pollsters) is that the root cause was that polling samples were too engaged, too political, too educated. We disproportionately got politically-aware graduates, the sort of people who follow politics in the media and understand what is going on. We don’t get enough of the poorly educated who pay little attention to politics. Since then several British companies have adopted extra weights and quotas by education level and level of interest in politics.

The relevance for Brexit polling is that there was a strong correlation between educational qualification and how people voted. Even within age cohorts, graduates were more likely to vote to Remain, people with few or no educational qualifications were more likely to vote to Leave. People with a low level of interest in politics were also more likely to vote to Leave. These continuing sampling issues may well have contributed to some of those pollsters who did it wrong in June.

One thing that Brexit does have in common with Trump is those demographics. Trump’s support is much greater among those without a college degree. I suspect if you asked you’d find it was greater among those people who don’t normally pay much attention to politics. In the UK those are groups who we’ve had difficulty in properly representing in polling samples – if US pollsters have similar issues, then there is a potential source for error. College degree seems to be a relatively standard demographic in US polling, so I assume that is correct already. How much interest people have in politics is more nebulous, less easy to measure or control.

In Britain the root cause of polling mishaps in 2015 (and for some, but not all, companies in 2016) seems to be that the declining pool of people still willing to take part in polls under-represented certain groups, and that those groups were less likely to vote for Labour, more likely to vote for Brexit. If (and it’s a huge if – I am only reporting the British experience, not passing judgement on American polls) the sort of people who American pollsters struggle to reach in these days of declining response rates are more likely to vote for Trump, then they may experience similar problems.

Those thinking that the sort of error that affected British polls could happen in the US are indeed correct… but could happen is not the same as is happening. Saying something is possible is a long way from there being any evidence that is actually is happening. Some of the British polls got Brexit wrong, and Trump is a little bit Brexity, therefore the polls are wrong really doesn’t carry water.

xxxxxxxxxxxxxxxxxx

*This has no place in a sensible article about polling methodology, but I feel I should point out to US readers that in British schoolboy slang when I was a kid – and possibly still today – to Trump is to fart. “Shy Trump” sounds like it should refer to surreptitiously breaking wind and denying it.


451 Responses to “What can British polling mishaps tell us about the US election?”

1 6 7 8 9 10
  1. SOMERJOHN

    Wrong again, my approach to managing the economys hasn’t changed at all. I want us to pay our way in the World, and reduce the size of the state, one of the many reasons I voted Brexit. Certainly I’ll join i’ll join you in saying Oh happy Brexit days. Certainly having one of those today.

    :-)

  2. Why don’t people see that the ‘good’ economic news has precisely nothing to do with Brexit? The weak pound has helped exports and for the rest the good economic results would have happened anyway. Brexit has not yet happened! I would like to see the situation in 2020 and see if there is still good news then – that will be the real litmus test.

  3. it’ll be Brexit 2 imo, landslide (in college votes) for Trump

    the reason is a small number of billionaires profit greatly from cheap labour economics – whether mass immigration or off-shoring – and as a result of all the money the billionaire class has been making the media and political class have been *persuaded* to cover up the negative consequences of mass immigration and off-shoring on the ex working class and new three-part-time-jobs-at-once working class

    it’s the total disconnect between what has actually been happening for the last 20 years and what people above a certain income level *believe* has been happening that causes the surprise

  4. @MILLIE

    “Many regional airports do not need extra runways. They already have surplus capacity.
    I probably would have gone for Gatwick, the cheaper and more deliverable option, combined with increasing usage at regional airports. Surely raising the status of Manchester and Birmingham are obvious in the light of Northern Powerhouse and Midlands Engine.
    Here in Exeter, we have an airport that is virtually disused, and which can take 747s. And in Plymouth, to widespread dismay, the airport has closed.”

    Well, it depends. I think that we have too many regional airports and the better approach would be to concentrate flights among a limited number of airports instead. You mention Exeter, but does Exeter really need an airport when you have Bristol not too far away, which could easily take on more flights? Likewise, in the north you have Liverpool and Leeds-Bradford, both of which could be closed in favour of expanding Manchester. I think the problem is the fragmentation of flights among several airports, whereas I prefer concentration into fewer, but more regional airports as opposed to the London-centric strategy adopted currently.

  5. @MRJONES

    Well, Trump seems to be fighting back a little. Polls show him narrowing Clinton’s lead slightly, though I think it’s now much too late for him to come back. He needs to pull out something really damaging on Clinton for him to have a chance of winning now.
    It’s significant that since the campaign started Trump has never actually been ahead of Clinton in the polls – this tells me that Clinton is probably much further ahead than the polls imply.

  6. Tancred

    “He needs to pull out something really damaging on Clinton for him to have a chance of winning now.”

    wikileaks already did that – media not reporting it but spreading via internet anyway

    what he needs to do is motivate the hidden vote which is what his speeches seem to be aimed at

  7. S Thomas

    Might be an idea to read the comments policy for this site – while you eat your cereal.

  8. CambridgeRachel – “However in relation to trump do we have any hard evidence that his voting bloc is similar to the brexit voting bloc?”

    It is not similar because America is a very different place. But something bad appears to be happening in middle America. Homicides are trending up, after about two decades of falling (and remember their figures start from a much higher level than ours).

    And mortality rates in general are up for the first time in decades – they believe the cause is overdoses from drugs, alcohol, plus suicide. In other words despair, and it appears to be affecting people in middle age – aged 55-65, and it is across the board, enough to show an increase for the whole population. If you are into demographics it looks a bit like the start of what happened to Russia in the 1990’s.

    We arn’t in an American situation in Britain, but we were edging there, and thankfully Brexit should put an end to it. People keep talking about how Brexit was down to this group or that group. They miss the bigger picture, which is that in order to get 52%, an extraordinary improbable alliance was formed between the pretty Tory shires, the Midlands industrial cities and the working classes of the north, to say “the condition of Britain matters, sort it out”. The nicest thing about Britain is that we care enough and trust each other enough to make these alliances. In the United States trust seems to have completely evaporated.

  9. TANCRED

    Again if you actually bothered to read other peoples posts you would find that nobody has claimed that. It’s just what it is, good economic news. Mind, it’s not what Osborne or the IMF of the CBI thought would happen in the first three months after Brexit though is it?

    Just enjoy the good news like the rest of us, whether you are a Brexiter or Remainer.

  10. should read ….or the CBI……………

  11. Candy

    Theres a 20 year lag between abortion laws and homicide stats, basically when abortion is made easier murder rates go down 20 years later. I think you will find that 20 years ago restrictions on abortion started to get popular in the states.

  12. Somerjohn

    Just an interesting quote for you from the BBC:-

    “A senior executive at Nissan Europe, Colin Lawther, said the company had received “no special deal” from the government.

    “It’s just a commitment from the government to work with the whole of the automotive industry to make sure the whole automotive industry in the UK remains competitive,” he told the BBC.”

  13. Mr Jones

    I don’t think trump can pull this one out, his core message about a rigged economy and political system resonates strongly but all the other carp that goes with it is going to sink him

  14. “We arn’t in an American situation in Britain, but we were edging there”

    http://foreignpolicy.com/2016/04/28/americas-suicide-epidemic-is-a-national-security-crisis/

    “The age-adjusted suicide rate in the United States increased a staggering 24 percent from 1999 to 2014.”

    the driving force behind this is the same in US and Europe (imo) but from the outside it looks to me like the US has two additional factors which magnify it much more than Europe

    1) if you’re screwed in the US you’re much more likely to be homeless as a result

    and

    2) the means of suicide are easier, guns of course but also the easy availability of opiode pain killers

    so increased motive and better means and opportunity

    in Europe the same people just slowly drink themselves to death so it’s less noticeable

  15. CambridgeRachel

    i agree it looks that way on the surface and maybe that’s how it will turnout – however my post just before i read yours explains why i think the “Brexit” effect in the US is potentially much larger

  16. TOH: “It’s just a commitment from the government to work with the whole of the automotive industry to make sure the whole automotive industry in the UK remains competitive,”

    Well, of course if the govt offers subsidies to Nissan it has to offer them to its competitors too

    At this stage, all they will have had to do is say, “We will work hard to achieve continued tariff-free access to the EU market and we are confident this will be achieved. If, despite our best efforts, no deal is achievable, we guarantee that we will find a way to neutralise the effect of tariffs on your business.”

    Potentially a very expensive commitment, of course, and fraught with problems regarding what’s permissible under WTO and EU trade rules.

  17. AW did not infer any causal connection between educational level and leave or remain, but if he had gone further and linked educational level with occupation, and then reported a correlation between membership of elementary occupations (agriculture, processing, services) and a rejection of the free movement of labour and thus the Single Market he would still not have been trying to persuade anyone that this link was the basis of voting for leave. That would be an inference that others might draw, including the editors of the Sun, Mail and Express, given the emphasis on immigration in the debate and on evidence that immigrants have tended to take low paid jobs to the exclusion of British born workers and in so doing have depressed wages. Editors and readers of the broadsheets and Mirror might be more likely, as having access to more complex and detailed information, to see that that was not the whole story, especially in respect of long-term productivity and working rights agreements offering stability within the UK economy and in Europe as a reason for staying in the EU.

  18. SOMERJOHN

    “Well, of course if the govt offers subsidies to Nissan it has to offer them to its competitors too”

    I see you have done it again, speculating in a way not supported at all by the quote from Colin Lawther of Nissan. You forgot the IMO again, your speculating with no facts to back it up.

    LOL

  19. TOH

    Re Lawther’s comment

    It always seemed unlikely that the UK were planning to give Nissan (or any other company) some kind of “special cash deal” or anything else that was going to be illegal.

    The interesting question (which I suspect you and I are too old to ever get an answer to :-) ) is what was said by the UK Government that successfully persuaded Nissan that “the whole automotive industry in the UK remains competitive” – given the doubts that exporting industries have expressed about the effects of losing membership of the Single Market.

    I suspect it had to be something more than political rhetoric!

    Also noteworthy that Faisal Islam reports that Business Secretary Greg Clark tells me: “It’s very important that businesses like Nissan and others help shape our negotiating mandate”.

    Interesting times!

  20. TOH

    My remark that you’ve quoted isn’t speculation, it’s a statement of the obvious. Or do you think that the government can offer subsidies to one car company and not another?

  21. john Pilgrim.

    That is a long way round of saying that you know better than the thickies(in your opinion) who dared to have a different view to you.

    unless it is a technical polling response why mention unless the intention is to disparage.

    we start by disregarding the views of some and then who do we move onto? which is the next group not worthy of us higher mortals?

  22. Somerjohn

    “My remark that you’ve quoted isn’t speculation, it’s a statement of the obvious. Or do you think that the government can offer subsidies to one car company and not another?”

    There is no indication at all that the Government have offered subsidies to anyone. A view supported by the quote from the Nissan executive. Pure speculation on your part.

  23. OLDNAT

    “It always seemed unlikely that the UK were planning to give Nissan (or any other company) some kind of “special cash deal” or anything else that was going to be illegal.”

    Exactly, amazing that others speculate as they do.

    A sensible post from you on the subject as usual, and yes these are interesting times.

  24. I give up. You said a remark of mine – which you quoted – was speculation. I showed that it wasn’t. You changed the subject.

    It’s as if you’ve just been clean bowled, stumps scattered to the boundary, and you say ,”ah, that was offside.”

  25. CBC on the (not yet dead) CETA deal

    http://www.cbc.ca/news/politics/belgium-canada-eu-trade-deal-1.3823624

    Looks like Canada (and its provinces who were also part of the process), and the EU have created some latitude on the “investor court” idea – which was a major concern for some about both CETA and TTIP.

    “The European Court of Justice will rule in the coming months on exactly who has the jurisdiction over this issue — the European Commission or individual countries. Belgium’s declaration Thursday said it will explicitly ask this court to rule on whether the investment protection system is in line with EU rules, as Wallonia continues to express reservations.”

    Some way to go on this.

  26. old nat.

    yes -thank you i will. I, like you, enjoy some milk over my condesension in the morning

  27. @Somerjohn

    Reuters is reporting that all Nissan has got is an assurance that they’ll get help IF things go pear-shaped (and if that happens I expect the govt will be scattering money to everyone).

    The govt is gambling that it won’t go wrong, and by itself Nissan’s decision to invest, tilts things in the direction of going well.

    Nissan could have said “this assurance isn’t worth the paper it is written on”, but then Brexit Britain would have turned on them in the same ferocious way they turned on Unilever, and that would have damaged their brand. So they made the sensible decision.

  28. TOH

    “A sensible post from you on the subject as usual,”

    Thanks – but one possible exchange between the UK Government and Nissan might have been along the following lines –

    Nissan – “We want a subsidy if we face tariffs exporting to the EU”

    UK Gov – “But we haven’t decided what to do yet, and we need businesses like yours to tell us how you can remain competitive. After all politicians know little of such things”

    Nissan – “That sounds interesting. So, after all, you might ignore the froth, and just pull the UK out of the EU – but not the Single Market or the Customs Union”.

    UK Gov – “Well, we can’t say anything for certain. It’ll depend on what our business advisers tell us. You will be one won’t you?”

    Nissan – “Happy to serve!”

    There are, of course, many other possible conversations. :-)

  29. OLDNAT

    Thanks for the update on CETA, not as straightforward as originally reported then. Will be watching with interest.

    Possible conversations. Of course, but the point I was trying to get across was that we don’t know, we can only speculate which was what Somerjohn was doing, speculating. Clearly he doesn’t like it when he is caught out doing what accuses others of……………speculating without any hard facts, which was why I was so amused.

    Taking a Granddaughter to the V & A tomorrow, a treat for both of us.

  30. Candy

    I think we can agree! There’s little difference between your speculation (nod to TOH) of what took place and mine, which was, a few posts back:

    At this stage, all they will have had to do is say, “We will work hard to achieve continued tariff-free access to the EU market and we are confident this will be achieved. If, despite our best efforts, no deal is achievable, we guarantee that we will find a way to neutralise the effect of tariffs on your business.”

  31. TOH

    Enjoy your visit. I’ll be taking granddaughter and grandson to gymnastics, then possibly to the Kelvingrove Museum.

  32. OLDNAT

    Sounds good, have a nice day as they say.

  33. TOH: “Clearly he doesn’t like it when he is caught out doing what accuses others of……………speculating without any hard facts, which was why I was so amused.”

    Oh, for heaven’s sake, what are you: 75 going on 13?

    I have no problem with reasonable, well-founded speculation. Can you not see that your “Clearly he doesn’t like it…” is a prime example of speculative comment?

  34. S Thomas.
    Well, if the cap fits, as my old Mum used to say. Actually I do think that thickies have every right to their view. It’s Boris and Gove, posh boys pretending to be thick enough to believe their own propaganda that get up my nose..

  35. @Somerjohn

    I expect the govt pointed out to them that the drop in sterling more than offsets any tariffs that would be applied.

    The govt will only need to help if they are facing both tariffs and a strong pound, which would make exporting difficult. But then a strong pound would be reflecting a strong economy, so the govt could just shrug if that scenario comes to pass!

    Nissan have made a commercial decision that they’re better off here.

    We saw something similar from AXA’s commercial unit. Immediately after the referendum they paused their plans to build the tallest tower in London. Today they’ve decided to unpause it and go ahead. What has changed is the mutters from the banks that they’d rather relocate to New York than to anywhere in Europe. So AXA are gambling that in fact they’ll all stay here and let offices from them, hard brexit or not (New York doesn’t work well because it is in the wrong time zone).

    When businesses relocate they usually don’t announce it, they just do it, very quietly. What we are seeing is a bunch of businesses shouting quite loudly because they are desperate NOT to go. They’re acting just like the Calais migrants in pretending that Europe is an utter hellhole. I wonder what they’re seeing over there that the rest of us are missing?

  36. @Tancred

    You may well be right about too many regional airports.

    But that was precisely my point: we have lots of spare capacity, which undermines the case for Heathrow expansion.

    Btw, Exeter is a long way from Bristol airport ( an hour and a half in the car ), and Exeter serves all those communities further to the west. In the absence of Plymouth’s airport, and Newquay all but defunct, we need a fully functioning regional airport at Exeter, which is much better served by rail and motorway than Bristol. Nothing wrong with Bristol – excellent airport – but we also need Exeter.

  37. Millie: Btw, Exeter is a long way from Bristol airport ( an hour and a half in the car ), and Exeter serves all those communities further to the west.”

    Yes, it was a bit rich for a Londoner to say Devonians should go to Brisol for a flight. I’ve yet to meet a Londoner who would react with anything other than incredulity and horror to the suggestion that they might catch a flight from Birmingham.

  38. MR JONES
    ‘it’s the total disconnect between what has actually been happening for the last 20 years and what people above a certain income level *believe* has been happening that causes the surprise’

    There is an interesting film on iplayer called Hyper-Normalisation. A phrase coined in the Soviet Union where what appeared in the media had nothing to do with reality. The film suggests we are in a similar state now in the West.

    http://www.bbc.co.uk/programmes/p04bkttz

  39. Is it a reverse Coué? Every day and every way this is getting worse and worse?

  40. @somerjohn

    CH4 reporting that Nissan sources have said that “compensation” could take the form of Government funding R&D. So yes it does look like UK Government has undertaken to provide some form of funding to compensate for any Brexit impact.

  41. As to the airports, I’m pretty sure that there are some out of use RAF bases …

  42. @ Somerjohn

    London to Leeds: 194 miles
    Penzance to Bristol: 189 miles

  43. @SOMERJOHN

    “Yes, it was a bit rich for a Londoner to say Devonians should go to Brisol for a flight. I’ve yet to meet a Londoner who would react with anything other than incredulity and horror to the suggestion that they might catch a flight from Birmingham.”

    I’m not a Londoner – I reside between Reading and London. But point taken, having Bristol serve all of the south-west would mean a long journey for a lot of people; still it’s much less than travelling to Heathrow, is it not?

  44. @MRJONES

    The hidden vote I presume is the white underclass vote which rarely bothers to vote at all. Just like Clinton needing to motivate the black underclass vote. Who is better at motivating wins the election!

    True, but I really think Trump blew it with his poor handling of the debates and excessive pugnacity.

  45. New YG poll on current attitudes to Brexit and its possibilities

    https://yougov.co.uk/news/2016/10/27/what-would-make-leave-voters-change-their-mind-abo/

    Doesn’t seem to have been much discussion of it (if any).

    From my perspective, the interesting question was “Do you think leaving the EU will make it more or less likely that Scotland will leave the UK?” – and the fact that Scots responses were almost identical to the GB one.

    43% – More likely
    2% – Less likely
    18% – No diff will leave anyway
    22% – No diff will stay anyway
    15% – DK

    As usual, with such speculative questions, the DKs probably have the most sensible response!

    However, with all due respect to YG, I think the question was inappropriately phrased.

    IMO there would be a significant difference (in the Scots responses, at least) if it had been “leaving the Single Market”, as opposed to “leaving the EU”.

  46. cloudspotter

    ty, will check it out

  47. Interesting news that UKIP is backing Goldsmith in the Richmond by-election.

  48. SOMERJOHN
    “Yes, it was a bit rich for a Londoner to say Devonians should go to Brisol for a flight”

    In the 90’s Bristol was plagued with hill mists, which at that date meant incoming flights were diverted. i was on a flight from Brussels with a guy from Exeter, who had left his car at Bristol. The flight was diverted to Exeter, so he had to take the bus kindly provided for us to get to Bristol and then to drive home. He didn’t seem very amused.

  49. John Pilgrim

    Exactly the same happened to me. It’s the only time I’ve used Exeter airport since flying to Jersey from there in 1957, aged 7.

    There is a story – possibly apocryphal, but I suspect not – that Lulsgate was built as a hilltop airfield in WW2 to give aircrew practice in fog landings, and as a pair with a sea-level airfield (RAF Locking at Weston super Mare, probably) so that when one was affected by hill fog, the other would be clear, and viceversa when Locking was affected by sea mist.

  50. Interesting day’s news, with something for everyone.

    Nissan is the obvious ‘good news’ story, and it is encouraging. However, there will have been some resistance to move such a big production unit, as this carries significant costs, and it’s churlish of Brexiters to suggest that no promises of assistance have been made. Even the ministerial briefings are now saying that ‘assurances were given’ regarding neutralising any negatives arising from leaving the EU, and it’s abundantly clear that Nissan have leveraged Brexit to their advantage.

    The key question is really about how much will the government have to do to maintain industries here, and what this will mean for other government spending. The other point to consider (for everyone, but Brexiters in particular) is whether May’s government is still saying one thing in public and another in private. Nissan publicly asked for two things; tariff free trade, and the ability to move staff across the EU at will. May is publicly giving no such guarantees, but Nissan are now happy.

    We now know that May has form on facing both ways at once, so what she has promised Nissan is significant, if it diverges from what she is promising leave voters. Watch this space.

    Slightly surprised by the fuss over the GDP figures as well. No one would seriously imagine the quarterly figures commencing seven days after an event would react fully to that event, but apparently some did. The fact that we maintained modest but slowing growth tells us next to nothing about the impacts of that event, especially as the least complete data is from the latter part of the quarter.

    It’s also worth noting that services as a whole are far more likely to be negatively impacted by inflation, unless wages keep pace. We’ve already seen some suggestions that the October data is beginning to show some more consistent effects hitting the economy for the first time, so the Q3 figures, while welcome, tell us nothing of note regarding the impact or otherwise of Brexit.

1 6 7 8 9 10