Jon Mellon and Chris Prosser from the British Election Study team have written a new post and paper on the emerging evidence from the BES data on what went wrong with the polls. Last month they wrote a piece, which I covered here, on some of the potential causes of error they could use the BES data to look at. Now the BES post-election data is out they’ve done so, and come back with some findings.

Firstly, late swing – the BES data finds virtually no net change at all between how people said they would vote pre-election and how they reported having voted after the election. The BES team conclude from this that late swing is unlikely. We’ve now got published re-contact data from the British Election Study, ICM, Opinium, Populus and Survation, only Survation found any obvious evidence of late swing in their re-contact survey.

Secondly, Shy Tories. This is essentially the most difficult potential cause to evidence – if people lie before the election, and lie after the election and we can’t check their actual ballot papers, how do you detect it? You need to look for circumstantial evidence. The BES team have compared levels of Tory support in their polling in different types of area, on the assumption that if people feeling embarrassed to admit voting Tory really was a problem it would be less of an issue in heavily Tory areas than in areas where no one else voted Tory. They did not find this pattern. They also have some experimental data about question order and priming, one suggested solution to the polling error. The first three waves of the BES, conducted back in 2014, randomised where it asked the voting intention question – at the start, or later in the survey. Asking it later in the survey only made a minimal, non-significant change to the Tory vote (while it doesn’t say so in the article, the full paper also makes clear it doesn’t change the Labour vote either!)

Thirdly the BES team looked a bit at sampling, specifically around age, looking at an issue Opinium have already commented on at the BPC inquiry meeting. All pollsters weight by age using various age bands such as 18-24, 25-40, etc, etc. But are people evenly distributed within those bands? Within the top age group, for example, the BES found that people over the age of 80 were underrepresented and people in their early 60s were overrepresented. Whether that makes any different to the results they can’t yet say, as it may be countered by other things like political weighting.

Finally, and most importantly, they wrote about turnout and suggested that people may have been overestimating their likelihood to vote, and that the people who were actually less likely to vote were increasingly skewed towards Labour. Currently this pretty circumstantial evidence – we don’t know if people lied about voting in the general election, but there’s evidence to suggest they might have. For example, a small proportion of people said they had already voted by post before most of the ballot papers had even been sent out, in areas where there were not any local elections this May there was still a chunk of people who reported having voted in their local elections. At the moment, these people who look as if they might be lying disproportionately break to Labour, so would explain some of the error. In their article Jon and Chris instead try modelling people’s likelihood to vote based on their demographics and characteristics of their seat, and that increases the Tory lead by 1.8%. They conclude that turnout, people saying they’ll vote when they won’t, is a major factor behind the error, thought they conclude that it’s one pollsters can probably address quite easily through a better turnout model.

Two caveats though, before you think things are solved. One – at the moment we’re going on indirect evidence of people overstating their likelihood to vote. In the fullness of time the BES are going to do a validation exercise of their data (that is, checking respondents names against the marked electoral register) so they will be able to conclusively prove whether or not there are a significant proportion of people who told pollsters they voted, but lied about it. Secondly, Jon and Chris estimate that getting turnout wrong probably explains about a quarter of the difference between the final polls and the election result, which would still leave us another three-quarters to explain…

Meanwhile there have been two new Scottish polls in the last week – both show the SNP still on course for another landslide at the Holyrood elections next year.

Survation for the Scottish Daily Mail have Holyrood voting intentions of
Constituency: CON 14%, LAB 20%, LDEM 7%, SNP 56%, Others 4%
Regional: CON 12%, LAB 19%, LDEM 8%, SNP 45%, GRN 11%, UKIP 5%
(Tabs here)

TNS have Holyrood voting intentions of:
Constituency: CON 14%, LAB 20%, LDEM 5%, SNP 60%, Others 2%
Regional: CON 13%, LAB 21%, LDEM 5%, SNP 51%, GRN 7%
(Tabs here)

301 Responses to “More BES findings on What Went Wrong”

1 2 3 4 7
  1. Hello Everybody, I have been visiting this site for a while, but this is my first post :)

    I just wanted to say that I voted in the Llay by-election, (the one in Wrexham), and I don’t think you should take it as a vote in favour of the LibDems. Most of this village knows Rob personally, I think he won despite being a LibDem, not because of it.

  2. If I may make a radical point, the purpose of opinion polls is not necessarily to accurately predict the result of an election, but rather to assess what public opinion is. The issues of registration and turnout emphasise the difference between these two things, and the Conservatives’ attempt to rig the electoral system by ensuring there are many less people eligible to vote in the average Conservative seat than the average Labour seat, will simply increase the divergence.

  3. Meanwhile, Corbyn continues to receive support at the bookmakers, with the best price available being 4/1. Cooper has drifted out to 3/1, and Kendall is now a rank outsider at 12/1. Burnham is steady favourite at even money.

    Kendall’s apparent demise is quite a signal that the ‘Blairites’, whilst strong in the parliamentary party, are not at all popular with the membership.

    I’m not quite sure why, but this is all beginning to remind me of the 1980s.

  4. MILLIE.
    Good Aftenoon to you.
    I agree; this is very much like 1981.
    There is a difference, IMO.
    Denis Healey, Roy Hattersley, John Golding, Gerald Kauffman, Peter Shore and the Kinnock team later were there to fight.

  5. 1980s ?

    Well just before the 1980s michael foot beat denis healey -only mps voted and it was quite a surprise .Then neil kinnock won a big first ballot majority using the electoral college for the first time.

    If we had the michael foot electoral system now -corbyn would be fourth without doubt.

    If we had the college back then you would think corbyn might have done even better than now .


    That’s just a rather blatant partisan post. Does not add anything to the discussion.

  7. Some interesting analysis on clp nominations from the labour list site .

  8. Chrislane,
    I don’t think it is quite like 1981 – Tony Benn was far more of a threat toLabour’s existence than Jeremy Corbyn is today.It remains unlikely that the latter will win, and I continue to believe there is a good prospect of Labour being back in the lead by Xmas.

  9. It is interesting to look back at what the polls were saying ten weeks following the Tory wins in 1983, 1987 and 1992. All were recording Tory leads of between 10 and 18% at that stage. In each case they were significantly exaggerating how they went on to perform at the following general election. Currently we have polls showing Tory leads ranging from 4 to 12% – so Labour is now performing better than at the same stage in 1992

  10. TNS poll has interesting numbers on extent to which people are currently “party loyalists”. VI for Holyrood is given by recalled WM vote.
    While some will express their main party support in constituency votes, others will take the (arguably more rational) approach of casting their 1st choice on the list, and vote tactically in HR or WM constituencies.

    Constituency VI by WM vote (committed voters, CTV)
    SNP : SNP 98%, few to other parties
    Lab : Lab 85%, SNP 8% , Con 7%
    Con : Con 80%, SNP 10%, Lab 6%
    L-D : LD 98%, few to other parties

    List VI by WM vote
    SNP : SNP 83%, SGP 8%, 4% Lab
    Lab : Lab 73%, SNP 812% , SGP 6%, Con 5%, LD 4%
    Con : Con 879%, LD 8%, Lab 6%, SGP 4%, UKIP 2%
    L-D : LD 61%, Con 23%, Lab 17%

    We’ll see if a new Labour leader makes any difference in Scotland but, on these figures, Labour are more likely to gain support from those who voted for one of the other Unionist parties last May, than from the ranks of SNP voters.

    (PS – Carfrew. I liked your “shy post” suggestion last night, but perhaps we should aim to be alliterative in our descriptions of party types responding to polls. As well as “Lying Labour” referred to by BES, we could have “Taciturn Tories”, “Lackadaisical Lib Dems”, “Strident SNP” and last but not least “Garrulous Greens” (which would include Alec and myself, though perhaps not MOG).

  11. For any Paul Mason fans, his piece in the G******n today is extraordinary (IMHO) – ‘The End of Capitalism has begun”

  12. @Catman

    Shhhh. Don’t tell Colin!!…

  13. @Oldnat

    You left out UKip…

  14. Carfrew

    The term was implicit in the list – the “Unmentionable UKIPpers”.

  15. Oldnat

    But you said the Greens were last. Anyway, you left out Plaid as well…

  16. Carfrew

    Jeez! you expect me to do all the work around here. :-(

  17. I never understood what paul was on about on newsnight and certainly dont understand his article.

    But if capitalism is finished that sounds quite good

  18. @Sunreada

    Mason’s basic idea is that more and more, information is coming to dominate economic activity, and since information is usually easy to copy and can be almost free to distribute nowadays, this could drive a stake through the traditional Capitalist model which depends on scarcity to generate profits.

    He then looks at emerging alternative approaches to consider what may replace Capitalism in the future. New forms of lending, ownership, contracts; Barter, co-operation etc., gradually supplanting Capitalism.

    It doesn’t necessarily just depend on this freedom of information, but also continued reduced costs of production as machines do more and more, while costing less (possibly I might add, if it were to happen, aided by 3D printing and eventually, if they can make it happen, nanofactories. Thorium might help…)

  19. @Carfrew @Sunreada

    A good summary

    I use Linux and it’s tied in with the open source movement. This movement has revolutionised IT in my view. It has led to projects like Raspberry PI. It has started to undermine the old IT monoliths like Microsoft.

    This is an example of how free distribution can radically change things.

  20. @OLDNAT

    “Jeez! you expect me to do all the work around here. :-(”


    Nah, don’t worry, you can just stick to euphemisms for parties etc. if you like. No one expects you to cover everything. Like the PMI (which is Alec’s gig, with Colin providing helpful critique), OrThorium. Or oil prices!!…

  21. @Catman

    Suppose I should add he says that typically some external shock prompts economic upheaval: e.g. the way the Black Death made feudalism untenable, and speculates as to what external shock might prompt Capitalism’s demise: energy depletion, climate change, ageing population, migration?

    He doesn’t mention the crisis in polling, or storage taxes, which might be considered a significant omission, but hey…

    On Open Source… Yes it has its virtues. But to what extent it’ll become dominant outside of some sectors is summat else. I find OSX quite handy: Darwin core but without the hassle…

    Lately I’m wondering if Apple haven’t put a bit of a ceiling on things like Open Source with their app store model. Apps are increasingly capable, usually very cheap, curated handily, a cinch to install and to reinstall on other machines.

  22. Well as capitalism collapses labour is in a deep depression but it seems Burnham is more foot sure than his rivals and will win.

  23. Open source software get better and better all the time.

    Not so many years back the software was hard to use, using a lot of command line work and editing files.

    It’s compatibility has improved substantially. I have loads more IT issues at work, where I am given the latest Microsoft rubbish. It grows ever bigger and still falls over.

    Many of the world’s IT internet structure is run on UNIX/LINUX systems. It has proven to be more robust more secure than similar Microsoft systems.

    I wonder how many large organisations are tempted to switch to open source software, and save a fortune in licences?

    On the shock issues, it does seem events are slipping faster than the the ability of the conventional authorities to control them.

    There looks to be plenty of uncertainty around, and another crisis could be close.

  24. On the issue of the impact of skewed figures resulting from those who say they will vote and do not.

    If you look at practically every poll for the London Mayor election, you see that “certain to vote’ is always more than 50% and when you add in the very highest category below this (almost certain to vote, or the 9th scale of certainty out of 10) it often gets to over two-thirds. This indeed was what was said days before the 2012 election…in which just over a third voted!

    It is just a point that those polls were surely comparatively accurate in predicting the final result, in spite of the fact that nearly half of those who said they would show up didn’t.

    Knowing as I did that it would be unprecedented if the turnout hit over 50%, and that it would be between a third and well below 50% (It has never hit 50%), it always seemed interesting that the poling of the outcome was as accurate as it was.

  25. Interesting Labour Leadership figure for aupporting CLPs:

    Overall nominations: Jeremy 65, Andy 62, Yvette 44, Liz 10.

    Within London: Jeremy 9, Liz, 8, Yvette 5, Andy 4

    So even in London, supposedly Liz’s best area, she is trailing Jeremy.

    Obviously Andy and Liz’s northern charm count for nothing inside the M25.

  26. Correction

    Interesting Labour Leadership figure for aupporting CLPs:

    Overall nominations: Jeremy 65, Andy 62, Yvette 44, Liz 10.

    Within London: Jeremy 9, Liz, 8, Yvette 5, Andy 4

    So even in London, supposedly Liz’s best area, she is trailing Jeremy.

    Obviously Andy and Yvette’s northern charm count for nothing inside the M25.

  27. @Millie

    I’d question your notion that the ‘Blairites’ are strong in the parliamentary party. Kendall has a fair bit of support from the remaining 1997 intake MPs but little from the 2010 intake beyond high profile backers like Chuka Umunna and Tristram Hunt, and virtually no support from the new 2015 intake. In fact only Wes Streeting of the new intake nominated her.

    The 2015 intake is dominated by ex-public sector workers (often trade union officials) and a very large chunk of ex-councillors and other veterans of the Labour machine, most of whom have positioned themselves on the left of the party. The nature of this intake – very few look to me to have serious front-bench potential – must be of concern for the future.

  28. CLP nominations mean nothing – many members are not even properly connected to their CLP and it seems that less than 10% even vote in their nomination process.

    But it does create an (albeit misleading) narrative.



  29. @Jack Sheldon

    It looks like the new intake means that white, middle-class, privately educated people will be desperately under represented…….

    Cough, cough. :-)

  30. You can see why Unite has lost a third of its paying membership since the merger. BECTU is merging with the right wing Prospect rather than McCluskey and his witless crew

  31. I can’t imagine that Comrade Corbyn will actually win it.

    But if he does , then Chris Lane’s worst nightmare-a re-run of the Labour 80s will be a racing certainty.

    People like Ummuna won’t hang around will they ?


    “Well as capitalism collapses”

    LOL, I see no evidence of that. Capitalism saw off The Nazi’s and the Soviet Union and will see off many other movements in the coming years.


    @”certainly dont understand his article.”

    That might be because of the addiction of Mason & his ilk to a world defined by ‘isms and apocalypse just around the corner.

    …or it might be because his desperate desire to make his beloved Syriza represent something meaningful has fueled this flawed search for signs of The New Jerusalem.

    I find it all too intense & pretentious & skipped a lot , but I soon have disagreement with the idea that “Information” is inimical to “Capitalism”.
    I think the opposite is true. It is lack of information which facilitates Monopolies & Cartels. Information facilitates knowledgeable consumers , aids competition & helps level the scales of price setting .

  34. A question which must haunt Mason , is the one he doesn’t mention.

    If Capitalism is about to collapse and be replaced by the Marxist New Jerusalem-why didn’t Tsipras show us how to do it?

    He had the opportunity-a Referendum Mandate to end “Austerity”.
    A growing chorus of support for Greek Debt relief.
    So why did he not do it?……Default on the Neo-Lib Bankers & their political acolytes in Berlin & Brussels-Ditch the Debt & the Tyranny of the Euro-and go it alone to build that Socialist Nirvana ?

    Why is he sacking his colleagues who wanted to.? :-

    “The highlight of the government reshuffle was the departure of
    Panagiotis Lafazanis, who had served as energy minister since Syriza came to power in January. He is the leader of the party’s hardline Left Platform faction and advocates Greece’s exit from the eurozone, as a way of restoring national sovereignty and liberating the economy from what Syriza ultra-leftists regard as the shackles of foreign capitalist oppression.”


    Why is he choosing to ignore the wishes of his voters & implement more Debt & more Austerity?

    Could it just be that unlike Paul Mason , Tsipras knows that it would be worse than the medicine prescribed by “Capitalism”.

  35. @Catmanjeff

    I don’t think that you can read too much into the Labour leadership nominations in terms of outcomes. There is clearly a widespread frustration indeed anger amongst the Labour membership at the comments of Kendall, Harman, Leslie et al. For CLPs to nominate Corbyn is a way for members to express that without much in the way of consequences, as it doesn’t commit to actually vote for him in the leadership contest. The nominations suggest to me that Corbyn will do pretty well in the leadership election, and may even make it to the final round, but I don’t think he’s going to win.

  36. @Adrian B

    You Gov are listening. I was polled yesterday. Presumably will be out tomorrow.

  37. According to New Statesman the current state of play re – CLP nominations is – Burnham 61 Cooper 53 Corbyn 64 Kendall 10.

  38. @ CMJ

    At least two of the British high street banks use Linux-based systems.

  39. @ Colin

    Syriza is not even a leftist party let alone Marxist (even if it has Marxist members). How much it is centrist was clearly visible if one watched Linke in the Bundestag (and it doesn’t define itself as Marxist either).

    Anyway, the unlikely event of JC’s election would certainly split the Labour Party, even if he is neither Stalinist nor communist.

  40. LASZLO

    I bow to your superior knowledge of the taxonomy of the Far Left :-)

  41. Has anyone seen a worthwhile analysis of the way forward for Greece? My usual news sources are BBC, Guardian and Independent (plus UKPR, of course) and all are dominated by negative commentary on the deal.

    What I would like to see is a discussion of what in Keynesian terms would be needed in the way of a new deal for Greece, kick-starting the economy with judicious capital injections. The aim might be to average 5% pa GDP growth for the next five years, for example. so what would it take to achieve that?

    I think there could well be enough in the way of guilt and genuine concern in the northern eurozone countries to make support for any necessary programme possible, provided it is seen as credible and totally verifiable and non-corrupt, i.e. not pouring good money after bad (or into the pockets of rip-off merchants).

    Maybe someone like Wiliam Keegan is beavering away on just such an article right now. And maybe Tsipras is wondering if he could be the Roosevelt of Greece…

  42. @Somerjohn

    Greece is an hopelessly bad position, and uncertainty over the long term prospects for the Euro and possible EU still exists. The deal did nothing to solve those issues.

    Perhaps someone has written something about the deal that isn’t negative. It needs to be by someone who also written ‘Thermonuclear war is good for you’, and ’20 great things about car crashes’ perhaps.


    @”I think there could well be enough in the way of guilt and genuine concern in the northern eurozone countries ”

    ………to take a one-off loss of 3% of GDP so that Greek GDP can grow by 5% pa?


    …….and if you’re little Slovenia, Malta, Estonia, Slovakia……… much guilt can you afford do you think ?

  44. RIVERS10

    Of course they were “at their peak” !!!

    They used State provided slave labour .

    There was no Minimum Wage, or Tea Breaks, These companies were not like Tesco or Rolls Royce.

    This was not “Capitalism”

  45. Palmos Analysis Poll – Seat Projection in Greek GE

    Syriza 164 (+15) : New Democracy 58 (-18) : XA 17 (=) ; KKE 15 (=) : Potami 22 (+5) : PASOK 16 (+3) :ANEL 8 (-5)

  46. somerjohn
    Hopes of a benign outcome rest on an eventual change in German attitudes, which in turn relies on a huge wave of reform in Greece amongst the requirements for which will be a confident leadership in Greece aligned to these changes.

    It all seems a tall order…but maybe just maybe Tspiras is the man who will deliver. He has a strong following despite the imposed deal. Support for exit is under 25%. The parties who rejected the deal are on very low levels of support, most particularly the right-wing coalition partners of Syriza.

    If the changes insisted upon go through with some obvious commitment, then the social democrats in the north will press for more help from Greece.

  47. At present Greece simply has a small bridging facility to enable it not to default on ECB & IMF repayment schedules.

    The new three year 86bn euro Bailout will take some hard negotiating.

  48. Corbyn still probably won’t win, but the fact that it is even being contemplated is significant. Certainly it now looks highly unlikely he’ll come a distant last. The CLP members may be unrepresentative but they are unlikely to be so different from the mass of voters that the candidate winning most nominations from them completely flops generally.

    Further up somebody said we need proper polls. I disagree. We know very little about who will be voting in this leadership election so it will be almost impossibly hard for YouGov or anybody else to get a representative sample IMO.

  49. chrisjam01


    Thanks for the info. That can often be the case with a council by-election that seems to “go against the run of play”.

    And a good thing too!

  50. @Laszlo

    There is some truth to what Colin is saying if you look at our most recent examples of fascism in power in western economies. Although fiercely anti-communist and anti-socialist, Salazar, Franco and the Greek Colonels ran crypto-corporatist economies that would be anathemas to the current crop of neo-liberal purists. It was capitalism, but not quite as we know it.

    Baled out by the Eisenhower government in the 50s, the Franco era was described by economists of that time as “the triumph of paleocapitalism–primitive market skills operating in a jungle of bureaucratic regulations, protectionism, and peddled influence.”

    Salazar in Portugal “was strongly influenced by Catholic corporatism and Leo XIII’s Rerum novarum. He favoured joint labour-management industrial commissions, compulsory arbitration, and Catholic trade unions.”

    As for the Greek junta in the 60’s “economic growth was driven by investment in the tourism industry, loose emigration policies, public spending, and pro-business incentives that fostered both domestic and foreign capital spending.” There was widespread corruption and numerous financial scandals but they did oversee a bubble of economic growth characterised by low inflation and low unemployment. But were they mending the roof when the sun was shining, I wonder? :-)

    When you look at the current economic travails of Portugal, Spain and Greece, I wonder how much of their difficulties can be traced back to their fascist pasts and the authoritarian regimes that hollowed out their economies in the 60s and 70s.?

1 2 3 4 7