Jon Mellon and Chris Prosser from the British Election Study team have written a new post and paper on the emerging evidence from the BES data on what went wrong with the polls. Last month they wrote a piece, which I covered here, on some of the potential causes of error they could use the BES data to look at. Now the BES post-election data is out they’ve done so, and come back with some findings.

Firstly, late swing – the BES data finds virtually no net change at all between how people said they would vote pre-election and how they reported having voted after the election. The BES team conclude from this that late swing is unlikely. We’ve now got published re-contact data from the British Election Study, ICM, Opinium, Populus and Survation, only Survation found any obvious evidence of late swing in their re-contact survey.

Secondly, Shy Tories. This is essentially the most difficult potential cause to evidence – if people lie before the election, and lie after the election and we can’t check their actual ballot papers, how do you detect it? You need to look for circumstantial evidence. The BES team have compared levels of Tory support in their polling in different types of area, on the assumption that if people feeling embarrassed to admit voting Tory really was a problem it would be less of an issue in heavily Tory areas than in areas where no one else voted Tory. They did not find this pattern. They also have some experimental data about question order and priming, one suggested solution to the polling error. The first three waves of the BES, conducted back in 2014, randomised where it asked the voting intention question – at the start, or later in the survey. Asking it later in the survey only made a minimal, non-significant change to the Tory vote (while it doesn’t say so in the article, the full paper also makes clear it doesn’t change the Labour vote either!)

Thirdly the BES team looked a bit at sampling, specifically around age, looking at an issue Opinium have already commented on at the BPC inquiry meeting. All pollsters weight by age using various age bands such as 18-24, 25-40, etc, etc. But are people evenly distributed within those bands? Within the top age group, for example, the BES found that people over the age of 80 were underrepresented and people in their early 60s were overrepresented. Whether that makes any different to the results they can’t yet say, as it may be countered by other things like political weighting.

Finally, and most importantly, they wrote about turnout and suggested that people may have been overestimating their likelihood to vote, and that the people who were actually less likely to vote were increasingly skewed towards Labour. Currently this pretty circumstantial evidence – we don’t know if people lied about voting in the general election, but there’s evidence to suggest they might have. For example, a small proportion of people said they had already voted by post before most of the ballot papers had even been sent out, in areas where there were not any local elections this May there was still a chunk of people who reported having voted in their local elections. At the moment, these people who look as if they might be lying disproportionately break to Labour, so would explain some of the error. In their article Jon and Chris instead try modelling people’s likelihood to vote based on their demographics and characteristics of their seat, and that increases the Tory lead by 1.8%. They conclude that turnout, people saying they’ll vote when they won’t, is a major factor behind the error, thought they conclude that it’s one pollsters can probably address quite easily through a better turnout model.

Two caveats though, before you think things are solved. One – at the moment we’re going on indirect evidence of people overstating their likelihood to vote. In the fullness of time the BES are going to do a validation exercise of their data (that is, checking respondents names against the marked electoral register) so they will be able to conclusively prove whether or not there are a significant proportion of people who told pollsters they voted, but lied about it. Secondly, Jon and Chris estimate that getting turnout wrong probably explains about a quarter of the difference between the final polls and the election result, which would still leave us another three-quarters to explain…

Meanwhile there have been two new Scottish polls in the last week – both show the SNP still on course for another landslide at the Holyrood elections next year.

Survation for the Scottish Daily Mail have Holyrood voting intentions of
Constituency: CON 14%, LAB 20%, LDEM 7%, SNP 56%, Others 4%
Regional: CON 12%, LAB 19%, LDEM 8%, SNP 45%, GRN 11%, UKIP 5%
(Tabs here)

TNS have Holyrood voting intentions of:
Constituency: CON 14%, LAB 20%, LDEM 5%, SNP 60%, Others 2%
Regional: CON 13%, LAB 21%, LDEM 5%, SNP 51%, GRN 7%
(Tabs here)

301 Responses to “More BES findings on What Went Wrong”

1 2 3 4 5 7
  1. BARNEY: “maybe just maybe Tspiras is the man who will deliver.”

    Yes indeed. I’m disappointed not to see more comments motivated more by goodwill towards Greece rather than enmity towards Germany. No-one expects generosity or solidarity from the UK, sadly, but hopefully others will want to get behind Tsipras if he turns out to be the Greek Roosevelt. Having looked over the abyss, maybe he will lead his country on a different path.

    Colin: not sure where you get your 3% of German gdp to boost Greek gdp by 5% from. German gdp is approx $4trn; Greek $237bn. So 5% extra gdp is $12bn, or 0.3% of German gdp. In reality, well-chosen investment spending will create multipliers of up to 4 times.


    The Bloomberg link I posted showed what % GDP a write -off of their Greek loans would represent for EZ creditors. -around 3%.

    The 5% growth for Greece was your objective.

    I was posing the question , in response to your suggestion that they might feel some guilt, as to whether a once-of loss equivalent to 3% of their GDP is price they are prepared to pay to assuage that guilt.

    I further highlighted that it is the smaller countries , rather than Germany & France, who have most to lose proportionately-see the Bloomberg data.

  3. Colin:

    If Greece could achieve 5%pa gdp growth, there would be no need for debt write-offs. Thus by committing 0.3% of gap to Greek rebooting, Germany could avoid writing off 3% of gdp. Sounds like a no-brainer to me. But then I always like to look on the bright side.

  4. gap = gdp

  5. CLP nominations update – Burnham 68 Cooper 56 Corbyn 69 Kendal 11

  6. Sunday Times Scotland tweet

    poll tomorrow points to possible game changer on Scottish independence

    Probably referring to Scotland qualifying for the World Twenty20 cricket thingimy in India next year.

    “Scottish captain Preston Mommsen closed on 32” but I presume that was for the back nine.

  7. I predict 90 per cent of scots are now against independence.

  8. Nah. Sunday Times just has another Panelbase poll – nothing interesting.

    SNP 53
    CON 15
    LAB 22
    LIB 5
    UKIP 2
    GRN 2


    SNP 48
    CON 15
    LAB 21
    LIB 5
    UKIP 2
    GRN 6

    Panelbase polled both Scots and English on EU

    51% of English voters would vote to leave the EU
    66% of Scots would vote to remain in the EU

  9. @Somerjohn – “If Greece could achieve 5%pa gdp growth, there would be no need for debt write-offs. ”

    Bailout 3 already assumes annual 3.5% GDP over a time span of decades, described by the IMF as ‘heroic growth assumptions’, and even at this level the IMF says that without debt write offs the Greek debt ratio will rise to 200% before easing back to 170% by the end of the bailout period (2022). It is currently at 170%.

    As the IMF thinks it is impossible for Greece to achieve 3.5% growth in these circumstances, thinking they might hit 5% seems at the extreme end of optimism.

    The entire bailout scenario is a chimera. Before the Troika got to work, the Greek debt ratio was 120%, and despite the IMF constantly telling them that debt write offs are an essential partner of austerity and reform in a recovery situation, especially in one where devaluation is not possible, the EZ and ECB blithely ignored them, not once, but three times.

    A fourth bailout will be needed, then a fifth, and then a sixth, and at some subsequent point the German public might then realise that in fact, the bailouts aren’t working and a relatively modest debt write off would have been much cheaper in the first place.

    Lions led by accountants – and really bad ones at that.

  10. With much hilarity I read in the Telegraph that three Blairite MPs who backed Corbyn so he could get on the ballot paper are now urging party grandees to try to ensure he doesn’t win, complaining that his candidacy is damaging the party.

    I posted at the time that backing a candidate you don’t believe in are the kind of games that make voters wonder what planet MPs are on, and lo it has come to pass.

    Labour are shadows, in more ways then one, and there seems to be a terminal stupidity stalking parts of the party.

    Please, for the sake of democracy, organise your party elections in a more sensible manner.

  11. Ah. “Fieldwork 26th Jun to 3rd Jul”

    The Times seem to be behind the times again! Perhaps the EU polling is new?

  12. If that 51% of English voters want to leave the EU figure is correct that is quite a change from recent EU polls that have been about 60/40 for in… Greece and Calais may not have helped.

  13. Jack Sheldon

    Presumably the figure is correct (given the sources tweeting it) , but it is just from the 2nd half of the poll they started publishing last week. Fieldwork ended 7 July, so recent events will not have affected opinion.

  14. Sorry – ended 3 July.

  15. @Jack Sheldon

    I think a good chunk of people with a favourable view of the EU are centre/centre left.

    The way Greece has been treated, and the heavy handed German approach has driven a wedge through that.

    I am the sort of person who would never have imagined taking the same side as UKIP. I would normally see the place of the UK as one in a cooperative relationship with other states in our continent.

    However, I am struggling to support EU membership. If I can’t support it, then the UK surely will vote to leave.

  16. [England’s] “Labour’s lost voters may never return again, study finds ”

    “These voters are a hair’s breadth from becoming Conservatives,” says the report, which is being sent to all four Labour leadership candidates. “Labour is now at risk of becoming irrelevant even to voters who have been lifelong supporters.” It adds: “As things stand many of these voters are for the Tories to lose at the next election.”

  17. I’m not surprised at the 51% would leave the EU. Until the last couple of months I have always argued that need to stay in the EU. However the German attitude in recent months has caused me to seriously rethink my position. If the referendum was to be held tomorrow I don’t know how I would vote. Reading recent comments of the G…s website would suggest that there are many in the same position. I think the @Catmanjeff’s comment re pro-EU people being mainly left wing is probably true for those now questioning the EU.

    It will be interesting to see the next couple of polls on the EU referendum. I am fairly sure that the majority will be for leaving. Assuming that is correct, where will DC stand re the date of the referendum

  18. CMJ

    Successive Scottish polls on EU question –

    Panelbase (this one) FW 26 Jun – 3 Jul – Yes 66%
    Survation FW 3 – 7 Jul – Yes 67%

    Doubtless there will be another one along soon to see if the later stages of the Greek crisis has had any effect whatsoever on Scottish opinion.

  19. OldNat – “51% of English voters would vote to leave the EU
    66% of Scots would vote to remain in the EU”

    Well, if Nicola Sturgeon follows through on her promise to put a second referendum conditional on the EU result in her manifesto for the 2016 elections, and she wins heavily, there will be another Scottish referendum, because there will be a mandate for it.

    I wonder how the Scots will vote, given the oil price has collapsed and Obama is determined to put the screws on the Saudis by increasing supply (see his deal with Iran). And also currency questions – Greece proves the Scots will need their own currency and they will need to build their own clearing system too (because they won’t be able to access the one run by the BoE).

    I doubt Labour will bother to fight to keep the Scots this time – their dynamic has changed. They no longer have a stake in Scotland, and Scotland going independent means the Conservatives can’t use the “Vote Labour Get SNP line”. So it’s now in their interests if Scotland goes it’s own way. And it’s already in Conservative interests for Scotland to leave.

    Interesting times.

  20. P.S. Regarding the Saudis, according to the following article their budget deficit is now a staggering $150 billion or 20% of GDP.

    They do have a sovereign fund worth $672 billion – but with a deficit of that magnitude, it will be entirely gone in a few years.

    At which point they are going to have to cut back on their international war games significantly – presumably this is what Obama wants and why he’s taking the policy steps he is.

  21. OldNat

    Obviously the Survation poll took place after a lot of the Greek crisis occurred – indeed polling happened of the weekend of the referendum:

    It asked If there was a referendum tomorrow with the question “Should the United Kingdom remain a member of the European Union?”, how would you vote giving the result:

    Yes 51.2%

    No 25.8%

    Undecided 23.0%

    This compares with a previous Survation poll in Scotland carried out 22-27 April before the latest phase of the crisis. This asked If there was a referendum tomorrow on whether the UK should remain a member of the European Union, how do you think you would vote?

    Vote for the UK to remain a member of the European Union 52.5%

    Vote for the UK to leave the European Union 26.5%

    Undecided 21.0%

    Not exactly the biggest movement in polling history. Given that you might expect some Scots voters to sensitive to larger members of a political union throwing their weight around, not much seems to have registered yet.

  22. @Alec

    Seriously this is your example of politicians being far removed from the public? Stopping the PLP from screening out any Left candidates, no matter how broad their appeal is to their membership and electorate at larger?

    If you mean their subsequent horror at the thought that many will vote for such a candidate, then I suppose that does make them seem awfully out of touch to not even factor in such a possibility when lending their vote.

  23. @OldNat

    [England’s] “Labour’s lost voters may never return again, study finds ”

    Misleading, when it’s talking about the minority of Lab>Con switchers. Just SNP & Green’s increase from 2010 amounted to double the number to put the amount this group constitutes in context.

  24. Regarding ‘Labour’s Lost Voters’.

    I’ve read the letter about this research, and it does appear to be based on ten very small focus groups.

    I wonder how representative they are? If someone put a poll forward of 200 people, even with the best weighting (although we all know that has issues), we would and should be sceptical.

    Also, people in group discussions can act in ‘group think’ mode. A discussion on any subject will often gravitate to the views of a strong and vocal minority. Additional views that contradict this, if held by more quiet and shy people, may be withheld in self-censorship.

    Based on these two concerns, I would not give much weight to the research. Sure, Labour are in a hole, but listening to a small unrepresentative focus group is not solution.

  25. ALEC

    @”The entire bailout scenario is a chimera.”

    I expect to see very tough bargaining over Bailout3.

    It is the Northers’ last, and best, opportunity to ensure that Greece breaks open its trade monopolies & cartels, privileged professions & state owned industries -and installs a civil administration free of clientilism & corruption which can collect taxes & efficiently administer a modern open economy.

    If Tsipras convinces them that he will follow Renzi’s approach in Italy, then I can see Debt relief in the form of extended maturities.-Extend & Pretend, but not Haircuts.

    But how can a man with Tsipras’ political beliefs do this?

  26. @craig – you’ve missed the point. We want politicians and leaders who believe in something. Not people who vote for something they don’t believe in, in the hope that it will damage someone else’s prospects.

    @Colin – “The new three year 86bn euro Bailout will take some hard negotiating.”

    Point of order.

    The 86bn isn’t quite what it seems, in that 35bn of it is existing EU infrastructure budget already allocated to Greece on an 85% funded ratio (eg Greece needs to provide 15% of the funding to draw down the structural funds).

    This part of it cannot therefore be described as a ‘bailout’ but instead is money due to Greece under normal EU operating arrangements. That the Germans insist on adding this to the bailout figure demonstrates how poorly the German voters understand these issues.

  27. @Colin – “It is the Northers’ last, and best, opportunity to ensure that Greece breaks open its trade monopolies & cartels, privileged professions & state owned industries -and installs a civil administration free of clientilism & corruption which can collect taxes & efficiently administer a modern open economy.”

    It’s funny how the German’s want to hang on to much of their regulation and prevent open access to their internal markets, and how about the French for their anti comeptitive state owned industries? It’s fine for them to buy up UK water and power companies, but can we buy theirs?

    Then there’s the small issue of the Commission president. Do we want to talk about Luxembourgs ability to (not) collect business taxes with it’s record of ‘clientilism & corruption?’.

    I just love the two faced nature of the way Greece is being attacked.

  28. Roger Mexico

    Thanks for linking the polling dates to the referendum date – meant to do that myself, but ……

    Altghough there was much angst on my Twittter feed from political geeks like us, political issues seldom seem to have much effect during the main summer holiday season (end June- mid Aug here) – and the Grexit crisis seems to have followed the pattern.

    Perhaps it had more effect in England & Wales, where school summer holidays were still a much longed for prospect?

  29. ALEC

    Call it what you like -Bailout3 is what Greece has asked for.

    Re IMF, perhaps I can-for the sake of balance-reference an interesting piece by Oliver Blanchard addressing the key criticisms of the shameful exploitation by Greece’s creditors.

    You might find it an interesting slant on things :-

    I took particular note of :-

    “Had Greece been left on its own, it would have been simply unable to borrow. ”

    “Fiscal austerity was not a choice, but a necessity. There simply wasn’t an alternative to cutting spending and raising taxes. ”

    “The 2012 private sector involvement (PSI) operation led to a haircut of more than 50% on about €200 billion of privately held debt, so leading to a decrease in debt of over €100 billion (to be concrete, a reduction of debt of 10,000 euros per Greek citizen).”

    “, interest payments by Greece were lower, as a proportion of GDP, than interest payments by Portugal, Ireland, or Italy.”

    Luxembourg & France aren’t asking for a Bailout. Greece is.

    There are EZ parliaments who will have to vote on Bailkout3-including Merkel’s. They will be deciding whether this looks like good money after bad.

    That’s democracy .

  30. CMJ


    Ah, focus groups.

    These people were apparently pre-selected as former Labour voters who did not vote Labour this time. Quite obviously, not a representative sample of anything.

    All this can tell us is an indication as to why a very small section of the electorate voted as they did.

    I wonder what would happen if they interviewed former Tory voters who voted Labour…

  31. CMJ

    What I find especially annoying about these ‘reports’ is that they invariably selectively quote one or two throwaway, unascribed remarks from the focus group, which always support the thesis of the author(s), and which mean precisely nothing.

    Don’t get me started…

  32. @Millie

    Persuading people to pay you to run a focus group is some trick.

    I think only those far to keen to liked (ie really insecure) would pay for it.

  33. Alec

    “Please, for the sake of democracy, organise your party elections in a more sensible manner.”

    There’s nothing wrong with the organisation (exceptexcept perhaps the high threshold for nominations). There are just a few idiots trying to play games. Sadly there’s no way way to get rid of them, other than deselection.

  34. Bugger, forgot to close the bold tag

  35. When people say they are swinging towards an exit vote on the EU could they say what they mean by leaving. There are 57 varieties of not being in the EU. This is what will have to be made clear in the run up to any vote.

  36. ‘Labour’s Lost Voters’

    It amazes me how people can’t see that ignoring things like grooming gangs* preying on your voter’s children might eventually have an electoral impact even on the most blind loyal voters on the planet.

    (*not just that of course – the grooming gangs are just one of the most extreme examples of what Lab did to their 1997 voters.)

  37. @Alec

    “…The 86bn isn’t quite what it seems, in that 35bn of it is existing EU infrastructure budget already allocated to Greece on an 85% funded ratio (eg Greece needs to provide 15% of the funding to draw down the structural funds)…”

    Turn it around. Why is Greece (a country that has accrued over 300 billion euros of debt it refuses to pay back) given any money at all? You seem to be saying that the EU gives Greece 35billion as a gift. If somebody gave me 35billion as a gift I would say “thank you” and pay back some of my debts.

  38. @Alec

    “…(eg Greece needs to provide 15% of the funding to draw down the structural funds)…”

    Good grief man! Greece actually spend its own money? How very dare you? Don’t you know it must only spend other people’s money? That’s silly talk! …:-)

  39. Mrjones,

    I knocked on what must have been the low four figures of doors across maybe half a dozen constituencies in the last parliament. I found both supporter and opponents of Labour, but not one of the opponents ever raised that as a cause.

    Labour held all the Rotherham constituencies comfortably. While it does make some sense that it might lose them support, and certainly anti-Labour people have taken it up as a stick to beat them with, it does not seem to have that much real world effect.

  40. Not posted on here for ages as managed to lose my addiction for UKPR after the GE- rightly or wrongly this was partly down to wanting a break from politics after your “team” loses but more so because it seems pointless basing theories on opinion polls when they were wrong.

    But just wanted to re-iterate what chrisjam01 said about the LD wins in last week’s locals. Their candidate in Kingston was the manager (or similar) of the local indie record store, also involved with the local footie team (although with crowds of 300 these days there might not have been a massive personal vote from one ward) and the ward has a high percentage of students.

    So as Anthony has always warned us the by elections are not a good guide because of personal circumstances. It will however be one of the interesting themes of this parliament to see whether the Lib Dems do make a recovery or are largely replaced by the Greens (and UKIP).

  41. @Mr Nameless
    “certainly anti-Labour people have taken it up as a stick to beat them with”

    At least here it appears. I am quite surprised that such repeatedly obsessive and insulting posts are allowed to remain here.

  42. @Alec

    Corbyn is normal for the Labour Party membership.

  43. Wolf,

    Rubbish. As you will see when he doesn’t win the election.

  44. @WOLF “…Corbyn is normal for the Labour Party membership…”
    @MRNAMELESS “…Rubbish. As you will see when he doesn’t win the election…”

    But losing elections is normal for the Labour Party membership…:-)



    OK, I’ll get me coat…

  45. Mr Nameless

    “it does not seem to have that much real world effect”


    “but not one of the opponents ever raised that as a cause”

    all those years where talking about it ran the risk of losing your job, livelihood, home or arrest may have something to do with that

  46. @Candy
    ‘Well, if Nicola Sturgeon follows through on her promise to put a second referendum conditional on the EU result in her manifesto for the 2016 elections, and she wins heavily, there will be another Scottish referendum, because there will be a mandate for it.’

    I don’t accept that the result of an election for Holyrood on a turnout of circa 50% should be allowed to override the clear decision of a Referendum in which 85% voted. Salmond told us it was a once in a generation decision. Cameron – and his successors – will be perfectly entitled to ignore Scottish elections to Holyrood until the 2030s. I hope they do so.

  47. Talking about it to a Labour canvasser would not have any of those results. We aren’t the Spanish Inquisition.

  48. @Colin – “Call it what you like -Bailout3 is what Greece has asked for.”

    That’s strectching it, after a comprehensive No vote. I think what you mean is that Greece has been forced to accept the terms.

    The IMF quotes are fine, so long as you set them alongside the IMF’s other statements, where they clearly state that the bailout will not work and Greek debts will rise.

    @Martyn – normally you are a man for details and sound sense, but on the Greek issue you seem to have left your coat at home.

    The 35bn is money already agreed to be allocated to Greece, for a specific purpose (growth inducing infrastructure investments). It cannot be spent on anything else. It is the price the rest of pay for being in the EU and belongs to Greece, under the Lisbon Treaty terms.

    You say – “Why is Greece (a country that has accrued over 300 billion euros of debt it refuses to pay back) given any money at all?”

    Again – this does not make sense. Greece hasn’t refused to repay anything. It has been unable to make some repayments and has asked for terms from it’s creditors to help it pay off as much of the debt as can reasoanbly be achieved, but they refused, even with the IMF telling that to do so would mean they lose more money.

    The point to bear in mind here is that no one has ‘given’ Greece anything. Greece has been lent money by many organisations, at commercial rates. Historically, these transactions attract an interest premium that reflects the perception of risk. In Greece’s case, those risks were high, so therefore the interst rates were high.

    Priced into these rates was the risk that there would be a default. this is what interest rates reflect. The rates have meant lenders have earned significant returns on these loans that they voluntarily gave. However, as in all things market based, rather than the lenders understanding that they have already been paid their risk premium (and in many cases this has actually earned them more than the capital sum) they also want their capital returned. When markets fail, the poorest must pay, seems to be the principle.

    I would have no objection to a refusal to write off debts if the loans were at cost, with no risk premium attached and no previous interest payment recieved as protection against default, but this isn’t the case here.

    You also need to bone up a little on how markets work in currency unions. The Northern Europeans have recieved huge transfers of wealth via normal trading activities within a defective currency area. The southern states have been held back due to the illogical valuations at the time of entry and the lack of transfer capital. The German’s don’t want a transfer union. Fine – leave the Eurozone, and then see how your exporters fare.

    The situation is far more nuanced than you appear to appreciate, which for you is unusual. You may have a Grecian blind spot (not the same as a Grecian bald spot – altogether different).

  49. @Alec

    Perhaps those MPs believe in democracy, and not for a narrow bunch to effectively screen out a particular type, disenfranchising the Left in the process. I’m not sure why you’re attributing it to a desire to hurt someone else’s prospects?

  50. Barney & Colin: It’s good to see a more thoughtful approach to the way forward for Greece than just, “Germany has been beastly to poor, plucky Greece. The bailout cannot work and the EZ will fail. I used to support UK membership but I’ve changed my mind.” And that’s a summary of many of the views on here, not a parody.

    We’re told Greek gdp has declined by 25% since 2010. I think aiming to recover that lost gdp in the next 5 years is a reasonable ambition. And I think any competent Keynesian economist could design a ‘new deal’ to achieve that, built on the foundations of a reformed economy and competent leadership (plus forensic oversight by tough-but-fair friends of Greece).

1 2 3 4 5 7