Jon Mellon and Chris Prosser from the British Election Study team have written a new post and paper on the emerging evidence from the BES data on what went wrong with the polls. Last month they wrote a piece, which I covered here, on some of the potential causes of error they could use the BES data to look at. Now the BES post-election data is out they’ve done so, and come back with some findings.

Firstly, late swing – the BES data finds virtually no net change at all between how people said they would vote pre-election and how they reported having voted after the election. The BES team conclude from this that late swing is unlikely. We’ve now got published re-contact data from the British Election Study, ICM, Opinium, Populus and Survation, only Survation found any obvious evidence of late swing in their re-contact survey.

Secondly, Shy Tories. This is essentially the most difficult potential cause to evidence – if people lie before the election, and lie after the election and we can’t check their actual ballot papers, how do you detect it? You need to look for circumstantial evidence. The BES team have compared levels of Tory support in their polling in different types of area, on the assumption that if people feeling embarrassed to admit voting Tory really was a problem it would be less of an issue in heavily Tory areas than in areas where no one else voted Tory. They did not find this pattern. They also have some experimental data about question order and priming, one suggested solution to the polling error. The first three waves of the BES, conducted back in 2014, randomised where it asked the voting intention question – at the start, or later in the survey. Asking it later in the survey only made a minimal, non-significant change to the Tory vote (while it doesn’t say so in the article, the full paper also makes clear it doesn’t change the Labour vote either!)

Thirdly the BES team looked a bit at sampling, specifically around age, looking at an issue Opinium have already commented on at the BPC inquiry meeting. All pollsters weight by age using various age bands such as 18-24, 25-40, etc, etc. But are people evenly distributed within those bands? Within the top age group, for example, the BES found that people over the age of 80 were underrepresented and people in their early 60s were overrepresented. Whether that makes any different to the results they can’t yet say, as it may be countered by other things like political weighting.

Finally, and most importantly, they wrote about turnout and suggested that people may have been overestimating their likelihood to vote, and that the people who were actually less likely to vote were increasingly skewed towards Labour. Currently this pretty circumstantial evidence – we don’t know if people lied about voting in the general election, but there’s evidence to suggest they might have. For example, a small proportion of people said they had already voted by post before most of the ballot papers had even been sent out, in areas where there were not any local elections this May there was still a chunk of people who reported having voted in their local elections. At the moment, these people who look as if they might be lying disproportionately break to Labour, so would explain some of the error. In their article Jon and Chris instead try modelling people’s likelihood to vote based on their demographics and characteristics of their seat, and that increases the Tory lead by 1.8%. They conclude that turnout, people saying they’ll vote when they won’t, is a major factor behind the error, thought they conclude that it’s one pollsters can probably address quite easily through a better turnout model.

Two caveats though, before you think things are solved. One – at the moment we’re going on indirect evidence of people overstating their likelihood to vote. In the fullness of time the BES are going to do a validation exercise of their data (that is, checking respondents names against the marked electoral register) so they will be able to conclusively prove whether or not there are a significant proportion of people who told pollsters they voted, but lied about it. Secondly, Jon and Chris estimate that getting turnout wrong probably explains about a quarter of the difference between the final polls and the election result, which would still leave us another three-quarters to explain…

Meanwhile there have been two new Scottish polls in the last week – both show the SNP still on course for another landslide at the Holyrood elections next year.

Survation for the Scottish Daily Mail have Holyrood voting intentions of
Constituency: CON 14%, LAB 20%, LDEM 7%, SNP 56%, Others 4%
Regional: CON 12%, LAB 19%, LDEM 8%, SNP 45%, GRN 11%, UKIP 5%
(Tabs here)

TNS have Holyrood voting intentions of:
Constituency: CON 14%, LAB 20%, LDEM 5%, SNP 60%, Others 2%
Regional: CON 13%, LAB 21%, LDEM 5%, SNP 51%, GRN 7%
(Tabs here)

301 Responses to “More BES findings on What Went Wrong”

1 4 5 6 7
  1. Alec

    I think the withering of the state is some way away. Having welfare is unavoidable in a Capitalist system.

  2. @Carfew

    Scotland will have it’s own currency plan at next referendum. The sharinga currency with rUK ship has sailed.

    My guess is the referendum will be 2019.

    Win Majority 2016 with conditional referendum in manifesto

    Decide on 3 year parliament (Holyrood decides whether 3 or 5 years so not to clash with 2020 GE). If Holyrood decides 3 years it is a clear sign of a 2019 referendum.

    Referendum in September 2019 in 2019 SNP manifesto



  3. Plus business needs the State to channel contracts its way and cover its screw ups etc.

  4. @Couper

    “Referendum in September 2019 in 2019 SNP manifesto


    OK, but I’m keeping the Joe Henderson CDs…

  5. @Jim Jam
    ‘Tory-Lite may be an exaggeration for the Blair/Brown years but it was the case that the LP leadership accepted the ‘consensus’ about the way the UK Economy should be run; and imo they would not have got elected without it.’

    Don’t agree with that – Michael Foot would have won in 1997. I don’t accept the line that if an election programme is rejected at one election that the party concerned must change its policies to have any chance of winning in the future. Circumstances do change including the opinion taken by the electorate on particular policies. I feel pretty sure that the failed 1992 Labour manifesto would have been accepted by the electorate in 1997. With hindsight there was no need to change because by 1997 the electorate was much more firmly in an anti-Tory mood and Labour would have won convincingly on its 1992 programme

    @ Couper
    Cameron should come out and state that there will be no Referendum on independence in the course of this Parliament . In no way should the result of an election for Holyrood on a 50% turnout be allowed to override the ‘once in a generation’ decision of the Scottish people taken clearly on a 85% turnout in Sett 2014. I hope that Labour would support such a statement too.

  6. @Hawthorn

    In the 1960’s few families had more than three children at the most. The UK wasn’t thought of as underpopulated even then.

  7. Wolf

    If it is about controlling population, then politicians should say so. I haven’t seen anyone arguing that this is about preventing poor people from “breeding”.

  8. One of the inescapable facts of politics, and probably life in general too, is that the victors tend to write the history whilst the vanquished sink into angst, self-loathing and temporary disarray. This entirely inevitable process can become all the more stark if the victory is as unexpected as the defeat was unanticipated. In recent political history, think 1997, 2001 and now 2015. Call it Apocalypse Again, if you like, with doom-mongers predicting seismic, irreversible shifts, the end of history and the victory of a certain idea as opposed to the extinguishing of another. Quite often, the vanquished will exaggerate the scale of their defeat and the victors the size and significance of their triumph and the victors, because they’re writing the history and are in control of the narrative, will have much fun making mischief for the vanquished and enjoying the seductive frisson of schadenfreude as they see their opponents suffer.

    Of course, the history written by the victors in the immediate aftermath of their triumph quite often, in time, proves to be bunk, and the vanquished, who may well have initially accepted the victors version of events, come to realise that they have unnecessarily added to their own woes by doing so. They listened, unwisely, to people who wished them nothing but ill and fell for a flawed diagnosis, hypnotised by some commentariat groupthink too.

    Labour has some thinking to do, as all defeated political parties have after electoral rejection, but this idea that the most unrepresentative election in British electoral history has signalled some sort sea change in the nation’s politics is, frankly risible. Osborne and Cameron would like us all to think that but it’s nonsense, as is all this talk about people “being out of power for a generation.”

    Things tend to unravel very quickly in politics, especially when hubris makes politicians think they have mandates that they don’t. Ironically in politics, it’s quite often the victors rather than the vanquished who misread the verdict of electorate.

    Much more silliness to come however, I fear.

  9. @Candy

    She is unelected as Leader, as opposed to Deputy Leader.

    That means that, however much she (mistakenly) believes that Labour ought now to tack hard to the right, she has absolutely no mandate to try and use her temporary stewardship to impose such a course on a parliamentary party that so clearly disagrees with her. Had she listened to and been guided by what her MPs were overwhelmingly telling her, this wouldn’t have happened so the mess is entirely of her own making.

  10. A great discussion and hope it is still going on when I come back later tonight.

    A useful reminder from some posters that parties do change their positions, and obviously fortunes can change. Perhaps polls (remember them?) will give us a clue before the various elections next spring.

    In the meantime, my take on this is that sooner or later the government will become unpopular, and at that point people will start listening to see if any of the opposition parties have something to say.

    Whether voters will turn to the left-leaning LDs, or to a social democratic Labour party – who knows? Or maybe UKIP or the Greens will tack to the centre if no-one else is there to own it? Stranger things have happened.

  11. CB11

    “One of the inescapable facts of politics, and probably life in general too, is that the victors tend to write the history whilst the vanquished sink into angst, self-loathing and temporary disarray.”

    The trouble about “inescapable facts” is that they aren’t facts and the circumstances are escapable.

    Witness politics in Scotland post-referendum.

  12. @Alec

    Re your comments on tax credits. I find your line of argument utterly bizarre, and I couldn’t disagree with you more.

  13. @Phil Haines
    ‘She is unelected as Leader, as opposed to Deputy Leader’

    Agree with that. Harman would have had a good deal more authority if she was now running to be confirmed as Leader herself.

  14. @Phil Haines – I’m genuinely sorry about that, and worried too. I respect your posts, and it worries me that you think me bizarre. Perhaps I have got this wrong.

    I thought tax credits were great when they were expanded in 2003. An ideal way to help fix a major problem that had allowed to develop.

    However, all policies have limits, in both scale and time. I don’t believe they should be seen as anything more than transitional arrangements, and a better way to improve living standards needs to be found. This doesn’t mean that I would abolish all of them by any means.

    I just think Labour is being blindsided on an issue that needs reform and careful thought, and by being so it is possibly harming the less well off far more in the long run.

  15. “Whether voters will turn to the left-leaning LDs…”


    Isn’t it a little soon? And doesn’t there have to be like an asteroid hurtling towards earth or summat….

  16. “assumes markets will adjust and wages increase to fill the gap. They won’t in the short term…”


    There’s nothing given about it in the long term either. This is why they invented unions and stuff.

    And, even if wages do go up in the longer term, little use if prices on Essentials go up to match…

  17. “Perhaps polls (remember them?) will give us a clue before the various elections next spring.”


    Well, it is entirely possible they may do exactly that. Trouble is, we won’t know till afterwards if they were telling porkies or not…

  18. Carfrew

    Are there many elections next spring in the porky telling areas of England?

  19. @oldnat

    Bit busy at the mo’, but you could always try asking someone else. (Perhaps even someone who votes…)

  20. Carfrew

    No need even to be able to vote! English elections for porky telling areas will be held for some local government, some Mayoral elections and, most exciting of all, Police and Crime Commissioners.

  21. Good evening all from a slightly mild East Ren.

    “Presumably they have all had maxillofacial surgery to correct the trauma to their lips”

    Can you get that free on the NHS? ;-)

  22. I know we have covered EVEL in great depth over the past months but I read an article today saying the SNP will probably hold the balance when it comes to voting on a 3rd runway for Heathrow.

    Surely it can’t be designated as a purely English matter because of the economic impact (positive) a 3rd runway would have for Scotland?

    Whoever said the SNP will have limited power at Westminster on the back of a Tory majority should take a reality check. If I were the residents who’s homes will be demolished in the event of a 3rd runway then I would be lobbying Queen Sturgeon and not the Tories.

  23. Does @AW still exist??

  24. Good Evening All, from a windy Bournemouth Beach.
    Good Times for the Blues.
    Divided Times for different shades of Red.
    The Lib Dems are looking healthier, IMO.

    Summer of silliness about to start for teachers, especially someone who has hit sixty.

  25. @Adrian B

    “Does @AW still exist??”
    He comes into view once every 76 years.

  26. Given the bookies had a good election, two markets to look at:

    Next Lab leader

    Andy Burnham 11/10 (drifting)
    Yvette Cooper 5/2
    Jeremy Corbyn 5/1
    Liz Kendall 16/1 (drifting).

    Next Tory leader

    George Osborne 3/1
    Boris Johnson 3/1
    Theresa May 7/1
    Sajid Javid 9/1

    Next PM

    Boris Johnson 3/1
    George Osborne 7/2
    Andy Burnham 9/1
    Theresa May 10/1
    Sajid Javid 14/1
    Yvette Cooper 14/1
    Jeremy Corbyn 25/1

  27. Came across the following interesting article about the nominations for the Labour candidates:

    Looking at whether the CLP was a safe Labour seat, a marginal or unwinnable, it broke down as follows:

    Burnham: 48% core, 33% marginal, 20% unwinnable
    Cooper: 29% core, 33% marginal, 38% unwinnable
    Corbyn: 39% core, 34% marginal, 27% unwinnable
    Kendall: 58% core, 8% marginal, 33% unwinnable

    So Kendal and Burnham are getting nominations from places that already have a Labour MP. Corbyn and Cooper are getting nominations mainly from places without a Labour MP.

    Make of that what you will…

  28. @RAF

    “Does @AW still exist??”
    He comes into view once every 76 years.”


    He modded me a few days ago. For asking Colin about Oyster cards.

    I got modded a while back for a comment involving popcorn. One senses a dietary connection… But peeps are being v. civilised since the election and maybe he woz bored…

  29. @chrislane

    “Summer of silliness about to start for teachers, especially someone who has hit sixty.”


    If you hang with peeps who run schools, they might tell you it has started already…

  30. Candy

    Members, and supporters, in every constituency each have one vote. A more useful analysis perhaps would have been by membership figures.

  31. @OLDNAT

    “No need even to be able to vote! English elections for porky telling areas will be held for some local government, some Mayoral elections and, most exciting of all, Police and Crime Commissioners.”


    Yes, I don’t know how peeps contain their excitement…

  32. Colin

    “……….and repeat those shameful victories of ’97 , ’01 & ’05 presumably?”

    I note with interest that your history of New Labour’s successes ends prior to 2010.

  33. Tweet

    YouGov #Labourleadership first preferences has Jeremy Corbyn ahead:
    Corbyn 43%
    Burnham 26%
    Cooper 20%
    Kendall 11%

  34. Hilarious new YG out on LAB leadership:

    Corbyn 43%
    Burnham 26%
    Cooper 20%
    Kendall 11%

    Corbyn would win 53/47 in the final round.

    Should come with a massive health warning, of course. Nigh on impossible to accurately sample an electorate that we aren’t certain of the shape of.

  35. Mike Smithson [email protected] 16m16 minutes ago

    The 1st published LAB leadership poll – YouGov for Times has Corbyn winning
    1st prefs
    Corbyn – 43%
    Burnham – 26%
    Cooper – 20%
    Kendall – 11%


    Jeremy is very close to winning there.

  36. Sam Coates Times
    YouGov: Deputy leadership election. 1st prefs
    42% Tom Watson
    21% Stella Creasy
    17% Caroline Flint
    11% Ben Bradshaw
    10% Angela Eagle

  37. ON – I read Candy was wondering if we could draw any conclusions about how the split may suggest the candidates will appeal beyond the base?
    Not as an attempt to forecast the result
    Begs the secondary question, which group of activist have the best insight in too who might appeal most to the voters needed to win key seats.

    If you think that those in marginals are best placed then it is anyone but Kendall. OTOH perhaps members in seats we already hold know what works best?

    Personally – not sure it is worth reading too much in to them

  38. Now, if Jeremy did actually win I suppose the questions are:

    a/ Would he actually take it on? If so, would he commit to giving way before 2020? (perhaps to one of the new intake of left-wingers once they’ve got some experience).
    b/ How many of the current establishment would serve under him? Or would the front-bench be made up of his awkward squad colleagues and the 2015 intake?
    c/ Would any centrists dare to try and breakaway? Or would they go for trying to force another leadership election?
    d/ What would the public think? (not necessarily hugely negative in the immediate term though hard to tell).
    e/ Would he go on espousing positions that would be very difficult indeed to pursue as Leader of the Oppo or moderate his policy?

  39. One finds onself more familiar with the contenders for the Deputy leadership than with some of those trying to be the actual top banana…

  40. Jim Jam

    I wasn’t suggesting that Candy was trying to predict the result – or had done the analysis either.

    Simply an observation that whoever did the analysis hadn’t selected a particularly appropriate methodology.

  41. I should also add that even if Corbyn doesn’t win huge damage will be done if he is second… the winner will have to concede to the left or face big internal problems.

  42. One finds onself more famil1ar with the contenders for the Deputy leadership than with some of those trying to be the actual top banana…

  43. @Jack Sheldon

    I don’t think it’s hilarious, but interesting.

    Given how senior Labour people have acted with leaden feet and also tone deaf to the grass roots members (for some time) I’m not surprised.

    Last night’s vote was revealing, in that Labour mostly seems all over the place, and scared to make a principled stand for something that most party members would want them too.

    It seems that Burnham, Cooper and Kendall are ultra self conscious about how anything they do might be interpreted, while Corbyn has spoke well and clearly to his party, and not afraid to say what he believes in.

    I’m not saying Jeremy will win, or if he becomes leader do well at the GE (I have no idea), but he is talking the language his party understands. Other seems to playing a detached, Westminster and media game. Just perhaps ordinary Labour members are tired of that.

  44. “ON – I read Candy was wondering if we could draw any conclusions about how the split may suggest the candidates will appeal beyond the base?
    Not as an attempt to forecast the result”


    I thought it might be an attempt to gauge how many voting might not be your usual Labour supporter as it were. Shows what I know…

  45. NC Politics makes a good point

    “So virtually all of this polling would have been done before the #WelfareBill vote… “

  46. @Catmanjeff

    You are absolutely right, especially on what you attribute his popularity to.

    However, it is pretty clear to me that electing Corbyn as leader would in effect end the Labour parliamentary party as we know it. It would not just be a change of leader but would inevitably become a transformational event for the party in a way that even the elections of Foot and Blair were not.


    “Now, if Jeremy did actually win I suppose the questions are:”


    It’s possible there are some other salient questions, including…

    1) Does he have a brother in politics?

    2) Is there anything on his Dad the Daily Mail might take an interest in

    3) What’s he like eating bacon

    4) does he trip on steps?

    5) what’s he like with speeches, does he leave bits out?

    6) does he speak English, or the strange, forgotten language of predistribution that survives mainly in the environs of Islington, Notting Hill, Primrose Hill etc.

  48. @Jack Sheldon

    I think whoever wins might well end the party as it is now, not just Jeremy. Liz would split the party badly, and Yvette and Andy might paper over the crack only briefly.

    I think politics has fragmented, and Labour’s old broad church seems no longer viable perhaps, without radical reinvention.

  49. @Catmanjeff

    Again you may not be wrong. I think Burnham would allow them to get back into the type of comfort opposition that they seemed to quite enjoy under Miliband. As you say that may just postpone the showdown until some later point.

  50. Carfrew – ” doesn’t there have to be like an asteroid hurtling towards earth or summat….” [before LDs pick up support]

    Maybe some pictures from Pluto will do the trick?

1 4 5 6 7