I’ve just got back from the BBC after working all night (you may have seen my bald spot sat just to the left of Emily Maitlis’s big touchscreen last night) and am about to go and put my feet up and have a rest – I’ll leave other thoughts on the election until later in the weekend or next week, but a few quick thoughts about the accuracy of the polls.

Clearly, they weren’t very accurate. As I write there is still one result to come, but so far the GB figures (as opposed to the UK figures!) are CON 38%, LAB 31%, LDEM 8%, UKIP 13%, GRN 4%. Ten of the final eleven polls had the Conservatives and Labour within one point of each other, so essentially everyone underestimated the Conservative lead by a significant degree. More importantly in terms of perceptions of polling it told the wrong story – when I was writing my preview of the election I wrote about how an error in the Scottish polling wouldn’t be seen so negatively because there’s not much difference between “huge landslide” and “massive landslide”. This was the opposite – there is a whole world of difference between polls showing a hung Parliament on a knife edge and polls showing a Tory majority.

Anyway, what happens now is that we go away and try and work out what went wrong. The BPC have already announced an independent inquiry to try and identify the causes of error, but I expect individual companies will be digging through their own data and trying to work out what went wrong too. For any polling company, there inevitably comes a time when you get something wrong – the political make up, voting drivers and cleavages of society change, how people relate to surveys change. Methods that work at one election don’t necessarily work forever, and sooner or later you get something wrong. I’ve always thought the mark of a really good pollster is someone who puts their hands up to the error, says they’ve messed up and then goes and puts it right.

In terms of what went wrong this week, we obviously don’t know yet, certainly I wouldn’t want to rush to any hasty decisions before properly looking at all the data. There are some things I think we can probably flag up to start with though:

The first is that there is something genuinely wrong here. For several months before the election the polls were consistently showing Labour and Conservative roughly neck-and-neck. Individual polls exist that showed larger Conservative or Labour leads and some companies tended to show a small Labour lead or small Conservative lead, but no company consistently showed anything even approaching a seven point Conservative lead. The difference between the polls and the result was not just random sample error, something was wrong.

I don’t think it was a late swing either. YouGov did a re-contact survey on the day and found no significant evidence of this. I think Populus and Ashcroft did some on the say stuff too (though I don’t know if it was a call-back survey), so as the inquiry progresses other evidence may come to light, but I’d be surprised if any survey found enough people changing their minds between Wednesday and Thursday to create a seven point lead.

Mode effects don’t seem to be the cause of the error either, as the final polls conducted online and the final polls conducted by telephone produced virtually identical figures in terms of the Labour/Conservative lead (though as I said on Wednesday, they were different on UKIP). In fact, having a similar error with both telephone and online polls is evidence against some other possibilities too – unless by freakish co-incidence unrelated problems with online and telephone polling produced almost identical errors it means things that only affect one modeare unlikely to have been the cause. For example, if the problem was caused by more people using mobile phones, it shouldn’t have affected online polls. If the problem was caused by panel effect, it shouldn’t have affected phone polls.

Beyond that there are some obvious areas to look at. Given that the pre-election polls were wrong but the exit polls were right, how pollsters measure likelihood is definitely worth looking at (exit polls obviously don’t have to worry about likelihood to vote – they only interview people physically leaving a polling station). I think differential response rates is something worth examining (“shy voters”… though I think enthusiastic voters is just as risky!), and the make-up of samples is obviously a major factor in the accuracy of any poll.

And of course, it might be something completely unrelated to these things that hasn’t crossed our minds yet. Time will tell, but first some sleep.


710 Responses to “Back from the election”

1 11 12 13 14 15
  1. It depends how much weight you give to the AMS. It worked well in Hungary between 1990-2010, but I have a preference for Neil A’s suggestion.

  2. @andrew111

    “I wonder what % increase 1000 new members on Friday was?”

    No idea but that number is getting bigger. Quoting Tim Farron from the BBC live feed:

    “He says the party will build itself from the “ground up,” claiming that it had gained more 2,500 new members in the past few days.”

    @assiduosity

    Thanks I’ll have a read.

  3. @DavidInOxford

    “It is pretty clear to any independent observer….”

    I’m afraid I don;t think the picture is nearly as clear or as universal as the one that you paint.

    The UKIP vote was far from uniform either in its scale or in terms of where it appears to have taken its support from.

    I do definitely agree though that Nigel Farage has been honest throughout in his appraisal that UKIP would damage Labour as much as the Conservatives and the results in many – though not all (London for example) – bear this out.

  4. @ David

    Spot on about the possible influence of the level-pegging polls on Labour’s strategy.

    @ Neil A

    Yes, like it or not, Labour is unlikely to win a majority again without the Lib Dems recovering in the South-West and elsewhere. I don’t expect any large moves on this by the Labour leadership contenders but at least some indication of a willingness to talk to other voices on the centre-left would be very welcome. Would be easier should Farron win the leadership I imagine.

  5. UKIP has certainly damaged the Labour vote much more than the Conservative vote in Clacton. There is no doubt about it.

  6. @ BM11

    Luciana Berger is considered a bit too divisive around here, and was the MP here who could barely (0.7%) increase her voting share. But as Labour has rather few MPs …

  7. @ David in Oxford

    The Blue UKIPpers returning home in Lab/Cons marginals was the reason I went for 298 Con seats in my prediction. I was minded to go even higher but could not believe all of those opinion polls could be so wrong!!

  8. @Laszlo

    I agree that multi-member constituencies elected on a list system (as per the European Parliamentary Elections) has its benefits as an approach.

    Though I think it may be a more difficult ‘sell’ to the electorate in some ways.

    Both would, of course, make most of the forecasting companies, redundant.

  9. Apologies for the second part of the sentence about Luciana Berger. Completely wrong.

  10. Stella Creasey is great, too, but she’s a bit posh, and a London MP. Definitely should be front bench, and it would be good to see the first female leader of Labour, but I don’t think she’s probably the right person to take over the reins at the moment.

  11. @NewForestRadical

    “Labour is unlikely to win a majority again without the Lib Dems recovering in the South-West”

    An interesting point. As I said earlier, I don’t think such a recovery is at all inconceivable. Many of these newly Conservative seats will be key LibDem targets in 2020, and I think that ‘incumbency’ is something of an over-rated commodity.

  12. Killary45

    That’s true but he still had plenty of people around him from the Kinnock era (Frank Dobson, Robin Cook, Jack Straw, John Prescott etc)

    Like I said though, generally I agree that it needs to be a new team with new little regard to Labours previous government.

    In that sense, Ed Balls losing his seat is possibly a blessing. I don’t think Balls as Shadow Chancellor was an asset to Miliband as he was too unpopular in the country and tainted by the crash.

    That’s despite the fact that he was probably the most qualified for the job In the Lab Party and undoubtedly more qualified than Osborne. Perceptions are everything.

  13. Balls should be ennobled and given a formal role in Labour’s economic team, but out of the limelight.

  14. I am not at all convinced that Andy Burnham is the right choice. Too tainted, not enough of a life outside politics and nothing really distinctive (eg ideas, charisma etc) to recommend him. A good, competent shadow minister, but a leader? No!

    In the meantime it is clear that the Blairites are on the march with a predictable back to the future argument. How depressingly simplistic! As if Scotland and UKIP had happened!!!

  15. Just got back from Brandenburg (Potsdam and Spreewald, delightful).

    Anything special happened while I’ve been away?

    Is there a YouGov poll expected for the Sunday Times tonight?

  16. I would like to state my confusion, as every where I’ve read that as you leave, someone physically asks who you voted for? But when I left the voting centre, the representative outside only asked for my polling number, and that was all, I found it a little confusing but though there was a reason for it, however, since looking around for explanations, I’m even more confused than before. Was she not meant to ask who I voted for?

  17. As if Scoland and UKIP had NOT happened!

  18. Labouwr is not going to win until it re-invents itself and addresses the regional disparities in the country. SImply replaced Milliband with another of his failed gang is not the answer.

    It is time for a new generation to come to the front. Those without the baggage of Blair and Brown. Milliband’s loss confirms that putting new paint on the old is not the answer.

    Labour need a reformation. It needs to look long and hard at itself and its shortcomings, and then move to address them.

    Delusional accusations that it was someone else’s fault will only repeat the defeat or take it into the realm of disaster.

    Let new younger leadership take over and use the five years to re-brand the party. Labour’s brand needs more than just tinkering.

    They also need to remember that voters given a choice by the real Tories and faux Tories will always choose the

  19. orginal

  20. A question to all the polling experts out there:

    If this election had been conducted using PR, e.g. Av plus, what would the result have been?

  21. @omnishambles

    The UK uses several different electoral systems. As well as FPTP it uses two slightly different forms of the Additional Member System (one in Scotland and Wales, and a slightly different version in the London Assembly), plus the Single Transferable Vote System (in most Northern Ireland elections and in Scottish local elections).

  22. @BristolianHoward

    Brandenburg’s very pleasant.

    Nothing special’s happened here. Same old same old.

  23. Almost forgot .. and a party list system for the GB regions in the UK European Parliament elections!

  24. @ David

    They couldn’t predict the result with the system we have, so I think it is probably be a bit optimistic to expect them to come up with an accurate answer for an entirely different system!

  25. @David

    The Electoral Reform Society have calculated the result using the d’Hondt method, the most proportional practical list system.

    Here’s the BBC report, I’m sure you can find more detailed results on the ERS website.

    http://www.bbc.co.uk/news/election-2015-32601281

  26. @David,

    Result would probably be Tory and UKIP in coalition. Maybe with the DUP/UUP supporting.

    But people don’t necessarily vote the same way once you offer them PR, so we couldn’t really know for sure.

  27. I’m not sure that I would call the d’Hondt method ‘the most proportional party list system’. It actually slightly favours larger parties.

  28. I love the touching continuing faith on here in our ability to know anything to a degree of accuracy that would be useful!!!

    ‘we couldn’t really know for sure’ just about sums up the state of opinion polling at the moment ;-}

  29. TOH

    Thanks for that. It was roughly what I thought you had done but was surprised to the level you suppressed the VI headline figure in your analysis. Very brave. But then great steps of insight sometimes require bravery to shrug off the weight of the status quo way of thinking and move forward without the armour of groupthink to protect you from looking silly.

    When I get the time I’ll try and see how strongly those other factors have been signals. It sounds like, certainly in this election they were more important than the overall headline figures.

    Historically, a big lead in “best PM” would tend to coincide with a lead in headline figures, it’s what happens when the two disagree that we might find out more about how people actually vote, your assumption that key questions were strong indicators is very different as I would have assumed that those would have shown up in the overall VI figures.

    Instead it seems that a large proportion of people who said “Voting Labour, Cameron best PM, Conservative best on Economy” actually voted Conservative, instead of voting as they said they would. Treating these indicators as separate to the headline figures and not the causes of the voting figures is a pretty radical idea.

    I’m busy too but I’ll probably come back to the issue later when everything has calmed down with a few questions once I have the implications of this sorted in my mind.

    Many thanks for your thoughts and your willingness to stand out like a sore thumb.

  30. It seems that the pollsters need to rethink their methodology.
    Labour share of vote in 2010 was 29.0% and in 2015 was 30.4%.
    Conservatives was 36.1% and 36.8% respectively.

    Before election mostly all the polls were predicting 33-34% for each party, whereas it appears Labour averaged 30% and Conservatives around 36.5%.

    Had they managed to record this 6.5% conservative lead over Lab
    then the election result may have not been that much of a shock

    Perhaps in future Anthony you should consider adding 3% on to Consewrvatives polling figures and subtracting 3% off Labour.

  31. Thank you to all who responded to the PR question – very interesting.

  32. Guymonde – “I don’t know enough about Jarvis to have an opinion.”

    I hadn’t heard of him till people started mentioning him here and elsewhere – but maybe it’s a good thing that he’s a bit unknown?

    There’s a thread on him on Reddit just now where they are debating whether the British voter would accept an ex-soldier as PM.

    Turns out we haven’t had a volunteer soldier as PM for well over a century. The PMs of the 20thC who had served all did so as conscripts in the world wars.

    Various people mentioned the American experience: McCain and Kerry doing badly, but they served in Vietnam, a war the Americans lost (bad karma associated with that). And Eisenhower doing well because he was associated with WW2, as was Kennedy.

    Anyway, it’s definitely a new sort of backstory (i.e. not spad).

  33. @UKElect

    “I’m not sure that I would call the d’Hondt method ‘the most proportional party list system’. It actually slightly favours larger parties.”

    Quite so, that’s why I didn’t, you’ve removed the word ‘practical’ from what I actually said: “the most proportional practical list system.”

    I intentionally inserted ‘practical’ as purely proportional systems can result in massively fractured political ecosystems where governments become strangulated by a huge multiplicity of minor parties. Even with d’Hondt (known as Bader-Ofer locally) Israel is a victim of this fate.

    Equally, whilst d’Hondt does have a weighting towards larger parties many ‘purer’ list systems (like Sainte-Lague) actually exist within a context where a representational cap is in place so that parties with below say 5% of votes receive no legislators at all (as is the case with German Federal elections to the Bundestag), which I would argue is less proportional in reality.

    So yes, it’s a view, but I’d go with the ERS that d’Hondt is a good combination of practicality and proportionality.

  34. It’s occured to me (actually yesterday) that Nick Clegg would probably like to go and hide in amongst his backbenchers now – except he hasn’t got very many. I think they will all have to double up on briefs and he will have to take one or more himself. So he’s not going to disappear… unless he resigns his seat.

    Also – any leadership contenders out there for Labour?

    I saw a (dodgy) poll in the Mirror yesterday asking who would make the best leader, with the predictable options: “someone else” came first. Oh dear.

  35. Hello all,

    YouGov just asked me a series of probing questions about my political alignment – where I place myself on the left-right scale, how I tactically vote, which party I identify with and some other questions that clear were about establishing where I am on the Authoritarian/Liberal Capitalist/Socialist spectrum.

    Think that’s something they’re using to set up new filters and weightings?

    Anyway – result was a shocker, I’ve grieved a bit. Mourned the passing of the LibDems and learnt today some very good local councillors lost their seats. Hey ho.

    At least with the passing of Bob Russell in Colchester I’m now free to vote Labour.

    I’m also very excited about Dan Jarvis who seems like the obvious choice to re-establish the Labour party, as early as pre-referendum he was talking about neglect of Labour heartlands, the lack of trust in politicians, the threat of UKIP in heartlands and the problems politicians have talking in plain English.

    Also ex-soldier, Served in afghanistan, doesn’t like SpAds and wants more people in politics with real life experience.

    He may be the man we need :)

  36. Amazing thing is how inefficient Labour’s vote was this election, actually less efficient than the Tories.

    Not sure how the boundary changes and reduction in MPs will effect that, but I expect negatively.

    If it’s not something that reverses, or worsens, Labour may become much more keen on a more proportional system. But for now should definitely focus on winning under the existing rules

  37. re. the Labour leadership (speaking as someone who will have a vote).

    I think two things need to happen before 2020. The first is that Labour has to re-find its roots in communities outside London. It has become a very technical party. Milliband’s resignation speech mentioned “social justice” for example, and all that talk of “progressive” politics. That sort of language doesn’t cut it on the streets of, say, Morley. They need to find out what real people want to vote for. The bedroom tax, nom-doms, ending free schools – all worthy policies which won’t actually get anyone down to a polling station. Zero hour contracts and rent regulation, much wider appeal, but were fluffed and unclear in presentation. Labour need to learn how to craft a simple policy that has a clear benefit to a large section of the population (eg. Introducing a minimum wage, which was a genius policy in this regard) and that only comes from understanding people that Labour clearly currently don’t understand.

    Secondly, Labour needs to rebuild its election winning machine, which used to be brilliant. It is I think a demonstrable fact that it is difficult for Labour to win elections. New Labour’s obsessive micro-management was much derided but it was there for a purpose. Where was the rapid-rebuttal unit, for example? Did someone think it wasn’t necessary this time around? Why did no-one tell Ed that the stone was a bad idea?

    So my choice of leader would be someone that addresses these two issues.

    I don’t think it will be Burnham (too nice, won’t be ruthless enough).

    I don’t think it *should* be anyone from London (sorry, that may seem unfair to some great candidates but we need to broaden our appeal).

    I like the cut of Dan Jarvis – military grade organisation, regional accent. But he is untested.

    I like David M, who I voted for last time around, but I don’t think we can do another Miliband just yet, and I’m pretty sure he will feel the same way.

    I would be in favour of the sort of selection process that has been mentioned further upthread. In fact I’d be in favour of a primary style election from anyone that was a declared Labour supporter (I think).

    At first I thought that it wasn’t so much 1992 as 1983 for Labour. But I am slightly more positive now.

  38. The parliamentary Labour party might do best to pick their last choice in leader, and that’d probably be the best choice with the electorate!

    Alright, that may be slightly flippant, but not entirely.

  39. @ Bubhubblebub

    On Jarvis, you may well be right! I would like to see him well tested in the campaign though. How well does he present himself to the public? How well does he think on his feet when challenged? His backstory certainly gives him something distinctive and interesting, and the ‘having another job other than politics’ is massively important, but we don’t know yet how well he can do politics nor what his ideas are.

  40. @Foolberry – I think anyone from the pre-2010 intake should be avoided, and I’m favouring Jarvis (as long as he doesn’t fluff the interview)

    He may be untested – but he is also unencumbered by baggage, and was remarkably prescient about how things were going to go down in 2015 last year (not that anyone in Labour wanted to hear it). He also had the sense to try and oust Miliband before the election.

    It’s worth remember Cameron was the new boy when he ran against David Davis, and other candidates who might have the right ideology may not be able to appeal broadly enough to win, or make significant headway in 2015.

    Ultimately, this wasn’t just Miliband (although, after reading ‘five year mission’ I did get the sense he paralyzed the party from the top and was unable to provide leadership from within) – but the whole strategy. A silver lining to the debacle is Balls and Alexander have gone.

    I agree Labour needs to go back to the country and win appeal in places it doesn’t currently understand, like my little corner of Essex, and to do that requires humility. I really think new blood is the best way forward.

  41. I feel a little more chipper. Despite Labour’s poor seat count, if the party is sensible it should be able to prevent another Tory majority at least. Next time, they won’t have the Lib Dems to rescue them.

  42. @Foolberry

    Agree wholeheartedly.

  43. @AW

    Why is my previous post in pre-mod?

  44. For what its worth I think a key lesson for Labour would be to find someone who doesn’t talk politics and is not partisan, and can talk to ordinary people.

    Much of their messaging has been about how bad the Tories are, how unfair the media is, how bad the rich are, milifandom, etc. ie they are talking to themselves in Westminster speak.

    Meanwhile the electorate is wanting to know who is going to run public services the best, sort out the wait to see GP’s, sort out the roads so we are not in traffic jams, etc.

    Hopefully all of those 4m conversations have produced a list of things people are concerned about. Now try talking about those issues – not in terms of who is to blame for them, but in terms of what your ideas are to fix them and that you recognise they are issues that need fixing.

    ie talk to the voters, not to each other.

    All parties could take that advice really. Adopt a servant attitude, stop squabbling amongst each other and with other parties and you may just see things turn around dramatically. And smile, like Nicola does, it works wonders.

  45. @Foolberry

    I can’t link here unfortunately, but I believe well. He speaks plain English, and talks sense.

    If you google him you’ll see video of him at his victory interview on thursday on the Daily Star, he also gets headlines in the Daily Mail about ‘staring down muggers’ and being spoken of approvingly as ‘Ex-SAS’ – the Telegraph also refers to him as ‘steely’. I don’t like the Right-Wing press, but they like him – and will find it much harder to attack him.

    He also has a regional accent, but speaks clearly. He’s the perfect ‘real bloke’ counter to Nige.

    Shame he wasn’t the leader in 2015 as I think we could have won with him, and my one worry about picking him for 2020 is that the battleground might change and we’ll end up fighting the last war.

    But he ticks all my boxes. As I said, I hope he interviews well – if he does he’ll be the obvious choice of leader for the party, as long as it doesn’t come down to some esoteric debate about ideology and Brown/Blair reruns.

  46. Perhaps in an effort to avoid the problem of polling inaccuracy, the polling question could be along the lines of “do you want person A, B or C to be PM? Perhaps state which party they come from too.

    With regard to the oft-mentioned Labour 35% strategy: perhaps they should have gone for a 45% strategy instead. That way when the DK’s break the wrong way, it doesn’t cost them the election. But… that probably needs a very different mindset, but it’s one they’ve had before. Perhaps they needed this defeat to wake them up.

  47. It is worthwhile to look at Jarvis’s speech after the by-election on YouTube.

  48. poor clegg, he goes and 3500 new Lib Dems pile in, the most toxic man in politic history

    rather odd considering he isn’t exactly unlikeable,

  49. @BubHubbleBub
    “I agree Labour needs to go back to the country and win appeal in places it doesn’t currently understand, like my little corner of Essex, and to do that requires humility. I really think new blood is the best way forward.”

    And Kent. Labour doesn’t have a single MP in Kent.

  50. bubhubblebub – that was probably the British Election Survey, though there is a great big “gather data for analysis” political survey out there too. As you might imagine, both are bits of data that we’ll be using to help work out what went wrong and set up a new model.

    Mark – “Perhaps in future Anthony you should consider adding 3% on to Consewrvatives polling figures and subtracting 3% off Labour.”

    That would be *cheating* :). We need to find a system that produces figures like that to begin with.

    Katie Mullins – the person who stopped you wasn’t an exit poll person, it was someone telling for a political party. They’ll have a copy of the electoral register back at the local party HQ which they marked you off of as someone who they know has already voted so that later on election day no one from that party would bother coming to knock on your door to encourage you to go and out and vote.

1 11 12 13 14 15