I’ve just got back from the BBC after working all night (you may have seen my bald spot sat just to the left of Emily Maitlis’s big touchscreen last night) and am about to go and put my feet up and have a rest – I’ll leave other thoughts on the election until later in the weekend or next week, but a few quick thoughts about the accuracy of the polls.

Clearly, they weren’t very accurate. As I write there is still one result to come, but so far the GB figures (as opposed to the UK figures!) are CON 38%, LAB 31%, LDEM 8%, UKIP 13%, GRN 4%. Ten of the final eleven polls had the Conservatives and Labour within one point of each other, so essentially everyone underestimated the Conservative lead by a significant degree. More importantly in terms of perceptions of polling it told the wrong story – when I was writing my preview of the election I wrote about how an error in the Scottish polling wouldn’t be seen so negatively because there’s not much difference between “huge landslide” and “massive landslide”. This was the opposite – there is a whole world of difference between polls showing a hung Parliament on a knife edge and polls showing a Tory majority.

Anyway, what happens now is that we go away and try and work out what went wrong. The BPC have already announced an independent inquiry to try and identify the causes of error, but I expect individual companies will be digging through their own data and trying to work out what went wrong too. For any polling company, there inevitably comes a time when you get something wrong – the political make up, voting drivers and cleavages of society change, how people relate to surveys change. Methods that work at one election don’t necessarily work forever, and sooner or later you get something wrong. I’ve always thought the mark of a really good pollster is someone who puts their hands up to the error, says they’ve messed up and then goes and puts it right.

In terms of what went wrong this week, we obviously don’t know yet, certainly I wouldn’t want to rush to any hasty decisions before properly looking at all the data. There are some things I think we can probably flag up to start with though:

The first is that there is something genuinely wrong here. For several months before the election the polls were consistently showing Labour and Conservative roughly neck-and-neck. Individual polls exist that showed larger Conservative or Labour leads and some companies tended to show a small Labour lead or small Conservative lead, but no company consistently showed anything even approaching a seven point Conservative lead. The difference between the polls and the result was not just random sample error, something was wrong.

I don’t think it was a late swing either. YouGov did a re-contact survey on the day and found no significant evidence of this. I think Populus and Ashcroft did some on the say stuff too (though I don’t know if it was a call-back survey), so as the inquiry progresses other evidence may come to light, but I’d be surprised if any survey found enough people changing their minds between Wednesday and Thursday to create a seven point lead.

Mode effects don’t seem to be the cause of the error either, as the final polls conducted online and the final polls conducted by telephone produced virtually identical figures in terms of the Labour/Conservative lead (though as I said on Wednesday, they were different on UKIP). In fact, having a similar error with both telephone and online polls is evidence against some other possibilities too – unless by freakish co-incidence unrelated problems with online and telephone polling produced almost identical errors it means things that only affect one modeare unlikely to have been the cause. For example, if the problem was caused by more people using mobile phones, it shouldn’t have affected online polls. If the problem was caused by panel effect, it shouldn’t have affected phone polls.

Beyond that there are some obvious areas to look at. Given that the pre-election polls were wrong but the exit polls were right, how pollsters measure likelihood is definitely worth looking at (exit polls obviously don’t have to worry about likelihood to vote – they only interview people physically leaving a polling station). I think differential response rates is something worth examining (“shy voters”… though I think enthusiastic voters is just as risky!), and the make-up of samples is obviously a major factor in the accuracy of any poll.

And of course, it might be something completely unrelated to these things that hasn’t crossed our minds yet. Time will tell, but first some sleep.


710 Responses to “Back from the election”

1 12 13 14 15
  1. that last comment regarding his personality, charm etc

  2. To all those discussing Labour’s need for a ‘new blood’ leader, how about Stephen Kinnock?

    That should ensure electoral oblivion at the next election!

    N.B. To Lab fanatics – this is an attempt at humour.

  3. My previous attempt to make this point went into pre-mod but the press would find an ex-paratrooper difficult to traduce in the way they dude to Ed Miliband.

    Also, would you dare insult an ex-Para?

  4. @Richard

    “they are talking to themselves in Westminster speak”

    I couldn’t agree more.

    Voters aren’t stupid, but equally are not party to the jargon of the politicians, SPADs and senior civil servants nor, quite rightly, do they feel they should have to learn a different language just to understand what these ‘alien creatures’ are saying to them.

    This isn’t a message just for Labour, we should remember that the Conservatives are stuck below 37% too. Moreover David Cameron’s attempts to ‘talk normal’ in the final days of the campaign came across quite oddly, Matthew Parris described the ‘gets me pumped up’ speech as the ‘fake org*sm’ moment of the campaign.

    One of the reasons for both Nicola Sturgeon and Nigel Farage having such success (at different ends, arguably, of the political spectrum) is their ability to talk in a language their audiences understand.

  5. Dude=did.

    Damn phone!

  6. I am really puzzled by the idea that any of those who have failed Labour in the last two elections should have a role in the next one. Does the party have a death wish?

    It is time to say thanks and goodbye to the whole lot of them. I hope that Ed Miliband leaves parliament and heads for a nice cushy job abroad like his brother and that we never see or hear of either of them again. If Ed Balls can get a decent job in Europe and take his wife with that would be terrific. Andy Burnham is a nice chap, but any Labour Health minister from the Brown government is permanently tainted by the Mid Staffs Hospital scandal (I know that this was nothing to do with Burnham, but politics is not fair). He can stay on the backbenches as long as he keeps a low profile. Alan Johnson should stick to writing books.

  7. Totally agree with getting rid of Spad-speak.

  8. There is something quite interesting with YouGow NowCast. If you look at some of the more marginals – I really have only a few – you would instinctively know that the leader would lose. But there and then it is “leaning to …” In the poll. There must be an equivalent in the national VI.

  9. @Katie

    Anthony’s explanation was spot on and in the old days the duplicated pads the information was written on were known as Mikardo pads in the Labour party in honour of a staunch socialist MP who invented them… (ok this is all probably mythology but it sounded good at the time)

  10. @ Bubhubblebub

    ‘as long as it doesn’t come down to some esoteric debate about ideology’

    I agree but we do need to know what is guiding framework is; where his instincts are likely to lead him when confronted by political and policy challenges

  11. @ Hawthorn

    The media can go after anyone. And can be very effective.

    Over many years I have taught officers who left the army and who have been in war from the first gulf war to Afghanistan. They are just as varied as “normal” people.

  12. To any of those who are suggesting David Miliband as a candidate for Labour leader. Ask yourself this question, how long before the message “David, he wasn’t even good enough to beat Ed, and he was a loser” gets pushed relentless by the Tory friendly parts of the media.?

    It’s a cheap jibe, but hey, that never happens does it.

  13. Balbs

    They were actually called Reading pads after his constituency.

    Funnily enough, I have been in Mikardo’s office. Memorably, it had a Spitting Image puppet of Thatcher in its bathroom. Kept people regular I suppose!

  14. @BubHubLeBub

    I’ve just googled him and found the following lengthy interview with him:

    http://www.newstatesman.com/politics/2015/03/war-westminster-labours-dan-jarvis-future-prime-minister

    Apparently his campaign to win his by-election in 2011 was called “Operation Honey Badger”.

    I think the Lab people can rest assured that he would never embarrass them with The Stone if he became leader :-)

  15. If there iwere a YouGov poll tonight wouldn’t it be fun if it showed the two main parties level pegging!! :-)

  16. Interesting that Lib Dem membership has suddenly started taking off. I decided to rejoin yesterday as well after a 5 year lapse.

    I wasn’t against the coalition, I’m not particularly anti-Tory, and the tuition fees thing was completely overblown nonsense (I wish my student loans were under the more favourable present arrangement!).

    I’d simply had enough of them being outplayed by their coalition partners and felt there was an arrogance in the leadership that needed fixing.

    As far as I was concerned, they had ONE job to do in coalition, which was to forcibly bring a decent PR system to UK general elections. A secondary follow-up was to introduce an elected House of Lords. That was it. And they completely failed thanks to their excruciating political naivety, consigning themselves to oblivion for who knows how long.

    However now they’ve been kicked, I’m quite happy to support them getting back on their feet. We need a strong, alternative, liberal voice in politics, particularly one that has an unyielding stance on civil liberties that neither of the other major parties (with the exception of the Greens) stands up for.

  17. Laszlo

    The flavour of my post got lost due to pre-mod.

    It is difficult to traduce Jarvis as disloyal to Britain or being a n€rd.

  18. @AW – I feel all flattered being responded to. I answer honestly, I feel I should state that as everyone else appears to be lying to pollsters.

    @Richard & Assiduously – Agreed, I’ve spent a good long while in politics and learnt to parse SpAd speak, but even I found ‘predistribution’ tough to link to an emotion. It sounds like a process undertaken at a factory farm.

    This problem was particularly evident in the Labour campaign, from the grimly hilarious irony of vague vacuous ‘pledges’ carved into a slab of limestone, to Miliband trying to get the phrase “Britain succeeds when working people succeed” to have some sort of ‘Yes We Can!” resonance.

    Utter fustercluck, but I went to the polls to see how it was taken, and the headline VIs provided false assurance :)

    Still, onwards and upwards. Time for a remorseless cull of the SpAds and detritus (the ones who survived the demise of Scottish Labour).

  19. They were called Reading Pads and Mikardo Pads. Both terms were in common use.

    Funnily enough, the Tories were using them on the polling station I was at on Thursday.

  20. @ Hawthorn

    I agree with your point.

    But I would advise that Jarvis makes sure that his MA thesis disappears from the library (I don’t know King’s collage’s policies about this). Just in case.

  21. Norbold

    I had not heard the Mikardo term. Thanks.

    I remember David Miliband talking about a “concordat with the British people” in his leadership bid. I am an Oxford PPE grad and even I found it difficult to know what he was getting at.

    I remember Burnham saying that 2010 was down to poor communication and discounted him at that point.

    Balls came across too badly and Abbott was Abbott.

    So by a process of elimination I voted for Ed. I still think he was the best of a weak field.

  22. @Candy – I don’t see why that’s embarrassing, it’s a bit funny and endearing to me. Honey badgers are terrifying :)

    @Newforestradical – Ideology is important to a party, Leadership qualities are tougher. Ultimately Miliband was an ideological leader, as was Brown. Both lacked the ability to inspire the public, or appeal outside groups that were not already ideologically positioned to agree with them.

    When the leadership election comes up we shouldn’t fall into the trap of picking the person who comes from ‘our school’ above everything else, but the person who is the face of the party and the one who leads the fight.

  23. Thesis = dissertation

  24. PETE B
    To all those discussing Labour’s need for a ‘new blood’ leader, how about Stephen Kinnock?
    That should ensure electoral oblivion at the next election!

    I laughed out loud at that. Conservative central office dream.

  25. I just stumbled across a coloured map showing 2nd places in the GE. In some ways gives a better picture of the changed landscape than the winner’s map. There is an awful lot of purple south of the Humber

    https://twitter.com/louisa_compton/status/596920398605869056/photo/1

  26. Rick
    :-)

  27. Not being privy to the inner workings of the Labour Party, I’ve no idea to what extent the National Executive Committee has latitude over the format of the election (I understand the electorate is set out in rules), but…

    Wouldn’t a series of regional primaries arranged over the course of a period of time culminating in a convention along the lines of the American model provide an opportunity for the party to reconsider its positioning at the same time as selecting a leader and keep it in the spotlight at the same time?

    It seems to me that, lacking a national leader, if they don’t hold the presidency, the American political parties essentially have to reinvent themselves every few years and do so through this process. It might work here.

    Many in Labour seem to be worried about the Conservatives ‘setting the agenda’ as they did in 2010/11. However, is that such an existential threat now?

    There is no Labour legacy to trash, that work has already been completed.

    The Conservatives will need to put forward a solid and attractive legislative programme, the perceived shortcomings of which Labour and others can oppose in a much more conventional way. Much of 2016/17 will be consumed by Europe anyway, which could well erase memories of the first part of the parliament.

    Time to redefine and select the right person would be better.

    A truly unusual suggestions, but if the LibDems and UKIP ran parallel processes this might actually attract the attention of the public (or at least the media) as the presidential primaries do in the US.

    And, best of all, they could provide pollsters with an opportunity to repair their reputations by accurately predicting the winners!

    Well, that’s that sorted.

  28. @Lazlo

    I agree that there are different types of ex-army officers.

    On the one hand there are the Dan Jarvis types, and on the other extreme there are the Ian Duncan Smith types. Being ex-army did not help IDS with the press.

  29. Nobody has mentioned the Council Election results as far as I know. It appears the Tories are doing OK in these as well. Figures from BBC website

    Tories +495 seat +31 Councils
    Labour -350 seats – 4
    LDem -350 seats -4
    Ukip +159 seats + 1

  30. AW

    Just flipping through the comments, it’s much too early to draw conclusions.

    Keep the faith, do. Polling is important to keep the politicos honest.

    You just need to fond out what was wrong and put it right.,

    In fact to do that you’ll need to run some polls!.

  31. @ExileinYorks

    Ukip’s future depends on the EU Referendum.

  32. @TOH – The council figures are scary because they show the election was not just about ‘keeping out the SNP’ as some in Labour and the LibDems believe, perhaps self-comfortingly.

    It shows there has been quite a resounding rejection of Labour, who are the primary opposition. That this happened five years into a parliament is terrifying.

  33. There’s a guy on a newspaper website bragging that he and his Tory friends knowingly withhold information from pollsters in order to deliberately distort the data, thus throwing the pollsters predictions.

    This apparent strategy to deceive is set up to create the illusion that Labour think they are doing better than they are.

    This sounds plausible I guess but either way, plausible or not, pollsters in this context will never be able to predict the outcome of an election if a sizeable set of respondents set out to mask their intent deliberately.

    I wouldn’t be surprised by this strategy as the right wing are more politically savvy than the vast majority of ordinary people.

  34. Interesting article –

    http://blogs.spectator.co.uk/coffeehouse/2015/05/there-was-one-pollster-who-predicted-a-conservative-victory-jim-messina/

    I’ve heard Crosby & Messina used Telex polling, which I haven’t heard of, and it produced extremely accurate showings of how seats were going.

    If Labour had their own pollsters, what was their methodology? Were they just waiting for the papers to do them for them and relying on canvassing returns?

  35. @Assidousity

    “There is no Labour legacy to trash, that work has already been completed.”

    You’d be surprised. Not for nothing is there a joke that runs:

    How many Tories does it take to change a lightbulb? Two. One to not do anything about it and one to try and blame the failure of the old bulb on the Labour party who put the original bulb in place 17 years ago.

  36. @sssimon

    “Interesting that Lib Dem membership has suddenly started taking off”

    Actually it started increasing like 6 months before the election

  37. TOH
    Firstly thanks for providing the local results, fantastic, secondly I could of sworn Colin announced excellent news about his health last week.
    He tells me he did not and that it was you not him. Whatever, I am equally delighted. Its all good news Howard, what a result, how you had the insight in the face of the polls, I don’t know.

  38. FASTFOURIERTRANSFORM
    “….pollsters in this context will never be able to predict the outcome of an election if a sizeable set of respondents set out to mask their intent deliberately.

    I wouldn’t be surprised by this strategy as the right wing are more politically savvy than the vast majority of ordinary people.”

    But would there be sufficient numbers of them to make a real difference?

  39. @FASTFOURIERTRANSFORM

    It sounds to me like some loudmouth spouting off. Shy Tories are just people who think it is none of your business how they vote. They rarely aim to deceive but are bound to distort the numbers. It is a very conservative, both large and small , thing.

    I don’t think online panels, No matter how large, can ever pick this up. The best they can do is to indicate the direction of travel. Most so called shy electors would never register for them. Telephone polls have a similar built in bias because the non responders are probably disproportionately Tory. They may however have a better chance of picking up the likely VI from the answers to other questions.

  40. Large and small C. Auto correct thinks I just can’t spell.

  41. FASTFOURIERTRANSFORM
    There’s a guy on a newspaper website bragging that he and his Tory friends knowingly withhold information from pollsters in order to deliberately distort the data, thus throwing the pollsters predictions.
    This apparent strategy to deceive is set up to create the illusion that Labour think they are doing better than they are.
    This sounds plausible I guess but either way, plausible or not, pollsters in this context will never be able to predict the outcome of an election if a sizeable set of respondents set out to mask their intent deliberately.
    I wouldn’t be surprised by this strategy as the right wing are more politically savvy than the vast majority of ordinary people.

    @Fastfour,

    I don’t buy this conspiracy rubbish. Labour need to look internally at why they didn’t get elected, not do the normal left of the party Union trick of blame it all on the Tories, thatcher, conspiracy.

    I can’t see how all of the last 11 polls could be distorted like this. And, just to put the nail in the coffin of your point, survation did produce a more accurate poll, but disgracefully didn’t use it as they wanted to fall in with the others. This is where you should be looking.

    It’s up to Lab whether they are willing to look at their own performance or not.

  42. RAF

    @ExileinYorks

    “Ukip’s future depends on the EU Referendum.”

    That simple statement covers a very complex situation with possibilities ranging from a splintering of the Tory party and UKIP growing dramatically, through to virtual oblivion for UKIP and a much stronger Tory party that no longer needs to worry about erosion on their right flank.

  43. @ catmanjeff

    “This is very interesting:”

    This rings very true with me. The North East has many of the pre-conditions that ultimately led to the LiS meltdown on Thursday. What the NE lacks is a strong enough regional identity for a local version of nationalism to take root. There is however, a vacuum and sooner or later something will fill it.

  44. “Only a Northern Party.”

  45. “Nobody has mentioned the Council Election results as far as I know”

    I think the Thanet council result shows that a percentage of the Ukip vote bottled at the last minute over the SNP scare.

  46. @ExileInYork

    Thanks for posting two really interesting links

    I think the map of second places is a key piece of information.

    On the one hand, it offers the LibDems a route way back into Westminster in larger numbers as there are clearly some areas and some seats where they still have opportunities.

    The bigger picture though are those thick veins of purple running through England, outside of London and south of the wash. How many of these will turn into genuine UKIP targets will depend on whether that party survives or is strengthened by 2017, this could turn UKIP into a major long term disruptive force in politics in the future.

    I’m less convinced by Paul Mason’s article. There are elements of truth here no doubt, but to presage the undoubted terminal decline of the Labour party on an election in which – despite the SNP phenomenum – it managed to fractionally increase it’s share of the national vote seems extreme.

    Whilst his observation of the ailment seems reasonable his prognosis is unduly pessimistic, perhaps because as a journalist he offers no prescription for cure.

  47. @CatManJeff

    Sorry, the above was partly addressed to you as I’ve just realised you posted the link to the C4 article.

  48. @ Assiduosity

    I can only claim credit for one of the links.

    Catmanjeff found the Paul Mason piece, I was just commenting on it.

    A repeat of the LiS fiasco needs a special combination of circumstances, but addressing the issues is much easier before a viable alternative for your vote share emerges and starts to devour your base.

  49. @AnarchistsUnite

    “You’d be surprised. Not for nothing…”

    A very good joke… though one that applies in one variation or another to all politicians when in election mode don;t you think?

    What I meant by my comment is that it is unlikely that the Conservatives are going to develop any new narratives on Labour failure over the next 12 months.

    The problem – so the argument goes – is that the attack line of ‘Labour’s Great Recession’ was established while the opposition were introverted in the midst of a leadership battle in 2010/11 and could never be undone.

    That situation is different from now as all the lines of attack on Labour’s record are well established. It may be a greater problem if Labour rush the selection of a new leader and get it wrong in terms of their future electoral chances.

1 12 13 14 15