The day after the election the British Polling Council announced it was going to have an inquiry into what went wrong with the polls, we’ve now got some more information about how the inquiry is going to proceed. Over on the National Centre for Research Methods website they have announced the membership of the inquiry team, timings and terms of reference.

The Chairman of the inquiry, Pat Sturgis, was announced earlier this month. The rest of the team include several names who regular readers will be familiar with: Steve Fisher from Oxford University who ran the ElectionEtc model and worked on the exit poll, as did Jouni Kuha of the LSE, Will Jennings of Southampton University who is part of the Polling Observatory team, Jane Green of Manchester University who is the current Director of the British Election Study and Ben Lauderdale of the LSE who did the ElectionForecast model that was on Newsnight and 538. The rest of the inquiry team are Nick Baker of Quadrangle Research, Mario Callegaro of Google and Patten Smith of Ipsos MORI.

The terms of reference for the inquiry are to assess the accuracy of the 2015 polls and investigate the cause of any inaccuracy, whether it’s connected to inaccuracy at previous elections, to look into the possibility of herding, to see if enough information was provided and communicated to people about how polls were done and what they meant and make recommendations on how polls are conducted and published in the future and on the rules and obligations of the BPC.

The inquiry are inviting written submissions via their website, and there will be a public meeting on the 19th June – it’s due to report to the BPC and MRS by the 1st March next year.


571 Responses to “Details of the polling Inquiry”

1 2 3 4 12
  1. Well, the LibDems leaked 2 against Labour (because the NS was against Labour, maybe not in Scottish terms, but in reality) and two against Tories. They didn’t leak anything against themselves.

    So one could call it balanced. I really feel sorry for LibDem voters, but perhaps better days are coming now with a clearer electoral base.

    The rest what I would write (about articles in the New Statesman and alike) would be against the comments policies.

  2. Am I correct in thinking the following:

    1) The report will conclude that the problem was caused by differential response and/or differential propensity to vote
    2) The report will conclude that the solution is to weight the responses differently.
    3) This solution will work for 2020, possibly for 2025, but inevitably it’ll go wrong again at some point.

  3. @Statgeek

    Maybe Carmichael can be sent to see out the remainder of his days in one of the dwellings at Skara Brae.

  4. Carfew

    Re the cricket. New Zealand, who are a very good side have looked in no real trouble against England,s bowling attack I can see them posting a big lead and then spinning us out on the last day. One ball today turned outrageously today so England look in a bit of trouble.
    Does that help?

  5. Martyn,
    Sounds about right. You’ve saved them a lot of time. They can pack up and go home (and you can keep their massive fees!).

  6. What would be my questions for an inquiry? Without order of importance.

    1) what are you doing with the reject answers? Is it accounted anywhere? In academic research in social science one of the main obstacles is the almost complete lack of articles on negative findings.
    2) what are exactly the research questions and how are these related to the questions asked
    3) is there any reason to believe that those generic demographic variables are useful? (Age – maybe, but the rest are “it depends” – e.g. a class “B” in public service, large private company, small private company, entrepreneur. House owner, tenant, with or without mortgage, etc). Demography is ultimate, but can it be make it more sophisticated to match the change in the economic life, and how do we validate it.
    4) how can it be accounted for that there is a clear movement to a more liberal value system, affecting all parties?
    5) how can it be accounted for the fact that all parties are coalitions, but to a different degree, and it also varies by the elections. LibDem of 2010 is the prime example. I genuinely think this was the major one.
    6) how can protest parties be handled? We had this problem with UKIP, but I guess we will have it with the Greens next time.
    7) what is the social process in which one rejects a party (Labour) and then expresses the desire for voting for it (doesn’t materialise anyway). If this exists, how it is filtered in the polls.
    8) considering the diversity of the UK, or even England only, what justifies the sampling design?
    9) is there a room for a Bayesian methodology (I think there is and offered three different ways, but I don’t think would happen).
    10) there is no shy Conservative, it’s an excuse
    11) there could be a shy UKIP voters, but it should be detectable
    12) should polling companies be allowed to reclassify the VI on the basis of supplementary questions?
    13) how do you work with fixed constituencies but without the knowledge of ward level voting in general elections (what’s the function of using local elections?). A lot of constituencies have both metropolitan and quasi rural wards.
    14) what made most polling companies enlightened on the 7th?
    15) are you actually using the right questions to gauge the values of voters?
    16) should party ID (I know not all uses it) include not only the last but the one before election?

    Many of these suggests conditional VI, which then would call for a radically different methodology.

    I think there shouldn’t be any poll in the last week, and complete ban on campaigning in the last 48 hours. I can elaborate on these.

  7. RAF

    There is a theory that Skara Brae was a suite of sauna baths, and not a village.

    Demands of RESIGN! are common in politics, but I can’t see Carmichael doing so (unless he is found guilty of a breach of section 7 of the MPs code of conduct – and not necessarily even then). He already stated that he intended to retire in 2020.

    It’s difficult to believe the claims of John Mann MP that this enquiry cost £1.4 million!

    I identified Euan Roddin as the most likely leaker, on here within hours of the story breaking. Heywood can’t possibly be so incompetent that he couldn’t do the same. Interviewing Roddin then Carmichael could have done in an afternoon (and before the election!)

  8. There you go. I knew I was right about Statgeek and Laszlo being on my inquiry. Now e just need to hear from the others.

  9. @Oldnat

    He does need to ride out the storm. This is different from your run of the mill scandal because what he did (or, should I say, didn’t prevent) was an attempt to influence an election result and he has got in without his voters knowing something quite important. He was not elected and then breached his trust – in which case he can say it is for the voters decide in five years time.

    I agree with you that Paul Flynn and John Mann are surely over-egging the cost…

    I must say I feel slightly sorry for him. Plenty of other ministers that would have done similar and either not been caught or let the SpAd take the blame.

  10. @OldNat

    You did indeed call it correctly at the time.

    As far as the prospect of AC resigning is concerned, I suprct that would depend on whether, like Roddin, he believed the contents of the memo to be accurate, or whether he knew otherwise.

  11. Reports Bank of England has hush hush study of Brexit.

  12. @ 07052015

    While Ms Kendall would agree with what is attributed to her, it’s still a parody ((surely).

  13. RAF

    In his letter to Sturgeon, Carmichael says “I accept that’s publication was a serious breach of protocol and that the details of that account are not correct.”

  14. So with the enquiry taking a year -will commissioners suspend polling westminster VI and concentrate on the elections next year and the EU referendum.

    Cant see much money being spent ,certainly not daily polls.

  15. STATGEEK

    For consistency in your constituencies, you won’t better the PA Numbers included in BES last time and this. See my post at end of previous thread.

  16. @Statgeek

    Do you want any data checking?

    I’ve got a few hours spare.

  17. @CMJ / Anyone interested in some data checking – http://bloochat.com/7j4a8

    If I just post the links, my site will get bombarded with hits for the link, many from non-interested parties.

  18. …of course it helps if I give you the password…sigh…

    “statgeek” – of course :))

  19. @SunReada

    I don’t see the point of polling for GE 2020 right now. Last time round some early polling was interesting to see how people reacted to the coalition but this time seems a bit pointless, at least until LAB have a new leader. And even then I don’t see how polling on the methods that failed would be useful.

    I guess we’ll see ICM and MORI as AW suggests. Others will prob focus on London (once we finally know who the candidates will be), Scotland and maybe a few for Wales. Oh, and the EU Referendum.

  20. @THE OTHER HOWARD

    “Re the cricket. New Zealand, who are a very good side have looked in no real trouble against England,s bowling attack I can see them posting a big lead and then spinning us out on the last day. One ball today turned outrageously today so England look in a bit of trouble.
    Does that help?”

    ————-

    Thanks Howard, yes almost anything would help right now, given the state of our cricket, which like the polling, is in a bit of a muddle. One fears for the summer against Aussies. Maybe ECB should hire Crosby…

  21. Carfrew

    “One fears for the summer against Aussies.”

    And how !!!

  22. @Martyn

    Pretty much, though as per my “voter-blame” gambit, it would be:

    1) Voters were telling porkies *

    2) Adjustment is made to allow for the “porkiness” of it all

    3) since porkiness depends on embarrassment over parties and policies, which is not a constant and can change over time, at some the porkiness factor will change, and…

    4) this will be hard to spot, since the point is people are embarrassed by their choice and don’t want to say

    * not me, obviously, since don’t vote. It’s the naughty voters what dunnit…

  23. (…at some point the porkiness factor will change…)

  24. Hmmm…

    http://www.bbc.co.uk/news/business-32826643

    It seems reducing inequality is no not left wing, but mainstream – even the OECD agree.

    Interestingly, they also say that taxing the rich isn’t the answer, but a much more activist approach to raising living standards of the poorest.

    Looking back, this again seems to be a message Labour didn’t properly capture. I think Ed was correct in his view that things had gone badly wrong and change was needed, but his policy prescriptions appeared too concentrated on the punishment at the top.

  25. So Carmichael is guilty -he should resign.

    But as snp and tory election messages played to a common narrative -was their an understanding ,just asking ?Or was it a massive coincidence.? And what are the odds on that ?

  26. @Statgeek

    Your West Midlands data is fine.

    While reviewing it, two things were striking:

    1. The success of UKIP
    2. In many seats the Lib Dems were fighting with the Greens for very distance fourth places

  27. @ CMJ and Statgeek

    Woof. I was doing West Midlands. Ok, I do some of the NW …

  28. Sorry……

  29. @Sunreada – everything was very well coordinated between Con and SNP, but in the absence of evidence we have to assume it was not exactly coincidence, but a simple recognition of mutual interest leading both to independently coordinate their election messages.

    All of this was pointed out on UKPR by some posters at the time, and was roundly dismissed by a number of nat posters, who denied outright that the SNP had any interest in wanting a Con government.

    After the election, some of them admitted this was the case, which I found funny.

  30. @ Norbold

    Any inquiry would need you – the ultimate disadvantaged campaigner (and I meant it in the kindest possible way) – you probably directly and indirectly picked up more messages than …

    By the way my friends in the LP still haven’t been approached for their experiences. Interesting inquiry (sorry Carfrew).

  31. @CMJ / Laszlo

    Laszlo – York & Humber (done), North West (pending)
    CMJ – Wales (Was that ok?), West Mids (Done)
    Geek – London (done), South East (pending)
    Martyn – Asked him to do South West – Pending

    I’ll also take Scotland. That leaves North East, East Mids and Eastern.

    Mucho thanks chaps. DO as much or as little as you fancy.

  32. @ Alec

    @Alec

    Have you ever seen How to steal a million?

    ” we did it Dave, we did it”
    “Yes …”
    “Well, you had the lead, but I did my part didn’t I?”
    “Yes.”
    “Not bad, considering that it was my first …”
    “Mine too …”

    It’s so easy to create a conspiracy theory. If you think it over the interest of the SNP and Conservatives met. They stoked the same fire, but from different directions. They did know of each other, but they didn’t see each other, hence the occasional disharmony of shovelling coal on the fire.

    They both got what they wanted, and Labour can’t reproach either of them.

    If EM had a stomach to fight it through … Anyway …

  33. @Statgeek

    I will do Wales this weekend, if that’s okay.

    I’ll need a link though….the zip file I downloaded was for England only.

  34. @CMJ (Lib / Green)

    Yes. I have a lorra, lorra charts coming once I get the data checked. Some detail the highs and lows of the parties’ fortunes. Maybe at the start of the week, once I figure out the best way to get 50-100 charts without overloading the readers.

    Then there’s the GE2015 pages for the site. Thankfully there have been no boundary changes, so the old maps are ready to go.

  35. @Barbazenzero

    Good point. I almost never have to do a like for like, so that column and many others probably disappeared early in the 2010 data gatherings.

    Will have to refer to that in future.

  36. @CMJ

    No probs. I had done Wales, but then decided to redo all anyway. I’ll do Wales and Scotland (and Northern Ireland). Not much to do tomorrow anyway. :))

    How about North East, East Mids or Eastern?

  37. @Statgeek

    North East for me thanks.

  38. Lazlo @ Alec

    On the squeezing of Labour, here’s a more analytic (avoiding any need for conspiracy theory) account of the election from a Green source.

    http://www.betternation.org/2015/05/apex-predator/

    The Tories are back, though, and they appear to have developed a new art, or refined an old one: a trick which should give any party considering working with them good reason to think twice about it. They have become very adept at destroying their partners, and it is no mere coincidence. They know exactly what they’re doing as they do it.

    The “partners” they destroyed being the LDs and LiS.

  39. @ OldNat

    Interesting article. Thank you.

    I tried to make it very clear that I didn’t think that there was a coordinated action (neither here or there really), but it worked.

  40. Lazlo

    I didn’t think for a moment that you subscribed to the conspiracy theory!

    You made that very clear.

    I thought you would find an explanation from a non-conspiratorial Green interesting, however.

  41. @ Statgeek

    The NW figures are correct. I have doubts your classification for Liberals, but it’s yours.

  42. @ OldNat

    Thank you. I found the perspective interesting, but I also found that it overestimated the Conservatives. I don’t think that there was a plan. But they were sufficiently flexible to adjust …

  43. Lazlo

    Yes. Probably attributes to pre-planning what is actually a skill to take advantage of developing opportunities.

    The latter is a great advantage when your targets show political ineptitude.

  44. @ OldNat

    Yes, with a hindsight (for me anyway) …

    The oddity is that the same message could have been sold in a very different (well, everyday) way. It could have worked.

    Completely different – it was the the polls that convinced me that his message could work. I had no trust in it originally. One of the oddities of this campaign is that EM left behind so many relics that could become extremely useful in the future for the LP … Well, we see how the LP can use or discard it.

  45. I just recognised that I used in my metaphors Audrey Hepburn for describing the conspiracy theory about NS. I hope it’s OK?

  46. @Statgeek

    …and i’m back in the room. Yes the south-west.png checks out OK against http://www.bbc.co.uk/news/politics/constituencies . Note that the totals were not on the bbc page and so could not be checked

  47. Lazlo

    I knew I recognised the script from somewhere! :-)

  48. Re Carmichael, perhaps this Tom Lehrer song is apposite.

1 2 3 4 12