We don’t have any more information on how the British Polling Council’s review of the election polls will progress beyond it being chaired by Pat Sturgis, but several pollsters have given some thoughts today beyond the initial “We got it wrong and we’ll look at it” statements most pollsters put out on Friday. Obviously no one comes to any conclusions yet – there’s a lot of data to go through and we need thoughtful analysis and solutions rather than jumping at the first possibility that raises its head – but they are all interesting reads:

Peter Kellner of YouGov has written an overview here (a longer version of his article in the Sunday Times at the weekend), covering some of the potential causes of error like poor sampling, late swing and “shy Tories”.

Martin Boon of ICM has written a detailed deconstruction of ICM’s final poll which would be have been an interesting piece anyway in terms of giving a great overview of how the different parts of ICM’s methodology come together to turn the raw figures into the final headline VI. Martin concludes that all of ICM’s techniques seemed to make the poll more accurate, but the sample itself seemed to be at fault (and he raises the pessimistic possibility that sampling techniques may no longer be up to delivering decent samples)

Andrew Cooper of Populus has written an article in the Guardian here – despite the headline most of the article isn’t about what Cameron should do, but about how the polls did.

Finally ComRes have an overview on their site, discussing possibilities like differential response and the need to identify likely voters more accurately.


674 Responses to “The polling post-mortem – some pollsters’ thoughts”

1 11 12 13 14
  1. JAMES

    I really don’t know what will happen with the referendum. There are too many variables to even guess.

  2. @Robin Hood

    “SWING BACK” THEORY HAS BEEN PROVED CORRECT”

    No, since we’re using capitals it is “THE POLLS CONSISTENTLY OVERESTIMATE LABOUR SUPPORT THEORY” (and variants on that theme) that has been shown to be correct.

    There is a difference. For “swing back” theory to have been proved correct, the polls would have had to be right in October 2014 and wrong in May 2015.

  3. @Roland

    “The Indie is hardly a Tory paper. ”

    Well it’s definitely not a Labour paper, since its editorial came out in favour of a repeat Tory/LD coalition. It probably didn’t make any difference since it has so few readers. And after that editorial it’s going to have lost a lot more.

    Since the i didn’t run the editorial I’ll continue to buy it.

  4. @Phil Haines

    I suppose the real question arising from your analysis is – do we really know when the polls were last right (if at all)? Without knowing that we can’t really validate either/any theory.

  5. @Rob Sheffield

    Yes, a Labour leader (and for that matter a Shadow Chancellor) that is media savvy would in itself make a big difference.

    I also agree that Labour lost the “Labour’s mess” argument comprehensively. The problem was that, while convincing next to no-one with what Balls tried to make sound like an austerity-lite agenda (but wasn’t really), in scaling down its ambition Labour also failed to galvanise those who really wanted change. “You’re all the same” was still the refrain on the doorstep.

  6. How do we think the pollsters will do on the EU referendum -lots of potential for shy I am a voter get me out of here.

    Wonder what a merkel,hollande,juncker vow would read like .

  7. Phil/Rob

    I certainly agree that we need a media-savvy leader.

  8. 07052015

    I don’t think there’s that much opportunity for polling suffering from Shy Brexit itus.

    Rather- it has become so fashionable (on both left and right) to [email protected] off the EU that I think the bigger polling issue will be to correctly measure the lead of the ‘Yes to staying in’ quiet middle ground majority.

  9. COLIN
    ‘Unemployment still coming down. Pay increases still going up.’

    I thought the latest report from the Institute of Personnel Managers showed that pay has stalled again. They said the main cause was there was no pressure on recruitment and pay in private sector is affected by the downward pressure on pay in the public sector. They also said there was a downturn in money spent on staff training and skills which is pushing productivity figures downward again.

  10. Phil Haines @Robin hood

    Indeed- “swing back” is not the same as “late swing” or “in the booth swing”. The latter two are a feature of almost all elections.

    There was very little evidence of “swing back”. More likely the Tories never really lost there smallish lead in the first place and a significant minority of voters decided literally in the last 24 hours.

    On Austerity-lite: having failed to accept that we DID spend too much in the period 2004-2007 we were always going to be in a tricky position trying to argue that it was ‘all the bankers fault’. I happen to think it was mostly their fault: but it was also ours as well.

    We did not apologise (other than for things like the Iraq war and not supporting Hamas) and refused for nearly 2 years to have even the outline of an economic policy.

    By the time we did it was too late to counter the simplistic charge that we were “the same”- patent nonsense that being.

    Two governments can both spend money and they can do it in different policy areas and to different levels. Ditto when you are cutting spending. Just because you are both spending, or both cutting, does not make you the ‘same’ !

    You seem to be suggesting we should have been proposing an economic policy based around taxing and spending; not cutting anything/ reducing the deficit?

    If so we would not have reached even 200 seats.

  11. Now we have a Conservative government I have been reading all policy documents, legislation and so forth to check what is/has been in the pipeline. No real surprises except I was astounded to learn that students whose parents are on universal credit (ie the bottom rung of earnings etc) will now have to pay £780 a year out of their grants in order to ‘reserve a room’ in the family home while living away. Given the cuts in scholarship funding this extra money amounts to over 10% of a poorer students grant. Looks like IDS is making some of his savings by passing on costs to his university minister colleague (if the money is to come from hardship funds). If the money cannot be found in the harship funds then there is a chance that this will deter potential students from poorer families.

    Am off to write to my MP on this one and ask for a response from the universities minister.

  12. Carfrew

    pressure going in one direction
    pressure going in the opposite direction
    net effect

    not complicated

  13. Phil Haines – is a good question.

    The polls in 2010 overestimated the Lib Dems, but got the Con & Lab balance about right.

    So a good question is when they started being wrong? (Or, if the problem with the polls last week is actually connected to the Lib Dem problem in 2010, when did it start affecting Lab & Con?). Were the Tories actually in a better position for years, or did we just miss some relatively recent movement?

    We can’t really tell at this point, the only obvious points to check are polling at the European elections which did tend to favour Labour a bit and underestimate Conservative a bit… but not as much, not as consistently, and not as many pollsters did any polling.

    Incidentally, for the purposes of Steve Fisher’s model he treated the change between polls at a point in time and eve-of-election-polls, and difference between eve-of-election-polls and actual result and two separate things and had both “swingback” and “polls normally underestimate the Tories” in his model. And, in hindsight, was obviously right to include both.

  14. ROB SHEFFIELD

    You need to realise that there were two things Labour needed to do.

    1: They needed to win the election.
    2: They needed to actually rescue the economy which means following mainstream economic theory rather than saloon bar economics.

    To be honest, even had Ed Miliband won, I doubt he could have rescued the economy. Brown was correct when he said it was a global economic crash, caused by global factors. By the same logic, a second-rank country like the UK is not going to buck those global factors alone.

    All the politicians of the UK could do after the crash was to kick the can down the road until the final reckoning. Cameron succeeded in doing that, which is why he is still PM. Who thought that interest rates would STILL be at 0.5%? Is that a sign of a healthy economy?

  15. @Mr Jones

    Unfortunately, you can’t seem to find much pressure in the opposite direction barring summat about foodbanks, to offset the relentless pressure re: immigration and welfare.

    And anyway your idea doesn’t work besides that ‘cos as I explained, the media don’t tend to manage such wholesale shifts as turning the majority to the left.

  16. Surely the EU referendum polling is going to be a much simpler matter than general election polling? There is a simple yes/no/don’t know yet scenario rather than the endless complications of churn and so forth within a complicated party political system, assigning of past votes and so forth.

    The main issue will be whether or not the problems with GE polling was because of a faulty sample base/faulty upweighting. If so this would have to be addressed even for the simplest of polls.

  17. ROB SHEFFIELD
    It is for people with a mortgage. For people who live in a council house in Healy or Walkley, with £400,000 in the bank not so much.

  18. HAWTHORN
    Who thought that interest rates would STILL be at 0.5%? Is that a sign of a healthy economy?

    No.

  19. I see Greece is back in recession and will be increasing taxes on Hotels. My dad has just paid our holiday so hopefully it wont affect us. I wonder how Greece being in recession will affect the EU negotiations through. Could easily delay them as If Greece collapses that is more of priority.

  20. THE MONK

    Having just seen a post by you I thought I’d offer my congratulations on your forecasting in the predictions game. I hope you saw the post with the results. :)

  21. BM11

    Greece is just one of the many unknowns about the EU referendum.

    The utter ineptitude of the EU’s economic policy (even worse than Osborne) along with the appalling treatment of Greece, even after they threw out the rascals who wrecked their country has certainly made me less pro-European than before.

    I would vote to stay in mind you, mainly for geo-political and geo-economic reasons.

  22. @ Catowsyn

    Sampling would still be an issue. So would be the turnout.

  23. HAWTHORN
    The utter ineptitude of the EU’s economic policy…. along with the appalling treatment of Greece, even after they threw out the rascals who wrecked their country has certainly made me less pro-European than before. I would vote to stay in mind you, mainly for geo-political and geo-economic reasons.
    _____________________________________________________

    That just about sums up my own current position. Still, there will be lots of arguments to hear for and against before its all over.

  24. Carfrew

    I know a lot of people on the left need to believe they’re the rebel alliance and not the establishment but the net effect proved the point pages ago.

  25. Carfrew

    “the media don’t tend to manage such wholesale shifts as turning the majority to the left”

    We’re talking about shy voters i.e. people who feel a pressure to conform to one side when they’re on the other.

  26. @Mr Jones

    “I know a lot of people on the left need to believe they’re the rebel alliance and not the establishment but the net effect proved the point pages ago.”

    ———

    I know some people would rather attempt teenage evasion strategies – and throw in a comically misdirected ad hominem for good measure – than admit to being wrong, but you didn’t prove the effect, you claimed it.

    I showed why it didn’t work, you have ignored all that and just continue to claim it like a broken record.

  27. CATOSWYN

    I was refering to the ONS report today.

  28. Rob Sheffield
    New Leader? Well put.

  29. @Mr Jones

    Here’s summat else for you to try and sidestep: the popularity of renationalisation: has there been some mass media cheerleading on that? Or do people make up their own minds based on experience?

    Did the Scots media tell everyone to vote SNP, or are Scots media more unionist?

  30. Carfrew

    The presence of shy voters shows which side has the dominant pressure.

    You’re presenting your subjective view of the two pressures (big on one side, minimal on the other) as an argument but that completely misses the point.

    1) There is no objective centre.
    2) Everyone has their own subjective position.

    So the only objective centre is the summation of those subjective positions.

  31. I wonder if any polling company will do polls of Tory, Ukip, SNP and LD voters as well as Non-voters to see what Labour Leader candidate appeals to them. It would be fascinating to see who does especially swing voters.

  32. Carfrew

    “the popularity of renationalisation”

    Are there people saying they are pro renationalisation who are secretly against it (or vice versa)?

    “Did the Scots media tell everyone to vote SNP”

    Are there people who publicly say they are pro SNP who secretly vote for someone else (or vice versa)?

    Because that is what the argument is about.

  33. @MRJONES

    “The presence of shy voters shows which side has the dominant pressure.”

    ————

    That’s what you claim. I pointed out what was wrong with it, you ignore all that and keep repeating the claim, yet again

    Shy voters as proof of dominance because:
    – media not as powerful as you claim
    – there are other reasons for shy voting

  34. ROB SHEFFIELD.
    Thank you for your post on how Labour lost the key public.

    If the Labour Party did not exist, would it be necessary to invent one now? I do not know what my answer is.

    Since I am a gloomy Celt, at heart, and in soul, I suspect I, at sixty years of age soon, may not see a Labour Government in my lifetime.

    I cannot see any future leader being any good, unless Tony Blair returns to Politics, as a member of the Labour Party.
    Just a couple of thoughts and off to the beach now!

  35. @Mr Jones

    “You’re presenting your subjective view of the two pressures (big on one side, minimal on the other) as an argument but that completely misses the point.”

    ———-

    Actually I am giving examples and posing questions to illustrate the subjectivity of your view. You think the media have had a net effect of bigging up welfare claimants because… summat about food banks!!

  36. Well, Labour didn’t have an economic policy in 1997 either. It seems to me that the consensus was restored in the 1991-92 recession.

    Labour spending on social services in the 2000s was always dangerous (even if necessary) – you are creating demand without supply, trying to make up 20 years of neglect, taking away resources from other parts of the economy. It was compounded by the need to remove the spending from the traditional way of presenting the UK budget because of the way it would be offered for public consumption.

    To my mind most of the 2000s was about spending the way out of the threat that showed its head in 2000-01, which eventually arrived in 2007-08. Was it reckless? Yes, but everybody were doing the same, and also there was a shared belief (independently of party affiliations that it would work). Could Labour apologise for this? Could, but it would have made an iota difference. The trouble is, the unless you completely broke with the dominant economic thinking, even without recklessness, the result would have been exactly the same. Countries without the banking crisis ((Canada, Denmark) eventually got to the same position.

    It is not the place to argue economic policies, but I believe it is about growth. Growth generated by the need to recover cost and not by stimulating demand. It is a kind of supply side stuff, but with severe limitations on M&A (yes, administratively), thus forcing business to invest or give up and release resources. I stopped on this.

    I found the argument between Labour and the Conservatives on the economy utterly boring in the last 5 years. EM brought in something which was meaningful: there are many people who need help now, and not when the economy grows, and painkillers for a severe injury or illness is not the best approach.

    Labour has never had a chance to win the economic argument without reframing. However, there was no way for the PLP to ever support such a reframing. And it won’t. So, unless the Conservatives make a mistake, or the economic cycle works against them, Labour will lose the economic argument again. Then they have to think what can counter-balance it.

  37. I thought I would compare the Thanet South council and parliamentary election results with all the excitement today.

    I went through all the wards in Thanet South constituency (14 from Thanet two from Dover according to Wikipedia)

    Because several seats were multi candidate, where a party had more than one candidate in that seat, I divided their results by the number of candidates.

    Basically the results and turnout were broadly in line with the general election result. C and UKIP down a few %, Labour up a couple of % in the council compared with the parliamentary.

    What was clear was that the Tory wards generally had higher turnout. What lost it for Farage IMHO is the two Dover wards which had >75% turnout and the Conservatives getting >50% of the vote in the combined wards, with UKIP coming 3rd behind Labour in one and not standing in the other.

    Turnout in Con and Con/UKIP marginal wards was as high as 80%. Turnout in Labour, UKIP or Lab/Ukip marginal wards was as low as 64%.

    In the Thanet South wards that are in Thanet District Council, UKIP were clear winners (roughly 33% UKIP 30% Conservative).

    My conclusions are that:

    *Labour and UKIP didn’t appear to get enough of their vote out.

    *There did not appear to be many Labour voters tactically voting Tory (as Labour got approx 2% higher vote in the council elections than in the parliamentary).

    *The two Dover Council wards were solid Tory and had 18% of the voters in the constituency. Given that, it was never going to be the easiest seat for UKIP to win. Farage probably should have gone for Thurrock.

    *The results and turnout appear to reflect the pattern in the rest of the UK.

    Usual Errors and omissions excepted disclaimer.

  38. James – “Quite interesting comments by ex Italian PM (and well connected Eurocrat) Mario Monti on Newsnight last night. Basically warning the Tories that if the UK leave the EU then Scottish independence is more likely.”

    Remember that Scotland elected a UKIP MEP when London didn’t! The results might not be what elites in Scotland expect…

    I’m not sure Mario Monti knows any more than anyone else how things will pan out.

    Europeans at the moment are clutching at all sorts of things in the hopes that it shows Britain is moving towards them.

    For example the Europe subreddit got very excited that the Danish Prime Minister’s husband got elected to the House of Commons. There was a whole thread on how it reflected the “new pan-european politics”! In reality I’m not sure the good people of Wales were aware they were electing the Danish Prime Minister’s husband. I think they thought they were going for someone associated strongly with Wales.

  39. Carfrew

    “I pointed out what was wrong with it”

    You pointed out the pressure you notice the most.

    However the net effect is you plus everyone else.

    .

    “media not as powerful as you claim”

    The existence of shy voters would actually imply the opposite as it’s people still voting the same way just lying about it.

  40. Paul

    Thanks for that.

  41. Carfrew

    “because… summat about food banks!!”

    You giving a list of the media pressure you subjectively notice most versus me giving a list of what I think other people may have subjectively noticed most is

    completely missing the point

    again.

  42. @Mr Jones

    Oh, so it was just me who noticed all the headlines about welfare etc., and the immigration vans etc!!

    And against that you have… your subjective food banks!!

    If you are going to prove your thesis about net media influence, you need to prove there really was a net media lefty bias.

    Just saying shy voters proves it doesn’t work because as I pointed out earlier, there can be other reasons besides media for shy voters.

  43. Candy,

    This doesn’t undermine your main point, but both Scotland and London elected 1 UKIP MEP in 2014.

  44. @Mr Jones

    “Are there people saying they are pro renationalisation who are secretly against it (or vice versa)?”

    ——–

    I see you don’t want to address my point, that people have acquired lefty views on renationalisation sans much media influence.

    As for your asking a question to change the subject, if you have evidence there are shy renationalisers, and that these were created by the media, then it would help your case.

    I doubt you do, in which case it’s a red herring.

    (Ditto the SNP example).

  45. I suppose I should add, significant numbers of shy renationalisers, in case you suddenly find you have a friend who fits the bill!!

  46. Carfrew

    This is going to go round in circles so this will be my last point.

    1. Do you accept that there is no objective centre?

    2. Do you accept therefore that the nominal centre for any particular population at any particular time can only be defined as the point where the number of people to the left of that point is equal to the number of people to the right?

  47. @Mr Jones

    It won’t keep going round in circles, I won’t let it.

    To deal with your final red herring: I agree there are difficulties defining the centre, and I sympathise since it may make it harder for you to prove your point.

    All you had to do to prove your thesis that net media bias created shy voters, was to prove the net media bias, and that only this bias could have produced these shy voters.

    You have yet to attempt this. And difficult to see how you can, hence the diversions.

  48. Carfrew

    Looks like I lied.

    “I agree there are difficulties defining the centre.”

    There is no objective centre.

  49. @Laszlo

    In 2020, Labour, the LDs and practically every other party in the HoC will be opposing the Tories. To me, this will be the fundamental difference with the 2015 GE.

    Regardless of specific polies, whether successes or mistakes, the Tories were able to present themselves this time around as collegiate pragmatists due to the Coalition. The usual referendum on the government was therefore a referendum on the Coalition and not the Tories. 2020 will be very different.

  50. Please excuse me if I have missed the explanation for the following:

    The LibDem share of the vote, at 8% or thereabouts, was broadly speaking, correctly predicted. Yet the number of seats they won was way less than everyone thought. Most were predicting 25-30.

    The assumption was that incumbency for LibDems was a much bigger factor than for other candidates. Was this the case, and if not, why not.

    Apologies if this has already been discussed at length.

    It just occurs to me that this might in some way hold the key to explaining the failure of the polls.

1 11 12 13 14