Yesterday the British Polling Council had an event talking about how the polls had changed since 2015. This included collecting up data from all the companies on what they’ve done to correct the error and what they are now doing differently – all those summaries are collected here.

In looking at what’s changed it’s probably best to start with what actually went wrong and what problem the pollsters are trying to solve. As all readers will know, the polls in 2015 wrongly overstated Labour support and understated the Conservatives. The BPC/MRS inquiry under Pat Sturgis concluded this was down to unrepresentative samples.

Specially, it looked as if polls had too many younger people who were too engaged and too interested in politics. The effect of this was that while in reality there was a big difference between the high turnout among old people and the low turnout among young people, among the sort of people who took part in polls this gap was too small. In short, the sort of young people who took part in polls went out and voted Labour; the sort of young people who weren’t interested and stayed at home didn’t take part in polls either.

So, what have polling companies done to correct the problems? There is a summary for each individual company here.

There have been a wide variety of changes (including YouGov interlocking past vote & region, ICM changing how they reallocate don’t knows, ICM and ComRes now both doing only online polls during the campaign). However, the core changes seem to boil down to two approaches: some companies have focused on improving the sample itself, trying to include more people who aren’t interested in politics, who are less well educated and don’t usually vote. Other companies have focused on correcting the problems caused by less than representative samples, changing their turnout model so it is based more on demographics, and forcing it to more accurately reflect turnout patterns in the real world. Some companies have done a bit of both.

Changes to make samples less politically engaged…

  • ICM and YouGov have both added a weight by respondents level of interest or attention to politics, based upon the British Election Study probability survey. YouGov have also added weights by level of educational qualification.
  • Ipsos MORI haven’t added political interest weights directly, but have added education weights and newspaper readership weights, which correlate with political interest.
  • Kantar have added education weighting, and also weight down turnout to the level they project it to be as a way of reducing the overall level of political engagement in their sample.

Changes to base turnout on demographics…

  • ComRes have changed their turnout model, so it is based more on respondents’ demographics rather than how likely they claim they are to vote. The effect of this is essentially to downweight people who are younger and more working class on the assumption that the pattern of turnout that we’ve seen at past elections remains pretty steady. ICM have a method that seems very similar in its aim (I’m not sure of the technicalities) – weighting the data so that the pattern of turnout by age & social grade is the same as in 2015.
  • Kantar (TNS) have a turnout model that is partially based on respondents age (so again, assuming that younger people are less likely to vote) and partially on their self-reported likelihood.
  • ORB weight their data by education and age so that it matches not the electorate as a whole, but the profile of people who the 2015 British Election Study who actually voted (they also use the usual self-reported likelihood to vote weighting on top of this).
  • Opinium, MORI and YouGov still base their turnout models on people’s answers rather than their demographics, but they have all made changes. YouGov and MORI now weight down people who didn’t vote in the past, Opinium downweight people who say they will vote for a party but disapprove of its leader.
  • Panelbase and Survation haven’t currently made any radical changes since 2015, but Panelbase say they are considering using BES data to estimate likelihood to vote in their final poll (which sounds to me as if they are considering something along the lines of what ICM are doing with their turnout model)

In terms of actual outcomes, the pollsters who have adopted demographic turnout-models (ComRes, ICM and Kantar) tend to show larger Conservative leads than companies who have tried to address the problem only through sampling and weighting changes. We cannot really tell which is more likely to be right until June 8th. In short, for companies who have concentrated only on making samples more representative, the risk is that it hasn’t worked well enough, and that there are still too many of the sort of young engaged voters who are attracted to Jeremy Corbyn in their samples. For companies who have instead concentrated on demographic-based turnout models, the risk is that the pattern of turnout in 2017 differs from that in 2015, and that Jeremy Corbyn’s Labour really does manage to get more young people to come out to vote than Ed Miliband did. We will see what happens and, I expect, the industry will learn from whatever is seen to work this time round.


199 Responses to “How the polls have changed since 2015”

1 2 3 4
  1. After reading this thorough and logical inquest on the pollsters` mistakes during the 2015 GE, I wonder if the Tory planners have made a similar, though opposite, mistake in this year`s GE.

    Could their focus groups have been too old and out-of-touch with young folk`s thinking and methods?

    It is very strange that our leaflets and outdoor blue boards in NE Scotland have just one message, that Tory votes will prevent a second referendum. And this from a party that is often keen (too keen, IMO) on giving people choice.

    An abiding memory in my mind when we came to NE Scotland in the 1960s was that public schools had no playing fields. I soon found this was the policy of the shire Tories, who called themselves the Progressives.

    When we complained I was told that if we wanted that sort of luxury for our children then we should send them to private schools. Keeping down the rates was what “people” wanted.

    Nowadays I am sure that less than 1% of Tories would want schools with no sports or facilities, but could those nice people with genuine (and wrong, IMO) attitudes be mirrored two generations on in the 2017 Tory focus groups.

  2. Last time comres posted a poll and it showed that big conservative lead, they were completely bombarded and abused with utter filth by all those lovely progressive left wingers on social media. I really detest that response.

  3. Last Comres poll was 2 weeks ago. The lead will drop for certain, but the poll will be very interesting if Labour % > 35.

  4. Slippage in Tory lead.continues in latest Opinium

    Westminster voting intention:

    CON: 45% (-1)
    LAB: 35% (+2)
    LDEM: 7% (-1)
    UKIP: 5% (-)

    (via @OpiniumResearch / 23 – 25 May)

  5. It felt originally like a khaki election it still does in many ways but housing the threat of loosing you house through social care commitments or never been able to afford one is getting traction for once.I think as always it will be a comfortable majority for the comfortably off.

  6. Opinium poll

    CON 45

    LAB 35

    LIB 7

    UKIP 5

    https://twitter.com/britainelects/status/868505634332913664

  7. @Rich

    Just for once I’d love to read a post of yours that wasn’t negative or complaining

    ;-)

  8. Opinium poll has CON lead at10, 3 down from their last poll

  9. Cheers RAF

  10. Paul Croft

    I actually had an out-loud belly laugh at your “Lesser of Two Evils” posting. If either party was brave (or perhaps honest) to make that their election slogan, they might sweep the country – and deserve to do so. Just priceless!

    On the posters front, there is nothing here – other than some of the SNP ones, which appear to never be removed.

    Interestingly, at the Independence Referendum, it was virtually all “Yes” posters which could be seen – until the last day or two, when more”No” posters began to appear. No, went on to win, as you are aware. Any significance? I suspect people more worried about getting a brick thrown!

  11. Opinion is actually a good poll for Cons. There you go Catmanjeff!

    10 point lead with 12 days to go, it’s not exactly a disaster.

  12. @Matt Wardman,
    “I think Mr Corbyn needs to explain how you understand the strong and weak points of a gang of genocidal, slave-trading murderers.”

    It was clear from the quote you gave that Corbyn was suggesting that some people had defected from the Iraqi army to Isil and that such people might be weak links, i.e. not fully committed to Isil, and if their allegiance to Isil could be undermined through the establishment of stable government and multiparty dialogue then Isil would be weakened. He specifically explained that it wasn’t about dialogue with Isil. And more recently he’s been very clear on that score.

  13. I agree with several other posters who’ve commented they have found little first-hand evidence of anyone really giving a toss about this election. So far this Bank Holiday weekend, I’ve been to two barbecues; not mentioned once.

    Granted, on Facebook I see comments from a couple of left-leaning “echo chamber” friends moaning about foxes and suchlike. However, I’m predicting the reality of the polling booth will see a majority for CON but lower than expected.

    Corbyn is running a good campaign, or someone behind the scenes is. However, he’s largely playing to his existing galleries and offering kids free tuition, so of course they’re going lap that up.

    Ultimately the quiet sensible types of middle England will be faced with May/Hammond/Rudd or Corbyn/McDonnell/Abbott and I just can’t see the “soft belly” of CON being tempted by the latter.

  14. Apologies for repeating info already posted, hadn’t refreshed :/

  15. As the different pollsters have different methodologies, in some ways the direction of travel (the reduction of the Con lead) is more significant than the absolute figures, which can’t be safely compared to Yougov’s.

    But true, if Opinium is right and Con win the election with a 10pt lead, Con will be very happy with that. And maybe the Yougov will turn out to have been a bit of an outlier?

  16. Rich:

    About as good as the YouGov one was. Fieldwork carried out at the same time after all. We haven’t seen any effect of the terror speech yet (if any).

  17. I can’t see much traction in polls next week. It’s bank holiday week, kids off, nobody will care. (Unless we see a major gaff by somebody)

  18. Apparently the Opinium fieldwork predates Yougov, not that much can be read into that. Both postdate the bombing.

    35% is easily Labour’s best score with Opinium for a considerable time.

  19. Redpoll, your left leaning friends may be more representative on some issues than you realize.

    http://www.independent.co.uk/news/uk/politics/conservative-manifesto-bring-back-fox-hunting-only-one-in-10-voters-support-a7757891.html

    It is an odd move you must concede.

  20. Opinium should be giving Tories everywhere a minor sigh of relief. Had they released one earlier in the week, it may have caught a smaller lead, but this is fairly stable.

  21. SSSimon:

    I think you have that mixed up, the Opinium fieldwork predates YouGov’s which showed the higher lead.

    Of course, YouGov’s could easily be an outlier, that’s something else though.

  22. *smaller lead, sorry.

  23. @ Relick

    I meant before Manchester had an effect. YG showed an almost instant reversal of Labour fortunes after Monday. Opinium fieldwork came after that.

  24. Poll of poll averages continuing to move in labours direction but time is quickly running out

  25. SSSimon:

    Ah, my apologies then.

  26. Opinium also took approval ratings:

    T.May: 45/34 (+11%)
    J.Corbyn: 31/42 (-11%)

    Still firmly in TM’s favour.

  27. CON lead down 6 with ComRes
    Con 46 (-2), Lab 34 (+4), LD 8 (-2), UKIP 5 (-), SNP 4 (-), Grn 2 (-1), O 1

  28. @MARKW I agree it’s not popular, but my point really is that not enough people care THAT much in order to change their voting on the back of it.

  29. @ Rich “”I can’t see much traction in polls next week. It’s bank holiday week, kids off, nobody will care.””

    What a strange post- “nobody will care”.

    And how many areas have schools closed all week?

    In our region Monday is a normal working day with shops, schools and works operating as normal.
    .

  30. Comres has tended to be the least favourable poll for Labour – and is not that well regarded by quite a few psepho;ogical commentators.

  31. Those polls are huge Tory wins, whatever the Labour vote is doing.

  32. Really! oh I see not so good for Labour.

  33. A 10% or 12% lead with 12 days to go can only be good news for Tories surely?

    I am impressed with the (apparent0 optimism of those who describe these figures as “slippage in Tory lead” and “lead down by 6” but surely it is just that, optimism. “Tory lead rises by 5% or 7% will probably be the headlines.

  34. @RP

    Thanks for the reply. The former Iraqi Army officers are the senior military people who have been organising the ground troops carrying out what JC refers to as ‘War Crimes which must be addressed.’ He also needs to explain how he will persuade all if those to lay down their arms given his other pilicy is to remove the military pressure on them.

    I suspect that this may be moot given that history may have bypassed his former views.

  35. Woody – thankfully hardly any newspapers do that sort of these these days (glances meaningfully at the Mail on Sunday). The vast majority do know to only compare apples to apples and oranges to oranges.

  36. @Mike
    ‘Those polls are huge Tory wins, whatever the Labour vote is doing.’
    Depends what you have in mind! On the basis of a uniform swing a Tory lead of 12% implies 21 gains from Labour compared with 2015. A 10% Tory lead of 10% would suggest 14 gains from Labour. None of this makes any allowance for the first term incumbency likely to be enjoyes by 12 – 15 of the Labour MPs at risk – or of the fact that the GB Tory lead will be boosted by by their strong performance in Scotland. As a result, the swing in England & Wales will be somewhat lower than the headline GB figures.

  37. I think the Tories will be pleased with those polls, steadied the ship somewhat.

  38. So Graham what you are actually saying is that the 10%-12% lead for the Tories is in fact great news for Labour?

  39. REGGIESIDE @ WOODY
    oh agree – whatever happens on Any Questions is very small beer.

    In “normal” times, of course, but this time JD’s brother DD will be the master of ceremonies at May’s next public grilling – this time in front of an audience.

    I have a feeling that the AQ incident well be one of the questions chosen.

  40. Blue Bob
    I am saying that the implied majority for the Tories from both polls is quite modest – particularly in relation to seats gained from Labour. I suspect that Labour will not be too unhappy with a 12 % Tory lead from Comres.

  41. On who ‘have the best policies for people like me and my family’:

    May and the Tories: 37%
    Corbyn and Labour: 42%

    (via @ComRes)

    Shy Labour – tell the pollsters what you think you ought to vote – strong and stable BUT in the polling booth vote what’s best for you & your family.

    Compounded by over compensation for GE17

  42. Hi All,

    Not posted on here since the referendum campaign.

    A few insights from a Labour partizan –

    I read an interesting piece on Political Betting I believe, saying that the UKIP to Tory flow might be quite varied across the country.

    I am in the North West, and have spent time on the doorstep and phone banking in a lot of seats in Cheshire and Greater Manchester.

    I would say in my experience, a lot more 2015 UKIP will ‘come back’ to Labour than the headline predictions of 4-1 Tory to Labour conversions. The Tory brand is still quite toxic to many in the north west despite swapping to UKIP.

    I also see more signs of a young voter surge than in 2015. None of this means that I think Labour can win – it seems the events in Manchester have taken a bit of wind out of our sails in the polls – wouldn’t say it seems so on the doors yet but to early to tell.

    The election does seem to be shaping up for an interesting finish, I would expect Labour to make the odd gain and to outperform in defence, but barring May coming unstuck in the QT show or some other revelation, I would expect her to limp home with a slightly increased, if not particularly strong or stable majority.

  43. Hi Anthony

    The numbers may not be the actual headline but it will be heavily reported in the text and the headline will be all about the actual lead not the change in number

  44. ANDREW111 @ BZ
    The Lib Dems will I think be hoping that if … Farron manages to pass the Andrew Neil and other tests … that they can get some Tory Remainers into their camp who will never vote Labour

    That’s almost certainly true, I think. There were probably a few who decided to do just that as soon as the GE was agreed, but some may well hold their noses and move on to Lab if their polling looks good.

  45. I’m sorry, but I laughed heartily at this:

    https://www.express.co.uk/news/uk/810062/Welsh-voters-Labour-Corbyn-May-Brexit

    It includes: ‘John Francis, a member of Navy, was a life-long Labour voter until Brexit and Theresa May came along. He told the BBC: “I will go Conservative because of Theresa May – Genghis Khan in Number 10. We will need her when the negotiations start and she will give them the shock of their lives. When the EU blokes come to the table she will tear them to shreds.”‘

    Ok, partisan stuff as expected. But in other news on the page:

    “Big-eared bomber Abedi ‘rejected by women for YEARS’ before bombing Manchester Arena”

    Nice to see Britain still has quality journalism. Are there voters that take this stuff seriously?

  46. If people really want to educate themselves on the history of the Middle East and what has created the forces existing today, you could start with this:

    https://en.wikipedia.org/wiki/Sykes%E2%80%93Picot_Agreement

    Other areas to look at should allow you answer these questions:

    * What is Sunni and Shi’a?
    * Where do you find each?
    * How do these relate to the ME?
    * We know Russia and China are involved. Why and how?
    * Where does Libya fit in?
    * IS, ISIS, ISIL and Daesh: what’s the difference?
    * They say military action is reducing IS (etc). Really? Is that the end of them?

    And just to add to the fun:

    * What role do the Turks and Kurds play?

    Understanding the answers to these questions will not answer everything, but they will help to build a reasonable background of this complicated area.

    Of course it is easier to simply say, bomb the bar stewards! That solves everything.

  47. Regarding the southwest The voodoo opinion poll shows more going for Labour than Lib dems in Southwest. Only Lib Dem areas are in Remain areas in South England. Suggests the young not going to Lib Dmes in SW

    https://ge2017.gritdigital.co.uk/#2017-opinion-poll

  48. re the debate

    Teressa May. not a car crash – but she’s not really a people person is she? Waffled too much and had some sticky moments but Paxman allowed her to finish strongly grandstanding on brexit.

    Jeremy Corbyn. relaxed, warm, good humoured. Bit evasive on the stickier questions. Turned the tables on paxman very effectively.

    paxman – terrible. annoying. rubbish questions. pointless interruptions. thought he was noticeably softer on TM than JC. career ending?

    Overall effect. marginal. Probably better for corbyn in that people may have warmed to him – but the ardent brexiters will have liked Teresa more.

  49. @Reggieside

    I tend to agree – I thought Corbyn had the better of the audience sessions – May’s cocooning from exposure to the public en masse in the campaign can’t have helped her to adjust to the set up. With Paxman I thought it was a game of two halves, Corbyn doing well in the first half and poorly in the second and vice versa for May.

    I doubt if it will have changed many minds except for those who like the warm and cuddly bits with the audience by Corbyn or found confirmation in his edgy end with Paxman

1 2 3 4