Yesterday the British Polling Council had an event talking about how the polls had changed since 2015. This included collecting up data from all the companies on what they’ve done to correct the error and what they are now doing differently – all those summaries are collected here.

In looking at what’s changed it’s probably best to start with what actually went wrong and what problem the pollsters are trying to solve. As all readers will know, the polls in 2015 wrongly overstated Labour support and understated the Conservatives. The BPC/MRS inquiry under Pat Sturgis concluded this was down to unrepresentative samples.

Specially, it looked as if polls had too many younger people who were too engaged and too interested in politics. The effect of this was that while in reality there was a big difference between the high turnout among old people and the low turnout among young people, among the sort of people who took part in polls this gap was too small. In short, the sort of young people who took part in polls went out and voted Labour; the sort of young people who weren’t interested and stayed at home didn’t take part in polls either.

So, what have polling companies done to correct the problems? There is a summary for each individual company here.

There have been a wide variety of changes (including YouGov interlocking past vote & region, ICM changing how they reallocate don’t knows, ICM and ComRes now both doing only online polls during the campaign). However, the core changes seem to boil down to two approaches: some companies have focused on improving the sample itself, trying to include more people who aren’t interested in politics, who are less well educated and don’t usually vote. Other companies have focused on correcting the problems caused by less than representative samples, changing their turnout model so it is based more on demographics, and forcing it to more accurately reflect turnout patterns in the real world. Some companies have done a bit of both.

Changes to make samples less politically engaged…

  • ICM and YouGov have both added a weight by respondents level of interest or attention to politics, based upon the British Election Study probability survey. YouGov have also added weights by level of educational qualification.
  • Ipsos MORI haven’t added political interest weights directly, but have added education weights and newspaper readership weights, which correlate with political interest.
  • Kantar have added education weighting, and also weight down turnout to the level they project it to be as a way of reducing the overall level of political engagement in their sample.

Changes to base turnout on demographics…

  • ComRes have changed their turnout model, so it is based more on respondents’ demographics rather than how likely they claim they are to vote. The effect of this is essentially to downweight people who are younger and more working class on the assumption that the pattern of turnout that we’ve seen at past elections remains pretty steady. ICM have a method that seems very similar in its aim (I’m not sure of the technicalities) – weighting the data so that the pattern of turnout by age & social grade is the same as in 2015.
  • Kantar (TNS) have a turnout model that is partially based on respondents age (so again, assuming that younger people are less likely to vote) and partially on their self-reported likelihood.
  • ORB weight their data by education and age so that it matches not the electorate as a whole, but the profile of people who the 2015 British Election Study who actually voted (they also use the usual self-reported likelihood to vote weighting on top of this).
  • Opinium, MORI and YouGov still base their turnout models on people’s answers rather than their demographics, but they have all made changes. YouGov and MORI now weight down people who didn’t vote in the past, Opinium downweight people who say they will vote for a party but disapprove of its leader.
  • Panelbase and Survation haven’t currently made any radical changes since 2015, but Panelbase say they are considering using BES data to estimate likelihood to vote in their final poll (which sounds to me as if they are considering something along the lines of what ICM are doing with their turnout model)

In terms of actual outcomes, the pollsters who have adopted demographic turnout-models (ComRes, ICM and Kantar) tend to show larger Conservative leads than companies who have tried to address the problem only through sampling and weighting changes. We cannot really tell which is more likely to be right until June 8th. In short, for companies who have concentrated only on making samples more representative, the risk is that it hasn’t worked well enough, and that there are still too many of the sort of young engaged voters who are attracted to Jeremy Corbyn in their samples. For companies who have instead concentrated on demographic-based turnout models, the risk is that the pattern of turnout in 2017 differs from that in 2015, and that Jeremy Corbyn’s Labour really does manage to get more young people to come out to vote than Ed Miliband did. We will see what happens and, I expect, the industry will learn from whatever is seen to work this time round.

199 Responses to “How the polls have changed since 2015”

1 2 3 4
  1. @Relick,

    I completely disagree. World trade centre first attack was 1993, a long time before the foreign intervention you talk about.

  2. Laszlo,

    They still have a few days left to apply for someone else to vote on their behalf. The application deadline for proxy votes is 5pm on 31st May.

  3. Our education system must be gravely flawed if people think foreign intervention in the ME only goes back to 9/11

  4. The other key factor that could relate to turnout on June 8th is about security. The threat level has been reduced (?) to severe but if there is anything in the news about safety issues at polling stations will it impact on who will vote and who will stay away ?

  5. And another point. Even moderate mosques now often have leaders who say the real enemy of Islam is liberalism and even liberal Islam. It’s been alleged this was the situation at Didsbury Mosque. The liberal left need to understand that some immigration, integration etc does eventually lead to some people who ironicly despise their liberal values. There is really no debate on this point? Or is there? Any views?

  6. @Ian,

    I never thought of that, good point.


  7. Laszlo, some greens are going red perhaps? We have many greens in Bristol East but less posters, and more lab than last time.

    I think the greens have asked supporters to vote tactically for Labour in some areas, but sorry i cant recall where.

    As for the young, well this weekend we have a festival in the park near me, Love Saves The Day, and sprayed up on the security fence in a few different hands, ‘ Tories out ‘ . It really is the eighties again ;-)

    The motorway type sign near the entrance where people, young people are queuing is saying ‘ expect thorough bag searches ‘ and ‘ dont forget your vote counts on election day ‘ .

  8. Mike:

    That’s what I meant to say, sorry if it came across like it is deliberate government policy to create terrorism.

  9. Danny

    “just posted on the last thread that I am listening to R4 any questions, where David Davis just tried to deny any link between foreign policy and the likelihood of terrorist action here”

    Was listening to Maajid Nawaz on LBC at the same time who described Corbyn’s linking events to international terrorism as naive. I know who I would trust more on this subject.

  10. “It really is the eighties again ;-)”

    Very possibly, and like the eighties, the Tories will probably get in again.

  11. Rich:

    You didn’t bother to read what I said, so I won’t bother to respond to what you said about the 1993 attack.

    With regards to your views about people hating us despite integration, I concur that does appear to be the case amongst a small minority of immigrants. The only ways that I see to really prevent this, however, is to either heavily restrict the religious and political freedoms of the populace or to ban immigration. You don’t have to be “liberal left” to strongly disagree with either of these options. If you have a much less contentious solution I’m welcome to hear it.

  12. whoops time to go out in the garden that should have read

    Was listening to Maajid Nawaz on LBC at the same time who described Corbyn’s linking events to foreign policy as naive. I know who I would trust more on this subject.

  13. MarkW

    My point about the Greens was specifically about they youth. I don’t think (ok, it’s an opinion) that they vote tactically that much. I could be very wrong, of course. I was just looking for a proxy measure to gauge the likelihood of youth vote turnout.

    As to posters, here in Liverpool Riverside, the only posters are Labour (there use to be some Green and LibDem). The only GE election leaflet came from the sitting Labour MP.

  14. Lols, Indeed, but I think the application of Popperian reasoning to election outcomes is hugely over baked.

  15. RP

    Thanks, I let them know.

    It looks like Labour had done their preparation well for dealing with the Tory response to Corbyn’s foreign policy speech by briefing the media. It’s unusual to see Fallon who is normally silkily imperturbable so thrown.

    Thanks for the heads-up, and surprising that he couldn’t manage to say something like “Boris will be Boris”. It now seems to be on most non-Con press sites.

    I’d be particularly interested in NEIL A’s opinion of his rejection of more police resources being a good idea.

    The full 5 minute interview is on C4 here.

  17. @MarkW re. Greens going red *anecdotal alert*: the Greens have stood down in Wells hoping to boost Lib Dem Tessa Munt’s chances of ousting the Conservative. Some of you might recall she resigned from her secretarial job for Vince Cable in protest over fracking. From what I’ve seen this has instead led to a significant number of Greens deciding to switch to Labour, who don’t realistically have a hope there (in fact Labour voters would more likely need to join in voting Lib Dem for the Greens’ plan to work out). In yet more evidence of how switched-on the public can fail to be, I’ve seen several local voters despairing that they fear voting for the Lib Dem candidate is a vote for fracking, despite her principled stance over it the last time she was in Parliament.

  18. Relick

    No, I didn’t think you meant it was deliberate policy to create terrorism, I thought you meant that our foreign policy created ISIS – it was that I was disagreeing with, as I don’t think our foreign policy created ISIS, but it was a factor in allowing it to thrive.

    Meanwhile, we are where we are and, although Corbyn’s plan to meddle less in the middle east might have long term benefits, in the short term we have a security problem that needs to be dealt with. As Jezza himself bragged “I’ve been involved in opposing anti-terror legislation ever since I first went into Parliament in 1983”, I wonder how he plans to monitor the 3,000 persons-of-interest currently roaming our streets?

  19. Laszlo, I think there are sound indicators that this election will see re engagement by those under 40 or 50. I would be very surprised if they are all wrong.

    It started with the membership surge for lab pre jc post last ge when many young people joined and that apppears to me to be continuing.

    Clearly the anti lab / jc feeling may motivate the older groups to get out more but it is smaller and overfished pool.

    It is also not certain as you say that the young will surely ‘ go left ‘ as some expect.

  20. Anyone who thinks posters in windows is any indicator whatsover of who will win the election is, quite frankly, nuts.

  21. I didnt say that.

  22. I can confirm that UKIP in my area of West Yorkshire are advising supporters to vote Labour where there is no UKIP candidate. There’re saying the Tories are will capitulate to big business and its globalist agenda in Brexit negotiations and can’t be trusted with a big majority. Strange times.

  23. Some people who put up posters don’t even vote I am sure. Putting one up can keep others away. Is complicated.

  24. MarkW

    No, I didn’t say you had. I meant that any discussion of posters in windows was irrelevent. It’s meaningless. I haven’t seen a conservative poster in London in years.

  25. Though having said that, I don’t think anyone’s voted Conservative in my London ward since about 1963

  26. Mike:

    Oh, I see. I don’t really see the distinction between ISIL existing and ISIL thriving to be that great so I didn’t quite understand your point at first. If ISIL existed but there wasn’t widespread instability in the ME as a result of foreign policy it would be quickly put down by the region’s governments IMO.

    Therefore I’m happy to accept your amendment that we contributed to it thriving rather than contributed to creating it. The thrust of the argument remains the same.

  27. stott
    Anecdote alert
    Sounds odd. I wouldn’t think that “globalist” is a word often used by UKIP. Sounds a bit.. corbinista.

  28. @relick,

    I agree it’s a small proportion, but it’s not a lone wolf, it’s a small yet active strain that isn’t helpful towards integration. I know it’s very tricky. Am guessing that is what Prevent was for, but clearly something else may have to happen if we get more of these attacks (or even attempted attacks as a the security services often allude to).

  29. I don’t think it is totally meaningless when you do a dog walk comparison to last time in the same streets.

    Like the one green in our street last time going red, but I am not trying to replace AW just batting stuff about cos thats what i saw and others may be interested.

    My poster sampling methods are reliant on where the dog wants to go to do her business.

  30. Stott/Thomas:

    It makes sense to me that UKIPpers would be opposed to globalism. They’re nationalist and have had a fair few protectionist policies after all.

  31. “globalist” is more of a far right thing – its kind of code for “new world order” “secret jewish cabal” “lizards”.

    I got a UKIP leaflet through my door – it concentrates almost exclusively on attacking May and the tories. The tories may be worried that the Manchester atrocity might provide a UKIP poll bost at their expense.

  32. MarkW

    I’m entirely convinced dog walking counts as a legit method of polling, but I am prepared to concede that it may be no less accurate than the opinion polls were in 2015…

  33. Sorry, that should be NOT entirely convinced. Doh.

  34. Mark:

    Pretty interesting to hear that people motivated enough to put up posters actually change their minds. I think if I were to show off to the world who I was voting for then I’d have to be damn sure about it first.

    I agree that it isn’t /totally/ meaningless if you are comparing week on week. But still, overall about as accurate as online voodoo polls IMO.

  35. Mike, lol.

  36. Anecdotes

    A drunk in our park constantly shouts out that corbyn is a C***. I have considered his view and in the light of it i advise posters to up their forecast of a tory majority

    My auntie’s best friends cousin has dementia. My auntie told me that the brother of her designated care assistant had told her that she had asked for her postal vote supporting the tories not to be sent. She had said that there was no way Mrs Pankhurst was getting her vote.

  37. On posters -in 2015 there were many tory posters in brighton kempton -I have yet to see one in 2017..

    Fewer labour than 2015 but still quite a few.

    Suggests turnout down and without greens and ukip the result anyones guess.Labour candidate is corbynista ,tory was sacked by May as brexit minister for the city after complaints from bankers that he didnt know his brief.

  38. As we’re into anecdotal territory here…. I spoke to my elderly neighbours just after the Tory manifesto was released and they were outraged. As Conservative voters, they felt betrayed. One said he was going to vote Labour, or not vote at all, the other said she was considering voting Lib Dem this time to teach the Cons a lesson.

    I spoke to them again this morning and asked who they thought they’d vote for if the election was tomorrow and they both said Conservative.

  39. Relick, yes the greens to red people I do know them and they are interesting. From a farming family but now townies. The houses that amuse me the most are the multiple party endorsements.

  40. The first World Trade Centre attack came 3 years after a quarter of a million Western forces came to the Middle East following the invasion of Kuwait by Saddam Husain in August 1990. Whatever the rights and wrongs of this intervention, the fact is that many thousands were killed an injured by Western forces, many of them civilians.

  41. S Thomas

    I shared a train journey with an old fellow who kept shouting “all Tories are bastards” much to the amusement of the carriage. When someone leant over to congratulate him and ask him if he was going to vote Labour, the old man said “only if they get rid of all the wogs in my street”.

  42. MarkW

    In our household we have solved that problem by not putting up posters but I guess there are many households with differing political afflictions

  43. mike

    Thank you for sharing. I love these anecdotes. I think posters will thank us both for our efforts and will find them equally useful in assessing the mood of the nation

  44. @MARKW

    Quite. I remember being in Cambridge just before the eu ref. I’d say there were at least 15 remain posters for every leave. Yet Cambridge voted 26% leave.

  45. @Mike:

    What you raise is one half of the questions that will decide the election. Firstly, have so many people actually changed their mind – or is it just a blip before they revert to their default position? Secondly, how many don’t knows/don’t cares are going to vote for Corbyn?

  46. Mike

    Of all the amusing anecdotal evidence produced I think that yours re the elderly neighbours is the most insightful.

    It is why polls, although wonderful, are deeply flawed in my opinion and can only give what they claim to give, a snapshot of today’s thinking which may well be different to tomorrow’s and yesterday’s. The trends they show are very useful and I would subscribe to the theory that the best indicators to a GE outcome are polls taken shortly before the GE is announced, hence my belief that the outcome will be an absolutely thumping Tory majority.

  47. S Thomas

    Indeed. And, taking into account your tramp and auntie’s best friend, and my old man on a train, it looks like mixed news for Conservatives and Labour, and very bad news for the Women’s Equality Party.

  48. Charles.

    Nice to hear from you.

    The straight answer is that I never listen to political stuff – it really does my head in, as we used to say in the 60’s.

    So I was going by the written reports, perhaps unfairly to ole Corby. It does seem to me though than, when offering up the [in my view] valid reasons by western intervention has not helped, it really is incumbent on someone in his position to be utterly clear that there is nothing at all to negotiate with a group such as isil.

    I also think that blaming the West doesn’t always seem to consider the cries for help, aimed directly at us, from those suffering in the ME. Libya and Syria are just recent examples of places from which I watched so many people who were crying desperately into camera:

    “Where is the West? Where is the UN?”

    The latter is another example of the was that some elevate an organisation to almost mystical powers of wisdom when, in reality, it’s decision making is at the mercy of Russia and China and their own, narrow interests.

  49. I wonder if this election might be a re-run of 1992 in terms of vote share for the two main parties, although this time delivering far more parliamentary seats, and a bigger majority, for the Tories. I remember from my yime as a regular on here how we all solemnly bid farewell to the days when the two major parties would share 80%+ of the vote and how, with the splintering of votes amongst a myriad of smaller parties (LD, UKIP, Greens, SNP, PC), something like 65% shared between Tory and Labour was how it was always going to be.

    Well, I wonder if we may be reverting to a more bi-polar world again where the smaller parties cede votes to the Big Two and we get many more straight Labour v Tory fights. In my view, this election has always been in the bag for the Tories due to three factors; Corbyn’s leadership, UKIP voters re-ratting to the Tories post referendum and the impending Brexit negotiations. The fact that they aren’t completely out of sight is down to the awfulness of their campaign and the slow but sure wilting of May under any sort of scrutiny and political pressure. This has allowed Corbyn to edge back into the fight and, giving credit where credit is due, he has earned the right by some solid campaigning. His strategy team are starting to find their feet too and are wrong-footing the Tories surprisingly often now.

    Hence my feeling that this looks like a 42-35 vote share win for the Tories, this time garnering them a 60-80 seat majority. I can’t see much tactical voting going on, or a strong Lib Dem showing, which helped Kinnock trim Majors’s majority in 1992. I think if the Tories get 42% they’ll have won far more seats than in 1992.

    All that said, I’ve been staggered at how poor this Tory campaign is and how mediocre May has performed and I haven’t entirely ruled out something rather surprising occurring over the next 12 days.

  50. I am surprised Labour hasn’t returned to ‘Dementia Tax’/social care issue…

1 2 3 4