Last year the election polls got it wrong. Since then most pollsters have made only minor interim changes – ComRes, BMG and YouGov have conducted the biggest overhauls, many others have made only tweaks, and all the companies have said they are continuing to look at further potential changes in the light of the polling review. In light of that I’ve seen many people assume that until changes are complete many polls probably still overestimate Labour support. While on the face of it that makes sense, I’m not sure it’s true.
The reason the polls were wrong in 2015 seems to be the samples were wrong. That’s sometimes crudely described as samples including too many Labour voters and too few Conservative voters. This is correct in one sense, but is perhaps describing the symptom rather than the cause. The truth is, as ever, rather more complicated. Since the polls got it wrong back in 1992 almost all the pollsters have weighted their samples politically (using how people voted at the last election) to try and ensure they don’t contain too many Labour people or too few Conservative people. Up until 2015 this broadly worked.
The pre-election polls were weighted to contain the correct number of people who voted Labour in 2010 and voted Conservative in 2010. The 2015 polls accurately reflected the political make up of Britain in terms how people voted at the previous election, what it got wrong it how they voted at the forthcoming election. Logically, therefore, what the polls got wrong was not the people who stuck with the same party, but the proportions of people who changed their vote between the 2010 and 2015 elections. There were too many people who said they’d vote Labour in 2015 but didn’t in 2010, too many people who voted Tory in 2010 but said they wouldn’t in 2015, and so on.
The reason for this is up for debate. My view is that it’s due to poll samples containing people who are too interested in politics, other evidence has suggested it is people who are too easy to reach (these two explanations could easily be the same thing!). The point of this post isn’t to have that debate, it’s to ask what it tells us about how accurate the polls are now.
The day after an election how you voted at the previous election is an extremely strong predictor of how you’d vote in an election the next day. If you voted Conservative on Thursday, you’d probably do so again on Friday given the chance. Over time events happen and people change their minds and their voting intention; how you voted last time becomes a weaker and weaker predictor. You also get five years of deaths and five years of new voters entering the electorate, who may or may not vote.
Political weighting is the reason why the polls in Summer 2015 all suddenly showed solid Conservative leads when the same polls had shown the parties neck-and-neck a few months earlier, it was just the switch to weighting to May 2015 recalled vote**. In the last Parliament, polls were probably also pretty much right early in the Parliament when people’s 2010 vote correlated well with their current support, but as the Lib Dems collapsed and UKIP rose, scattering and taking support from different parties and in different proportions polls must have gradually become less accurate, ending with the faulty polls of May 2015.
What does it tell us about the polls now? Well, it means while many polling companies haven’t made huge changes since the election yet, current polls are probably pretty accurate in terms of party support, simply because it is early in the Parliament and party support does not appear to have changed vastly since the election. At this point in time, weighting samples by how people voted in 2015 will probably be enough to produce samples that are pretty representative of the British public.
Equally, it doesn’t automatically follow that we will see the Conservative party surge into a bigger lead as polling companies do make changes, though it does largely depend on the approach different pollsters take (methodology changes to sampling may not make much difference until there are changes in party support, methodology changes to turnout filters or weighting may make a more immediate change).
Hopefully it means that polls will be broadly accurate for the party political elections in May, the Scottish Parliament, Welsh Assembly and London Mayoral elections (people obviously can and do vote differently in those elections to Westminster elections, but there will be a strong correlation to how they voted just a year before). The EU referendum is more of a challenge given it doesn’t correlate so closely to general election voting and will rely upon how well pollsters’ samples represent the British electorate. As the Parliament rolls on, we will obviously have to hope that the changes the pollsters do end up making keep polls accurate all the way through.
(**The only company that doesn’t weight politically is Ipsos MORI. Quite how MORI’s polls shifted from neck-and-neck in May 2015 to Tory leads afterwards I do not know. They have made only a relatively minor methodological change in their turnout filter. Looking at the data tables, it appears to be something to do with the sampling – ICM, ComRes and MORI all sample by dialing random telephone numbers, but the raw data they get before weighting it is strikingly different. Looking at the average across the last six surveys the raw samples that ComRes and ICM get before they weight their data has an equal number of people saying they voted Labour in 2015 and saying they voted Tory in 2015. MORI’s raw data has four percent more people saying they’d voted Conservative than saying they’d voted Labour, so a much less skewed raw sample. Perhaps MORI have done something clever with their quotas or their script, but it’s clearly working.)