Next Friday is the public meeting of the British Polling Council inquiry into the failure of the polls at the 2015 election, at which point I expect we’ll get some insight into what the different polling companies are thinking, though probably not many firm conclusions yet.

In the meantime the British Election Study team have published some thoughts from Jon Mellon about what the BES data could tell us about why the polls were wrong. It doesn’t include any conclusions yet, but goes through a lot of the thought processes and ways of identifying what went wrong, which I suspect may reflect what many of the pollsters are doing behind closed doors.

As yet only the online BES data from during the campaign is available for download, but in time it will be joined by their online recontact survey after the election campaign, their face-to-face survey after the campaign and voter validation data for the people interviewed in the face-to-face survey. The article has some thoughts about what they can learn from the data that’s already available and what can be learnt from the bits that are still to come:

1) The BES campaign data appears to show some movement towards the Tories over the last couple of days, though not one that is beyond the margin of error. This is in contrast with YouGov’s daily polling data, despite them coming from the same panel. This is interesting, but as Jon says, the real proof will be when the BES publish their post-election data, showing if people actually did change their minds from their pre-election answers

2) If you only take people who said they were very likely to vote it would have been more Tory… but that’s very much a “Pope is Catholic” finding. The interesting bit here is what the BES team plan on doing in the future – they are once again going to validate their face-to-face data against the marked electoral register, to see if people who claim they voted genuinely did, and how well people’s stated intention to vote compares to whether they actually did. They are also going to match the online respondents to the electoral registers before and after the new electoral registers, to see if drop off from individual electoral registration was a factor.

3) Sampling and weighting. Jon hasn’t really said anything on the data so far – he’s waiting for the face-to-face probability sample, to compare that to the results from the online polling and see if it is significantly closer to the actual result.

4) Don’t knows. According to Jon the people who said don’t know before the election were a mixed bunch – their attitudes towards the leaders, issues and party id did not point to them being obviously likely to switch to Conservative or Labour. Again, the interesting bit will be to see how they said they ended up voting in the post-election wave.

5) “Shy tories”. Jon makes two interesting points. One is about question order. While the BES campaign data came from YouGov’s panel, its results seemed to show a movement towards the Tories that the main YouGov data didn’t show – in his article Jon presents Peter Kellner’s hypothesis that this may be because of question order. As regular readers will know, the published voting intention polls all religiously ask voting intention first, but the BES actually asks some questions about the most important issues facing the country and party leaders before asking VI. However, Jon also mentions what he judges to be “weak” evidence against “shy Tory” hypothesis – the BES included a grid of questions aimed at identifying people who tended to give socially desirable answers to questions, and Conservatives scored higher, not lower, amongst those people.


TNS Holyrood poll

TNS have released what I think is the first Scottish voting intention poll since the election. Prior to May Scottish polls concentrated on Westminster voting intentions (though many asked both Westminster and Holyrood), the focus now shifts over to Holyrood intentions ahead of next May’s election.

Voting intentions for Holyrood in the TNS poll are:
Constituency: CON 15%, LAB 19%, LDEM 3%, SNP 60%
Regional: CON 14%, LAB 19%, LDEM 5%, SNP 50%, GRN 10%

This is a big increase for the SNP on top of the 45% they received at the 2011 election where they won a majority in a electoral system designed to avoid them. These figures would only build upon it. Note also the dire state of Labour’s Scottish support, just four points ahead of the Conservatives, who are marginally up on their 2011 performance.

Full tables are here


-->

ComRes have released their first voting intention poll since the election, and have topline figures of CON 41%, LAB 29%, LDEM 8%, UKIP 10%, GRN 5%. Full details are here.

The ComRes poll also had the first attempt at a methodology change to address the failings of the polls at the 2015 general election – though as ComRes make clear in their explanation this is not ComRes’s final word on the topic, they are continuing their internal review and may make extra changes too.

As with all the pollsters who use political weighting, the initial change is to move from past vote weighting using the 2010 election to past vote weighting using the 2015 election, something that would have been done anyway. The second change is a new model of turnout weighting. This is based on the theory that a cause of the error was people overestimating their likelihood to vote in an uneven way – that is, we all know people overestimate their vote, but ComRes suggest they overestimate it unevenly, that people in some social groups (who happened to support Labour this time) overestimated their likelihood to vote more than other groups, thus skewing the polls.

In the past almost all the pollsters accounted for likelihood to vote using a straightforward system of asking people to rate their likelihood to vote on a scale of 0 to 10, and then either filtering out those people who gave a low score, weighting people according to how likely they said they were to vote, or a combination of the two. ComRes’s new method still filters out people who say they are less than 5/10 likely to vote, but after that bases likelihood to vote weighting on demographics, based upon patterns of turnout at the general election, specifically that there tends to be lower turnout in areas of social deprivation and in areas with a high proportion of social classes DE and low proportions of ABs.

The mechanics of this aren’t completely clear yet (I’ve asked ComRes for some more details which I’ll update later), but essentially it looks as if younger and more working class respondents are assumed to be less likely to vote than they claim they are and weighted downwards accordingly. It means, in effect, that the final headline voting intention figures are made up of 41% AB, 31% C1s, 19% C2, and just 9% DEs, so the effective sample once it’s modelled for the sort of people who actually turn out to vote is far more middle class than the pre-election samples that got it wrong.

The impact of the change is, as you might expect, to produce significantly more Conservative figures. In this particular poll it increased the Conservative lead from eight points to twelve points. In ComRes’s final pre-election poll it would have changed the result from a one point Tory lead to a five point Tory lead, significantly nearer what actually happened.

UPDATE: ComRes have got back to me with some more details of their turnout model. In my original version of this post I’d assumed ComRes were still weighting people according to their 0-10 score, but were adjusting this score based on demographics too. In fact ComRes are now only using the 0-10 score to filter out people who say they are less than 5/10 likely, otherwise the turnout weights are all based on demographics.


The Sun this morning have YouGov voting intention figures – their first since the election – of CON 41%, LAB 30%, LDEM 7%, UKIP 13%, GRN 4%. Note that in terms of methodology, the figures are past vote weighted to the election result for the time being, obviously the pollsters are all still looking into their methods in the light of the pre-election polling and there will be presumably be more changes once the different companies’ internal inquiries and the BPC external inquiry are complete.

The YouGov/Sun poll also had a question on people’s preferred Labour leader, currently Andy Burnham leads the other candidates, but is a mile behind “don’t know”. As was the case a couple of weeks ago, we’re really seeing a race between candidates who have extremely low public profiles, so the figures are pretty much just name recognition.

The YouGov/Sunday Times results from yesterday are here, and largely covered the issue of British perceptions of FIFA and corruption. There is a broad consensus amongst the British public that FIFA, the decisions on Russia and Qatar and Sepp Blatter himself are all corrupt (83% think FIFA corrupt, 78% the hosting decisions, 57% Blatter personally). A majority think the corruption is widespread throughout FIFA, and 46% now think the problem is so deep seated that FIFA is beyond reform and should be disbanded and replaced (a shift from a year ago, when people tended to think FIFA was corrupt but could be mended).

Looking forward people think the Russian World Cup should be cancelled and held elsewhere by 50% to 19%, and think the Qatar World Cup should by cancelled by 67% to 7%. 78% think that Blatter should stand down. There is, however, very little expectation that any of these things will happen – 73% think the Russian World Cup will go ahead, 53% that the Qatar World Cup will go ahead and 51% that Blatter will remain in office.

54% of people think that the England football team should boycott the World Cup if FIFA is not reformed, 18% of people disagree. This is not just people who don’t care about football – even amongst those who say they are interested in football 62% of people would support a boycott of the World Cup. This sounds a little high to me – after all, we’d just asked people lots of questions about what rotters FIFA and Blatter are, which probably disinclined them to say “let’s go along anyway and do nothing about it” but I expect we’ll see some more World Cup boycott questions in days to come.


Just a quick line to point out that the Constituency guide part of the site has now been updated to reflect the general election results and the new MPs elected, including the target and defence lists for the parties (SNP and UKIP to follow). Before anyone points it out there’s still lots to do – including new swingometers and updating MPs profiles to reflect the reshuffles.