Next Friday is the public meeting of the British Polling Council inquiry into the failure of the polls at the 2015 election, at which point I expect we’ll get some insight into what the different polling companies are thinking, though probably not many firm conclusions yet.

In the meantime the British Election Study team have published some thoughts from Jon Mellon about what the BES data could tell us about why the polls were wrong. It doesn’t include any conclusions yet, but goes through a lot of the thought processes and ways of identifying what went wrong, which I suspect may reflect what many of the pollsters are doing behind closed doors.

As yet only the online BES data from during the campaign is available for download, but in time it will be joined by their online recontact survey after the election campaign, their face-to-face survey after the campaign and voter validation data for the people interviewed in the face-to-face survey. The article has some thoughts about what they can learn from the data that’s already available and what can be learnt from the bits that are still to come:

1) The BES campaign data appears to show some movement towards the Tories over the last couple of days, though not one that is beyond the margin of error. This is in contrast with YouGov’s daily polling data, despite them coming from the same panel. This is interesting, but as Jon says, the real proof will be when the BES publish their post-election data, showing if people actually did change their minds from their pre-election answers

2) If you only take people who said they were very likely to vote it would have been more Tory… but that’s very much a “Pope is Catholic” finding. The interesting bit here is what the BES team plan on doing in the future – they are once again going to validate their face-to-face data against the marked electoral register, to see if people who claim they voted genuinely did, and how well people’s stated intention to vote compares to whether they actually did. They are also going to match the online respondents to the electoral registers before and after the new electoral registers, to see if drop off from individual electoral registration was a factor.

3) Sampling and weighting. Jon hasn’t really said anything on the data so far – he’s waiting for the face-to-face probability sample, to compare that to the results from the online polling and see if it is significantly closer to the actual result.

4) Don’t knows. According to Jon the people who said don’t know before the election were a mixed bunch – their attitudes towards the leaders, issues and party id did not point to them being obviously likely to switch to Conservative or Labour. Again, the interesting bit will be to see how they said they ended up voting in the post-election wave.

5) “Shy tories”. Jon makes two interesting points. One is about question order. While the BES campaign data came from YouGov’s panel, its results seemed to show a movement towards the Tories that the main YouGov data didn’t show – in his article Jon presents Peter Kellner’s hypothesis that this may be because of question order. As regular readers will know, the published voting intention polls all religiously ask voting intention first, but the BES actually asks some questions about the most important issues facing the country and party leaders before asking VI. However, Jon also mentions what he judges to be “weak” evidence against “shy Tory” hypothesis – the BES included a grid of questions aimed at identifying people who tended to give socially desirable answers to questions, and Conservatives scored higher, not lower, amongst those people.


151 Responses to “The British Election Study on why the polls were wrong”

1 2 3 4
  1. @phil haines and Millie

    Parris seems to believe that Governments can do no more than hold the ring. If one thinks about it, Government or Government funded universities have made most of the major scientific breakthroughs and these relate to such things as the Internet and satellite technology as well DNA and the Higgs Boson. Similarly Government educates our children, funds or subsidises our roads and transport systems, and provides the police without whom we would no doubt be a failed state. Capitalism depends on the state not only for its ethical regulation but for its efficiency, Why isn’t Labour making these kind of arguments and concentrating on showing how they can work in those authorities where it is still in control?

  2. I am interested in why Labour lost and have looked at what predicts an increase or decrease in their vote share since 2010. To do this I used the 614 Non-Irish constituencies in which UKIP stood . I assumed that the 2010 Labour share of the vote would be a strong predictor of the 2015 result and that stable social factors such as the proportion of the population living in council housing would add little to this. I did, however, think that changes since 2010 – the rise of UKIP, the collapse of the Libdems, changes in regions and countries, notably Scotland and the South West would all have an effect. I also suspected that David Cameron’s cosseting of the old and the continuing disaffection with politicians of all kinds might be influential.
    The resulting equation was:

    Labour share of vote= 37 + .91 *Labour share of vote in 2010 –(.31*turnout + .21 proportion over 6o + 21.97 if Scottish constituency+.39 *UKIP vote share)

    This equation accounts for 94 per cent of the variation in the Labour vote share and is an improvement on one that only used the 2010 labour vote share which only accounts for 77 per cent of the variation. Adding in the Wales and the English regions makes a further marginal improvement with Labour obviously doing well in London and the North of England..
    A similar equation for the Conservatives using the same variables but substituting the conservative 2010 vote share for the Labour one accounts for 95% of the variation. The differences were that their vote share was unrelated to turnout, the negative effect of Scotland was less (they had less to lose) and the negative effect of a strong UKIP vote was also somewhat less (-.24 *ukip vote as against -.39). Although Scotland and UKIP and the proportion of elderly voters were statistically significant predictors, they addition explained very little more of the variance than the conservative share in 2010 with the total explained remaining at 95 per cent.

    What all this suggests to me is that in terms of winning seats the changes since 2010 have had much more influence on Labour than the conservatives. It is obviously more affected by Scotland. It is also more affected than the conservatives by disaffection with politics insofar as this is expressed in turnout and in the rise of protest parties such as UKIP (UKIP’s vote is high in constituencies where turnout is low). Finally it has tended to fall back or fail to grow in constituencies with a high proportion of elderly people. This need not be to do with how the elderly vote but my own suspicion is that they see Labour as more of a threat to their living standards than the Conservatives.

  3. @Charles

    “Capitalism depends on the state not only for its ethical regulation but for its efficiency, Why isn’t Labour making these kind of arguments and concentrating on showing how they can work in those authorities where it is still in control?”

    ————–

    Good question, and there are various reasons.

    One is that it’s complicated, requiring a discussion of what we mean by efficiency and knock-on effects. Make cuts to support services might increase hospital costs etc.

    Same is true of regulation. Increasing or decreasing in different ways can have different knock-on – sometimes counter-intuitive – effects.

    Same with government spending. Many don’t see it as an investment; once you’ve established that then it’s a thorny discussion about rates of return.

    But beyond that, to hear political geeks and activists talk about it online, they act like it’s a burning issue for the populace at large. In truth, these aren’t things I am in the habit of hearing people moan about and if you think that’s just anecdote, look at the issue trackers. The amount of regulation on business does not feature greatly, for example.

    Many people worry about outcomes, not methods to secure outcomes.

  4. I should add… You get a bunch of business peeps together, and they may enjoy a whinge or two about regulation. Whether business peeps would agree on whether we should have more or less, in different circs, is summat else. A small trader might disagree with what the head of a big company intent on hoovering up more of the market, may want…

  5. Charles
    ChrisLane
    CMJ

    I agree that ‘socialism’ like ‘capitalism’ is really quite hard to define, and its pretty pointless trying.

    What I think I am saying, and probably Parris too, is that the current Labour narrative is simply out of date, and does not relate well to the modern socio-economic environment. Of course it has to address the problems of the disadvantaged, but it must also appeal to the upwardly mobile, the moderately successful, the struggling but surviving…

    Labour has nil appeal to so many who inhabit this middle ground of British life. Whilst this remains the case, they cannot win, in my view, barring a massive ‘own goal/calamity for the Tories.

    As Charles says, the frustration for many of us is that there is an obvious message to send and explain, but they are stuck in the soundbites of the 1980s.

    It really is very unhealthy for British politics.

  6. Charles

    An interesting calculation, but I’m not sure it’s aided by including Scotland (where you have to introduce a special weighting factor).

    What happens when you just run it for England (or even E&W)?

  7. “Of course it has to address the problems of the disadvantaged, but it must also appeal to the upwardly mobile, the moderately successful, the struggling but surviving…”

    ————-

    One difficulty here, is that the austerity meme limits what they can offer to spend to assist.

    Another is that there are factors which are perceived to trump such concerns, e.g. immigration and devolution.

    A third is the question of how many really wish to do all that much striving, as opposed to having the rewards without so much striving*, especially when seeing others rewarded handsomely even for failure. Hence house prices being quite effective electorally.

    * some have asserted that this is a hangover from the aristocratic thing, not having to work for your money etc.

  8. Good late evening all from a sunny (yes it’s still sunny) Giffnock.
    ……….

    THE OTHER HOWARD
    Allan Christie
    Just back from four days walking holiday on the South Down. Splendid weather for walking and excellent hotel.

    Although I forecast the election accurately I do not think my views very helpful to the pollsters. Remember I have always said I take very little notice of the actual poll headline numbers, it’s the answers to detailed questions on the economy and leadership which are important in forming my predictions
    _______

    Lovely part of the World the South Down’s and great for walking. Glad the weather was good. I’m just back from a long weekend mountain biking up in Fort William. Weather okay but midges were horrific in the evenings.

    Now.. I agree with you on the polling and although unlike you I never predicted a Tory majority, I did however say the election would be won on the key performance indicators and strong leadership.

    I’m interested to see if the current Tory government can withstand the current rumblings over the EU and ministers being able to campaign for a EU exit!!

  9. @oldnat Good question!

    Omitting Scotland doesn’t make a difference to the variance explained and leaves the coefficients of the other variables more or less unchanged. What it does do is enormously strengthen the correlation between 2010 and 2015 Labour share. So I was correct in saying that changes since 2010 had disproportionately affect Labour. What I should have tested was how much that was simply the Scotland effect which it largely was.

  10. @Carfrew
    “A third is the question of how many really wish to do all that much striving, as opposed to having the rewards without so much striving*, especially when seeing others rewarded handsomely even for failure”

    Good point. It always annoys me when they all talk about ‘hard-working families’. What about those of us who are retired after a career of working reasonably hard, but not too much, or those who do the bare minimum to get by?

    It’s a bit like the quote that someone on here came up with recently which went something like ‘there are a lot of racists in my constituency, and they’ve got to be represented too’!

  11. @Pete B

    In the end it’s where the votes are. Question is, how many votes in striving vs not. And how many ways parties can come up with giveaways for the floaters that require minimal striving, without alienating too many others.

    Tories under Thatch and NuLab were both quite good at this… (Cut-price privatisation shares, right-to-buy etc. vs. Tax credits, House prices, heating allowance etc…)…

  12. @ Millie

    “I agree that ‘socialism’ like ‘capitalism’ is really quite hard to define, and its pretty pointless trying.”

    Relativism has no bounds :-). Both are quite easy to define (with nuances to variations), but not here.

  13. @Pete B

    Or to put it more clearly, I wasn’t saying anything about what ought to be the case etc., but simply looking at one neglected aspect of how parties may make their calculations in terms of VI, and the policy choices that may go along with that…

  14. @ Chales

    If you run your comparisons with 2005 (E&W), it is even more interesting. You just have to let the assumptions about LibDems go …

  15. Apologies for the tablet cutting out the “r” (blaming the tablet …)

  16. @Laszlo Sounds interesting. Could you explain? (II don’t have the data for 2005)

  17. Lazlo

    Tell us more!

    As for the debate about “Does the Labour Party exist?” which seems to rage on here, I liked this comment –

    “Gods evaporate when people stop believing in them, also true for political parties.”

  18. @ Charles

    BES has data for 2005. What I found was that there many of the 2010 LibDem voters who disappeared in the North didn’t go to Labour, but to UKIP. It is strange, I admit. The 2005 is important because this when the LibDems start to stack up “real” protest vote, the anyone but Labour vote.

    It makes the analysis more nuanced. While it happened in the East as well, but in a different way. So the churn has distinct regional differences. London is a different “country” altogether.

    In my view the presentation of the Lab<UKIP churn is way out the way. It happened, but its effects were far from the one assumed.

    Start the analyis from the disappearance of the LibDem vote and match it with UKIP. Unless one assumes that a convenient number of Labour supporters switched to UKIP to make the mathematical room for the LibDem switchers to Labour (many of whom would not vote Labour in previous elections), there is a problem with the Lab<UKIP switch at the scale people suggest.

  19. @ OldNat

    What a bait! :-)

    I can’t speak for Scotland (honestly or not, I found only about a third of the wedding guests admitting to voting for SNP).

    The simple demarcation line is about the control over the surplus value the economy produces. There are nuances, let’s say between the German and the English model in that, but it is not Cordon Sanitaire.

    From this it follows that the difference between the Conservative Party and the Labour Party are nuances too. These nuances can be important, depending on the context, but it doesn’t make Labour socialist (as far as I know, it never declared itself as such, unlike the sister parties on the continent). But equally, it doesn’t make Labour Tory Lite.

    This is where it becomes tricky. There is actually no socialist party in the UK (I include Scotland now, as the Socialist Labour and its versions are just various factions of the Labour Party who broke off for various (contextual) reasons.

    The tricky one is the Green Party (parties in the UK). Their policies are outside the Parliamentary system (although they compete for seats, and a considerable faction is ready to make the leap …). All their policies are about the control of the surplus value, and they surprisingly flexible. While there are conservative members, it is largely lefty, intellectual, and a lot of the argument, narratives are developed by socialists.

    At the same time, they suffer from infantile disorders (CP at about 1920?). GMO, local production, organic … These are not what define the party politically, but create emotional links. And these will have to be broken in the name of reason … So, in my view, it is the only party that has the perspective of becoming a socialist party, not by name, but by policies.

  20. We live in a mixed economy.

    Both Labour and Conservatives advocate a mixture of Capitalism and Socialism, and both Capitalism and Socialism are present in the economy in significant measure.

    Under Capitalism, owners of the assets, own the means of production.

    Under Socialism, the workers own the means of production.

    In our economy, some people work for the Capitalists, and some people work for themselves, owning their own equipment. Tradesmen, Consultants etc. adopting a socialist mode of production. Someone on a market stall, or who owns their own shop: Socialist. Until they start employing others, whereupon they become Capitalists UNLESS the employees get a share in the business. E.g. partners in a law firm, whereupon it’s Socialist again.

    There’s a form of Socialism, known as State Socialism, in which the state owns the company and employs people. This is considered by some to still be Socialist, as in the end the people own the State. Thus if you work for the state you work for yourself really. A kind of Socialism-by-proxy. Not sure I really buy it myself…

    But nonetheless, we have some state ownership as well in the mix. None of the mainstream parties are advocating an end to either Capitalism or to Socialist modes of production.

    That’s one part of Capitalism vs Socialism: who owns the means of production. The other part, is Marx’s dictum “From each according to their ability, to each according to their need”. Commonly taken as ” redistribution”. We have that too under Tories and Labour to some extent.

    Neither party could be branded as just Capitalist or Socialist. They are a bit of both.

  21. In case anyone’s worrying, I know, I left out Rentiers and stuff, but there aren’t any mainstream parties advocating an end to that either…

  22. Even this blog of Anthony’s: Socialist mode of production*. (Unless there’s a secret owner we don’t know about…)

    * unless one gets into whether anyone owns the net etc.

  23. @Everyone

    Hopefully the posts immediately above may clarify things for anyone who still thinks it’s simply a case of: Tories are the Capitalists and Labour are the Socialists.

  24. @Laszlo

    For clarification, all the statistics I did show is that both labour and UKIP tended to do well in seats where turnout was low and that when allowance was made for this, for the proportion over 60 in the population and for the labour vote in 2010 a high UKIP vote was associated with a low Labour one.
    (If no allowances are made for these things the proportions voting Labour and UKIP are not associated with each other)

    The explanation for these findings could be a direct switch of Labour to UKIP voters, a general disenchantment with politics leading some Labour voters to abstain and others to switch to UKIP or no doubt other things.

    Personally I think that it is disenchantment. The task for Labour is therefore to craft a message that people can believe in. Some of this task is probably intellectual but for me some ot it is showing that Labour an really do a good job at a local level.

  25. This thread is no more. It has ceased to be. It’s expired and gone to meet its maker. This is a late thread. It’s a stiff. Bereft of life, it rests in peace. If you hadn’t nailed it to the perch, it would be pushing up the daisies. It’s rung down the curtain and joined the choir invisible. This is an ex-thread.

  26. @Laszlo – Could one interpret the point you are making as follows? In Labour heartlands many people have become disenchanted with them, probably seeing them as corrupt, or complacent or untrue to their roots. The beneficiaries of this change have varied with region and timing, being the Lib Dems in 2005, and the SNP and possibly UKIP in 2015.

    If that was true it would reinforce the need for Labour to get its local government in order and ensure that the likes of Rotherham does not damage it further.

  27. Lol Col., you despairing at us all again?! Puts one in mind of Howard… Haven’t seen him for a while, hope he’s OK.

  28. @Charles

    “Could one interpret the point you are making as follows? In Labour heartlands many people have become disenchanted with them, probably seeing them as corrupt, or complacent or untrue to their roots. The beneficiaries of this change have varied with region and timing, being the Lib Dems in 2005, and the SNP and possibly UKIP in 2015.”

    ————-

    Well, you could interpret it that way, if really keen on the idea that Labour ought to be more left wing and this is what peeps are crying out for.

    Alternatively, if you look at where the votes went, you might conclude that peeps were happy enough to let Labour poll at 45%, before simply deciding following press campaign and Indy ref that immigration and devolution were more important. Hence votes haemorrage to Ukip and SNP.

    Being as immigration topped the issue tracker etc.

  29. @Laszlo etc.

    I think your interpretation assumes that these people in these seats wanted to vote for Labour if only they were less corrupt, complacent, untrue to their roots. There may be some people for whom that is true but really we’re dealing with a dealigned electorate that increasingly don’t identify with a party as they did 50 years ago. That presents both challenges and opportunities to the main parties.

  30. I should also add that Labour’s problems in northern heartlands are a bit overplayed. Sure, UKIP came second in a lot of seats but they are a long way behind in the vast majority and I see no other realistic challenger on the horizon.

  31. CARFREW

    Just joshing ( as they used to say)

    Re your musings on Capitalism & Socialism-a few interesting parallels in my Times today:-

    Melanie Philips on “Temple” at the Donbar ( Simon Russell Beale)-the agonies of the Dean of St. Pauls on the rights of Occupy vs the rights of Worshippers ( going to see it actually)

    The ever excellent Mat Ridley on remembering 1815 for Wellington’s ability to kill lots of of people, rather than for Stephenson’s invention of the machine which brought unimagined prosperity to millions.

    He mentions too the never celebrated textile merchant Nathan Rothschild who devised a system for financing Wellesley’s army, wherever it was.
    ( We now call this banking & condemn it out of hand)

  32. @Carfrew Personally I would not say that they necessarily wanted Labour to be more left wing. Probably they think it is largely irrelevant and don’t want it to be anything particularly. A lot of them think that they have seen it in operation and don’t think it has done them a lot of good. They may adopt various explanations for their ills including immigration (if you are UKIP minded), England and London in particular (if you are SNP minded) and selfish capitalism or something like that if you are for the Greens. The point is that lacking a core political belief they are suckers for some kind of political enthusiasm when it comes along.

  33. Allan Christie

    Sorry you had an attack of the dreaded midges. I love walking in Scotland but prefer late April or early May.

    How Cameron deals with the splits in his party over Europe is of great interest. He started badly I thought by talking about cabinet ministers having to resign to back an “out” campaign but seems to have rowed back from that. I thought Wilson handled it well when he had a similar problem

  34. The rumours are Jeremy Corbyn is 5 nominations short at this time.

  35. Personally, I find apparently serious Labour politicians trying to get a 66 year-old backbencher with no front-bench experience and who is practically a Marxist on the ballot paper to be rather hilarious. If they are successful and he then wins because every extreme left-winger in the country pays their £3 to vote/ signs up through their union it would be even more hilarious.

  36. Corbyn is apparently on the ballot. Good news for Burnham (harder to cast him as the left candidate). Will be interesting for Kendall (gives her something to argue against but she runs the risk of coming across as too right wing in the process).

  37. He’s on the ballot. Labour leadership contenders are:

    Andy Burnham
    Yvette Cooper
    Jeremy Corbyn
    Liz Kendall

    (Listed alphabetically but coincidentally also in order of my likely preferences).

    Trivia fans: Chuka Umunna’s withdrawal meant that not since John Prescott in 1994 will any contender for the Labour leadership have had a name in the latter half of the alphabet.

    Jack Sheldon,

    He won’t win. There aren’t enough far leftists who’d be willing to sign up and vote for him, even if they were organised enough to try, which they aren’t.

  38. Apparently Corbyn made it. I don’t know if it’s good for the LP or not. They justify it with “debates”. Well, that would be something new …

  39. @Jack Sheldon

    Jeremy represents a constituency that needs to be heard for a full debate.

    Frankly, he would be soundly beaten if he get to the ballot. That would help the party as they could demonstrate that the left had a candidate, and was rejected solidly.

    The left of the party will also be unable to claim their candidate was blocked if he is beaten fair and square.

  40. On balance its a good thing corbyn made it as it forces the other three to articulate an alternative to anti austerity -which is what they have to do in part to make any progress in scotland.

    Also Unites political committee and other unions will probably recommend a vote for corbyn making it harder for the tories to say burnham is the trade union backed candidate(assuming he wins of course).

  41. If my facebook feed was realistic Corbyn would win by landslide. While being a left wing Labour person I hope he don’t as I cant see him standing a chance of beating the Tories at the next election.
    I wonder when the first opinion polls on the Labour Leadership and Deputy Leadership will be held. Is anyone doing an exit poll for Wednesday debate.

  42. Given OMOV and the likelihood of Corbyn attracting some new members, it’s not out of the realms of possibility that Corbyn could beat Kendall. He’ll either come third or fourth though.

  43. I would say it is more likely Corbyn could beat Cooper than Kendal through. Both will get many first preferences and many last while Burnham and Cooper may get less first preferences but more 2nd and 3rd which may not be enough to overrule Corbyrn strong first preference vote.

  44. My bet would be Yvette and Andy to top the poll, with Liz and Jeremy to fall out first, although I’m not sure of the order.

    I can see an Andy vs Yvette fight in the end.

  45. @ Charles

    I suppose my argument is that there are about 4 million protest votes out there who are unlikely to vote eithe Conservatives or Labour (and LiBDem after 2010).

    Yes, I suggest that the loss (or small increase) of Labour vote has regional characteristics, depending on the size of the protest vote and the ways in which LibDem and Labour vote churned (to Labour and UKIP).

    Obviously, there are places where a better run council can help, but I don’t think it is decisive for the GE.

    If you take Liverpool, the council is super-dominated by Labour. It is very effective (I know people from outside might be surprised as Liverpool traditionally hasn’t been associated with such an attribute). The collapse of the LiBDem vote (which has always been anyone but Labour here) is roughly the same as the increase in UKIP, although some might have gone to Green (and some Labour went to Green). The increases in Labour majority derives from higher turnout.

    Manchester is a bit different (as in some parts voting for Conservatives is socially acceptable). The council of the city proper (I think) 100% Labour. Considering the complexities of the city it works well, although there are significantly more complaints – according to the figures released by the council – than in Liverpool. There is a higher churn from Labour to UKIP (but not very high). In parts the LibDem vote held up well, as it is an anti-Tory vote in the affluent areas. There might even have been Lab<COn switch in a smaller scale. So, turnout there is everything.

    These two examples are very different from the way in which it went in East Midlands.

    I don't think going to the left or right matters. labour is a coalition, but there is a difference between being undefined (which seems to be a case) or broadly defined (even with contradictory policies).

  46. Some commentators are furious corbyn has made it.

  47. There does seem like they will be a campaign in the parties on the far left to encourage them to pay the £3 and vote for Corbyn. Could see many greens doing so as well.

  48. Unions can of course campaign for a candidate, but it is expensive, and it will be the LP who sends out the voting cards this time.

    Also, I doubt that the turnout would be very high. So, yes AB or YC are the likely ones.

  49. @Colin

    Well, calling them musings does kinda suggest a tenuousness to my post that perhaps undersells my sterling efforts to make things less tenuous. Unless you think it is Conservative policy to outlaw partnerships or summat.

    It does tend to disorient some on the right to discover that they might be a bit more Socialist than they think though!!

    I agree that there can be tensions between using protest to get attention over summat peeps think is important, and the desire of others not to be inconvenienced: we see that with strikes too of course. Equally, agree that engineers may be insufficiently appreciated.

    Dunno how much people are against “banking” per se, as opposed to assorted peeps in banking taking the mick, which extends beyond what happened in the crunch, to scandals like Libor and PPI, moving Carney to despair at the endemic nature of it all. Accompanied by all the lower-level everyday examples that banks use to relieve people of their cash. One almost dreads going into a bank these days for fear of what they are gonna try next…

  50. @Charles

    Agree that some tend to externally attribute their ills, looking to blame someone or something else. Though as I posted recently, things can be a bit more complicated than they may seem at first.

    I’m not sure about the core beliefs thing. It can create more problems than it solves. Because once one is invested in something to the extent that it is “core”, it can be harder to let go when it turns out to have issues.

    Both the over-enthusiasm to blame summat else, and the reluctance to be wrong over core beliefs, can be signs of ego issues, though of course there may be other reasons. Sometimes a core belief may be adopted ‘cos easier to get ones head around, alternative explanations being harder to understand or even perceive.

1 2 3 4