Rob Hayward, the former Tory MP turned psephologist, gave a presentation at ComRes on Monday which has stirred up some comment about whether the polls are underestimating Conservative support.

Historically the polls have tended to underestimate Conservative support and/or overestimate Labour support. It was most notable in 1992, but was a fairly consistent historical pattern anyway. Since the disaster of 1992 this bias has steadily reduced as pollsters have gradually switched methods and adopted some form of political control or weighting on their samples. In 2010 – at last! – the problem seemed to have been eliminated. I hope that the polling industry has now tackled and defeated the problem of Labour bias in voting intention polls, but it would be hubris to assume that because we’ve got it right once the problem has necessarily gone away and we don’t need to worry about it anymore.

In his presentation Rob compared polls last year with actual elections – the polls for the European elections, for the by-elections and for the local elections.

I looked at how the polls for the European election did here and have the same figures as Rob. Of the six pollsters who produced figures within a week or so of the election five underestimated Conservative support. The average level of Tory support across those polls was 22.2%, the Tories actually got 23.9%. The average for Labour was 27%, when they actually got 25.4%.

Looking at by-elections, Rob has taken ten by-election polls from 2014 and compared them to results. Personally I’d be more wary. By-election campaigns can move fast, and some of those polls were taken a long time before the actual campaign – the Clacton polls, for example, were conducted a month before the actual by-election took place, so any difference between the results and the polling could just as likely be a genuine change in public opinion. Taking those polls done within a week or so of the actual by-elections shows the same pattern though – Conservatives tend to be underestimated (except in Heywood and Middleton), Labour tends to be overestimated.

Finally in Rob’s presentation he has a figure for polls at the local elections in 2014. I think he’s comparing the average of national Westminster polls at the time with Rallings and Thrasher’s NEQ, which I certainly wouldn’t recommend – the Lib Dems for example always do better in local election NEQ than in national polls, but it’s because they are different types of election, not because the national polls are wrong). As it happens there was at least one actual local election poll from Survation.

Survation local election: CON 24%, LAB 36%, LDEM 13%, UKIP 18%, Others 10%
R&T local election vote: CON 26%, LAB 36%, LDEM 11%, UKIP 16%, Others 12%

Comparing it to the actual result (that is, the actual total votes cast at the local election, which is what Survation were measuring, NOT the National Equivalent Vote) these figures were actually pretty good, especially given the sample size was only 312 and that it will be skewed in unknown ways by multi-member wards. That said, the pattern is the same- it’s the Conservatives who are a couple of points too low, Labour spot on.

So, Rob is right to say that polls in 2014 that could be compared to actual results tended to show a skew away from the Conservatives and towards Labour. Would it be right to take a step on from that and conclude that the national Westminster polls are showing a similar pattern? Well, let me throw out a couple of caveats. To take the by-election polls first, these are conducted solely by two companies – Lord Ashcroft and Survation… and in the case of Survation they are done using a completely different method to Survation’s national polling, so cannot reasonably be taken as an indication of how accurate their national polling is. ICM is a similar case, their European polling was done online while all their GB Westminster polling is done by telephone. None of these examples includes any polling from MORI, Populus or ComRes’s telephone polling – in fact, given that there were no telephone based European polls, the comparison doesn’t include any GB phone polls at all, and looking at the house effects of different pollsters, online polls tend to produce more Labour-friendly figures than telephone polls do.

So what can we conclude? Well, looking at the figures by-election polls do seem to produce figures that are a bit too Laboury, but I’d be wary of assuming that the same pattern necessarily holds in national polls (especially given Survation use completely different methods for their constituency polling). At the European elections the polls also seemed to be a bit Laboury… but the pollsters who produced figures for that election included those pollsters that tend to produce the more Laboury figures anyway, and didn’t include any telephone pollsters. It would be arrogant of me to rule out the possibility that the old problems of pro-Labour bias may return, but for the time being consider me unconvinced by the argument.

UPDATE: Meanwhile the Guardian have published their monthly ICM poll, with topline figures of CON 30%(+2), LAB 33%(nc), LDEM 11%(-3), UKIP 11%(-3), GRN 9%(+4) – another pollster showing a significant advance for the Green party.


152 Responses to “Are the polls underestimating the Tories?”

1 2 3 4
  1. Old Nat

    The point I made was that ‘true allegiance’ in the eyes of Sinn Fein is ‘no allegiance at all’

    I’m sure the Irish are quite capable of understanding such use of the English language……!

  2. John B

    Yes. We share the same stance. In reality, SF are a political party and will do whatever they think is in their best interests – like every other party!

  3. @ Old Nat

    I think what Nameless is saying (certainly what I’m saying) is that the Greens will not have enough seats to form a coalition and very unlikely they will have enough seats to have a say as to who runs the next government. I also think UKIP have only a very slightly better chance of being in that situation either.

    So I think if people think that by voting UKIP or voting Green, their preferred party will have any sort of say in government in 2015 they are likely to be disappointed. However if they want to build up a party that closely resembles their views and gain a foothold at Westminster as a springboard to future elections then maybe they will be relatively content although in the case of the Greens it might be a long slog as they seem unlikely to get more than one seat this time but maybe some strong second places.

  4. A Lab Green coalition will mostly not happen for the reasons David sets out, but on Oldnat’s point there are two major policy sticking points for Lab and Green supporters.

    One is Foreign policy, as I do not believe Labour voters would endorse removing border controls, dismantling the armed forces and legalizing membership of Al Qaeda and the IRA.

    On Fiscal policy, I do not believe Labour supporters would support zero or negative economic growth.

  5. Irish citizens?
    Huge numbers have sworn allegiance without repercussions, for example serving in HM Forces. An exception perhaps were those who left the Irish Army to join the British Army in World War 2.
    An agreement on army co-operation was signed a couple of days ago between the UK and the Republic which the UK government says is like something the UK has never signed before with the Irish government stressing it is voluntary and could conceivably be reversed.

  6. Shevii

    My guess is that if the result of the GE is as tight as polls suggest, even a party having only one or two seats will find it is enough to gain the attention of anyone wishing to form a government!

  7. @ John B,

    My concern, however, is how long a minority government could last at Westminster. And who would get the blame for its collapse?

    Whoever brought it down, presumably. The Tories of course would be quite happy to get the “blame” for the collapse of a Labour government, since their supporters want a Tory government. The SNP… might be less happy with that outcome.

    A coalition is a specific thing, in which both parties provide government ministers. Labour certainly couldn’t go into one with the Tories as it would destroy them. They tried it once, it was a terrible idea, never again. (And actually, I think the Tories are in a similar position, although their voters seem a bit more loyal- if they make Ed Balls Chancellor would t’Other Howard ever vote for them again?)

    However, carrying specific votes with the support of other parties is a very different matter, and not uncommon. Cameron has relied on Labour in this Parliament to pass same-sex marriage and block an EU referendum proposal, for instance. Ukippers are unhappy about the specific policies concerned, but as far as I know no one has denounced Cameron as a Labour-loving traitor for walking into the same lobby. It’s very easy in these circumstances for the government party to frame themselves as the policy-makers and the opposition party as the tag-alongs.

    The situation in Scotland adds a wrinkle, because if Labour rely on Tory votes to pass legislation the SNP opposes the SNP will never, ever let them hear the end of it. But that’s not a coalition, and in the case of Trident they might be forced into it.

  8. PAUL
    “A potential SNP and Tory agreement would be around constitutional reform – EVEL, Devolution plus in Scotland. I don’t believe a coalition is likely, but in return for further autonomy, the SNP might support them in the lobbies”
    ______

    I think the SNP are quite vocal regarding not doing any deals with the Tories because it stops Westminster Labour in Scotland exploiting any uncertainties on Tory/SNP deals even though Labour themselves haven’t ruled out a left/Right coalition with the Tories.

    If the Tories (and most of the experts suggest they will) become the largest party and Labour can’t form a government just through sheer parliamentary arithmetic then I think EVEL and DEVO MAX might be on the lips of UK and Scottish ministers.

    However with scrapping Trident on the SNP/Greens agenda and the Tories and Labour’s backing for Trident then I’m thinking a minority Tory or Labour government will be the most likely outcome in this regards.

    It could be that the SNP and Greens may have overplayed their hands.

    Or I’m just waffling!!

  9. Shevii / Shev II / She VII

    (Unsure as to your preference! :-) )

    As I said about SF above – parties will do whatever is in their best interests. With Greens seemingly nibbling at Lab in England, it might be in Lab’s interest to seek C&S with a number of smaller parties to be allowed to govern – or they might choose to try minority government – or abdicate responsibility altogether.

    In a hung parliament, the discussions in the vape filled back rooms would probably depress all democrats. :-(

  10. Best for you to disown that disgusting comment.

  11. SPEARMINT

    Many thanks for the info re strike…../strike. I’ll certainly use it when appropriate.

    Odd since it’s being phased out in HTML5 so that only the old s…../s construct will be available per http://www.w3schools.com/tags/tag_strike.asp

  12. @ Barbazenzero,

    I think the WordPress comment thingie is hard-coded to allow only specific HTML- that’s why you can’t insert images, for instance- and it may not be up to date with the tags. Hence [strike] and not [s].

  13. Some quite striking numbers in our daily @YouGov poll tonight. Will be on @TheSunNewspaper online at 10.30pm.

    Don’t you just hate teaser tweets?

  14. SNP on 20% as Scottish population booms.

  15. . How can one poll have the Greens on 3%, yet another has 11%? That’s like one pollster having Labour on 10% and another having them on 38%! Would we challenge them if that was the case?
    I think the betting markets are a better judge of the results than pollsters.

  16. Hoof Hearted

    But aren’t the betting markets just a self-selecting panel for a “Wisdom Index”?

  17. RAF

    “Following the money”

    Stalk a bookie!

  18. RAF

    Indeed follow the money.

    “This is not the only way we could have built a forecast from our odds. If you total the probabilities of each party winning each seat, you’d find the SNP doing a few seats better (mostly at the expense of Labour) and UKIP doing a lot better. As it happens, I think the method above probably underestimates the SNP’s chances, which may indicate that there are still some good bets on them in some of the individual seats”

  19. OLDNAT
    Some quite striking numbers in our daily @YouGov poll tonight. Will be on @TheSunNewspaper online at 10.30pm.

    “Don’t you just hate teaser tweets?”
    ______

    Lib/Dems possibly on double figures?

  20. But aren’t the betting markets just a self-selecting panel for a “Wisdom Index”?
    _——_—————–_

    Well as the pollsters haven’t got a clue.

  21. Another thread hijacked!

    “Are the polls underestimating the Tories?

    Personally, I acknowledge that they might be but rather than shy Tories it could be ‘couldn’t be arsed Labour’

    Tories imo, and evidentially I believe, are more likely to vote in seats where their vote does not matter especially in their safe ones.

    The key is marginal seats where Labour voters will be subject to the party’s still strong grassroots GTVO organisation.

    In 2010 ( and even more in 1997 for the cons) I think the desire to vote for some Labour sympathisers was not acute leading to more DNVs and contributed to the result.

    Perhaps if the polls are wrong and the Cons are underestimated the FPTP Labour advantage (which could be said to be about efficiency as much as anything else) may be maintained or even greater?

  22. @ OldNat

    RAF
    “Following the money”
    Stalk a bookie!

    Lol!

    @AC
    Of course we shouldn’t attach any real value to fools and their money, but you can be sure that someone, somewhere is in all probability monitoring whether the bookies or the pollsters will deliver a more accurate result (while munching on a Toffee Crisp).

  23. Hoof Hearted.

    As Anthony has mentioned in previous threads, Comres and Populous heavily weight against the Greens (and UKIP) and so massive deflate their vote. Which is probably unrealistic now. The 11% is in line with trends and recent bounce, though probably at the higher end, of course.

  24. IG1234 – Actually (putting on my pedantry hat) Populus’s low Green score appears to be due to weighting them down, ComRes’s low score in their online polls appears to be due to filtering out many Green voters through their turnout filter – so the reasons for their low scores are very different.

  25. @Mr Nameless

    One is Foreign policy, as I do not believe Labour voters would endorse removing border controls, dismantling the armed forces and legalizing membership of Al Qaeda and the IRA.

    On Fiscal policy, I do not believe Labour supporters would support zero or negative economic growth.

    Would you care to provide evidence for these points?

  26. Why are YG trailing tonight’s poll so heavily ? If they know the figures now, why don’t they publish now?

  27. “Some quite striking numbers in our daily @YouGov poll tonight. Will be on @TheSunNewspaper online at 10.30pm.”

    36-24-36…….probably funnier yesterday than today!

    Peter.

  28. CMJ,

    I’m going off this Telegraph article: http://www.telegraph.co.uk/news/politics/green-party/11356354/Drugs-brothels-al-Qaeda-and-the-Beyonce-tax-the-Green-Party-plan-for-Britain.html

    It is the Telegraph but claims to be sourced from the “policies for a sustainable society” document adopted at conference. I can’t find the document in question, mind.

  29. Jasper22

    Because release of polling data is the right of the client – not the pollster.

  30. Peter

    :-)

  31. @MrNameless

    I have seen the claims in the Telegraph, but they do not state the source (ie the policy reference).

    From the GP website:

    Terrorism

    PD440 Terrorism is an extremely loaded term, frequently used by those in power to justify excessive use of force or the weakening of controls on the exercise of their power. Sometimes governments justify their own terrorist acts by labelling any groups that resist their monopoly of violence “terrorist”. A Green government, by implementing principles laid out elsewhere in this manifesto, particularly those of self-determination and non-interventionist foreign policies, would seek to overcome the unjust divisions within our global and domestic society and address the desperate motivations that lie behind many atrocities labelled “terrorist”.

    Translating that to mean membership of the IRA will legalised looks like rot to me.

  32. @ AW

    Re: “Shy Tories”

    Just to elaborate on a point I made yesterday, the total number of votes cast in the EU election was about 16.5 million. The corresponding total for GE2010 was about 27 million. By their actions, voters treat the EU elections as far less important events than general elections.

    Given this, it might be the case that the polls were unbiased but that the Tory voters were somewhat more diligent in acting on their stated intentions.

    On this account the effect might disappear in an election that is seen to be closely fought and important, and where all the incentives are there to go out and have a say in the future government of the country.

    The ‘shy Tory’ hypothesis may have seemed plausible when Thatcherism was in full swing. But with Ukip now outflanking the Tories, there hardly seems any reason now to be shy about supporting a more moderate party.

    The actual explanation of any such bias may make all the difference to its putative reappearance in the coming election.

  33. @JimJam

    You make some interesting points about the strength of party organisation, especially in marginal seats. This is where UKIP may have done more below-the-radar damage to the Tories than some people are imagining. A significant number Tory activists have gone over to UKIP and, as happens with all governing parties, they’ve lost many councillors since 2010, the likely foot-sloggers and door-knockers during election campaigns. We should be prepared also for surprisingly strong and well funded local UKIP parties, even in constituencies where they have little chance of winning or even competing strongly. I’ve seen some of it at first hand and it’s impressive. In the 2014 local and Euro elections UKIP were the most hard working and passionate of all the teams campaigning in our locality

    The Tories, as they always do, will win the air war in the forthcoming election , dominating the hoardings and outspending their rivals by vast amounts, but I have a funny feeling that this election, more than any other, will be won and lost on the ground. That’s where I sense the Tories are at their weakest too.

  34. Anthony – my bad! Thanks for the clarification.

  35. Apparently the Torries are in the lead in a poll but I imagine by a fair margin if its front page worthy.

  36. Of course how trustworthy a poll like that is remained to be seen. Wide difference from the other polls are not at all trustworthy.

  37. @ BM11

    Whatever tonight’s poll shows, it will be worth putting it in the pot with the others and letting it bubble for a week before we all start predicting the end (or the beginning) of the world.

  38. Yes, the polls are likely to be underestimating the Tories.

    Why? Because in our new […] system of Individual Electoral Registration, which incredibly has been foist upon us in untested form in none less than a general election year by a gung-ho Electoral Commission, the young and transient private renters are much less likely to be registered than before. And by and large, these people are not going to vote Tory.

    No polling company has yet made any changes to their methodology to cope with this […] new system. If the pollsters got it right by and large in 2010, at least in the Con-Lab margin, then in the absence of change for this I think they’re likely to get it wrong.

    Before the “likely to vote” question, there should be an “are you on the electoral register” question, with responses of “No”, “DK” and “What’s the Electoral Register?” being all but discarded.

  39. @ BM11

    You got there first with the advice for everyone to ‘calm down’.

  40. Curious about this poll now… Meanwhile I’ve put some charts together – it does seem like the phone polls are showing bigger moves (in the same direction)

    http://www.ncpolitics.uk/2015/01/uk-opinion-polls-far-2015.html/

  41. Having posted for the first time in two weeks (yes, monotonous threads dominated by Scotland really do drive other posters away) only to find my contribution directly on the subject of this thread go straight into mod, I do wonder what was the point of trying to return.

  42. One poll is one poll. As AW says, we need a few…people saying not a Tory lead but huge Green score??

  43. Thanks Anthony for the detailed response to the Rob Hayward presentation. I don’t know who is right, but I learn lots when the experts have these discussions/ disagreements.

    When you think about all the moving variables – ‘shy’ respondents/ changing minds at the ballot box/ different turnout by different groups/ tactical voting just for your constituency, but really supporting someone else / voting different ways in different elections etc it is amazing the polls get at close as they do to the actual results.

  44. @ Jasper – I would guess so, given that we’ve had 7 point LAB leads and 6 point CON leads… Surely it’s something besides the lead

  45. Meanwhile, the BBC reports:

    The BBC’s Nick Robinson said inquiry chairman Sir John Chilcot is expected to confirm in a letter to David Cameron that it will be delayed until after May’s poll.

    http://www.bbc.com/news/uk-politics-30908938

    If true that’s bound to have polling implications.

  46. The delay with Chilcott is a scandal.

  47. I think the contents are too politically sensitive to release during a GE Campaign.

    (Although I am sure that one party very pleased about the delay and every other one is disappointed.)

    I’d personally rather see an election campaign without the distraction. Given the delay to date, waiting until June or so makes no difference.

  48. JASPER22
    The delay with Chilcott is a scandal.

    Agreed. The $64,000 question is which parties will benefit from the delay.

1 2 3 4