Rob Hayward, the former Tory MP turned psephologist, gave a presentation at ComRes on Monday which has stirred up some comment about whether the polls are underestimating Conservative support.

Historically the polls have tended to underestimate Conservative support and/or overestimate Labour support. It was most notable in 1992, but was a fairly consistent historical pattern anyway. Since the disaster of 1992 this bias has steadily reduced as pollsters have gradually switched methods and adopted some form of political control or weighting on their samples. In 2010 – at last! – the problem seemed to have been eliminated. I hope that the polling industry has now tackled and defeated the problem of Labour bias in voting intention polls, but it would be hubris to assume that because we’ve got it right once the problem has necessarily gone away and we don’t need to worry about it anymore.

In his presentation Rob compared polls last year with actual elections – the polls for the European elections, for the by-elections and for the local elections.

I looked at how the polls for the European election did here and have the same figures as Rob. Of the six pollsters who produced figures within a week or so of the election five underestimated Conservative support. The average level of Tory support across those polls was 22.2%, the Tories actually got 23.9%. The average for Labour was 27%, when they actually got 25.4%.

Looking at by-elections, Rob has taken ten by-election polls from 2014 and compared them to results. Personally I’d be more wary. By-election campaigns can move fast, and some of those polls were taken a long time before the actual campaign – the Clacton polls, for example, were conducted a month before the actual by-election took place, so any difference between the results and the polling could just as likely be a genuine change in public opinion. Taking those polls done within a week or so of the actual by-elections shows the same pattern though – Conservatives tend to be underestimated (except in Heywood and Middleton), Labour tends to be overestimated.

Finally in Rob’s presentation he has a figure for polls at the local elections in 2014. I think he’s comparing the average of national Westminster polls at the time with Rallings and Thrasher’s NEQ, which I certainly wouldn’t recommend – the Lib Dems for example always do better in local election NEQ than in national polls, but it’s because they are different types of election, not because the national polls are wrong). As it happens there was at least one actual local election poll from Survation.

Survation local election: CON 24%, LAB 36%, LDEM 13%, UKIP 18%, Others 10%
R&T local election vote: CON 26%, LAB 36%, LDEM 11%, UKIP 16%, Others 12%

Comparing it to the actual result (that is, the actual total votes cast at the local election, which is what Survation were measuring, NOT the National Equivalent Vote) these figures were actually pretty good, especially given the sample size was only 312 and that it will be skewed in unknown ways by multi-member wards. That said, the pattern is the same- it’s the Conservatives who are a couple of points too low, Labour spot on.

So, Rob is right to say that polls in 2014 that could be compared to actual results tended to show a skew away from the Conservatives and towards Labour. Would it be right to take a step on from that and conclude that the national Westminster polls are showing a similar pattern? Well, let me throw out a couple of caveats. To take the by-election polls first, these are conducted solely by two companies – Lord Ashcroft and Survation… and in the case of Survation they are done using a completely different method to Survation’s national polling, so cannot reasonably be taken as an indication of how accurate their national polling is. ICM is a similar case, their European polling was done online while all their GB Westminster polling is done by telephone. None of these examples includes any polling from MORI, Populus or ComRes’s telephone polling – in fact, given that there were no telephone based European polls, the comparison doesn’t include any GB phone polls at all, and looking at the house effects of different pollsters, online polls tend to produce more Labour-friendly figures than telephone polls do.

So what can we conclude? Well, looking at the figures by-election polls do seem to produce figures that are a bit too Laboury, but I’d be wary of assuming that the same pattern necessarily holds in national polls (especially given Survation use completely different methods for their constituency polling). At the European elections the polls also seemed to be a bit Laboury… but the pollsters who produced figures for that election included those pollsters that tend to produce the more Laboury figures anyway, and didn’t include any telephone pollsters. It would be arrogant of me to rule out the possibility that the old problems of pro-Labour bias may return, but for the time being consider me unconvinced by the argument.

UPDATE: Meanwhile the Guardian have published their monthly ICM poll, with topline figures of CON 30%(+2), LAB 33%(nc), LDEM 11%(-3), UKIP 11%(-3), GRN 9%(+4) – another pollster showing a significant advance for the Green party.


152 Responses to “Are the polls underestimating the Tories?”

1 2 3 4
  1. Good Late evening All.

    Ed Miliband must hope that the poll and other events do not de rail the Labour campaign, in the way the 1983 campaign was de railed.

  2. Phil

    Commiserations.

  3. CATMANJEFF
    I think the contents are too politically sensitive to release during a GE Campaign.

    It’s certainly “unfortunate” that the information is so late, but it is an affront to what little democracy there is in Westminster for the electorate not to be informed of the outcome ASAP.

    If there is some real reason for the delay, it should be made public and the GE postponed for a month or so.

  4. CL – do you have in mind the bizarre press conference in the middle of the campaign to say that Foot was still the leader?

  5. I personally wouldn’t delay GE for a report – I’d delay a report for a GE.

    However, we have fixed terms Parliaments now, so this is entirely hypothetical.

  6. Surely the holding of the GE should have priority over all other events. After all the United States held a presidential election in the middle of a civil war (1864).

  7. The rise of UKIP and the seeming rise of the Greens seem to me to demonstrate all that is most facile about the electorate.

    “Oooo……. it says on telly people are voting UKIP – that’s what I’ll do then.”

    “Oooo………… Greens are going up in the polls now; I’ll say them instead.”

    Do they understand their policies? Do they f*ck.

  8. JIM JAM.
    I remember that press conference well. Ron Hayward was General Secretary of ‘the party’ I think.

    Michael Foot’s first press conference of ‘the campaign’ was walking up a welsh hillside singing The Red Flag.

    Mr Kinnock made the ‘Guts on goose Green’ speech.

    Labour lost Scunthorpe and Barrow that year.

  9. Never mind the GE. The delay is a disgrace. This is Great Britain not some banana Republic for crying out loud.

  10. @R and D

    That is harsh.

    I went Labour to Green entirely by reading the respective manifestos cover to cover, and picking the one I supported the most.

    Are people’s reason for voting Lab. Con or Lib Dem any worthier?

  11. Chrislane
    The general Secretary was Jim Mortimer in 1983.

  12. CMJ,

    Will you do that again?

  13. @Jasper22

    It’s just a few months away.

    The report will change nothing anyway. Whatever is said will be treated by those who support it’s view as the Gospel according to Chilcott, and to those who think something else will consider it a conspiracy and cover up.

  14. @Hal

    Do what?

  15. cmj

    No I don’t think they are but they probably have more of a historical base and raisn d’etre.

    I don’t doubt for a moment that many people have strong motivation for such decisions but when many thousands change at the same time I genuinely believe it is part of a herd instinct rather than the result of serious analysis of different parties’ distinctive policies.

    No doubt it is also an indictment of Labour and Tories.

  16. CMJ,

    Read the manifestos cover to cover and decide who to support?

  17. @ChrisL1945

    Much as I like harking back to 1983, mainly because Villa were reigning European Cup champions for some part of that year, I really think it’s pushing it a bit to hark back 32 years to Ron Hayward press conferences, the Falklands War and Michael Foot.

    Politics inhabited a different world way back then and beyond nostalgic reminiscing, and I love a bit of that on winter nights with a good cup of cocoa and a roaring fire, any possible precedents for 2015 are, quite frankly, for the birds.

  18. GRAHAM!

    Many thanks, again

    CROSSBAT 11.
    Thank you.
    I smell disaster, that is why.

  19. NC,

    Interesting graphs.

    Is an explanation that the phone pollsters are getting samples that are biased according to the time of year?

    Hard to imagine that the volatility of the phone pollsters is correct and the online polls have somehow have a cancelling bias.

  20. @Hal

    I reread the 2010 manifestos in 2011/12, made my choice and moved party, and I’ve heard nothing since that has made me think of changing back.

    I do read all the manifestos every election.

  21. Barrow loss was mainly dueTrident on top of UNS

    (Also Barrow-in-Furness became Barrow and Furness between 1979 and 83 adding conservative areas making the notional majority closer)

    Albert Booth had the guts to be a CND supporter who was the Labour MP.

    (some might say stupidity)

  22. Labour hit five-year poll low as Greens surge

    UH OH….

    CATMANJEFF what have you done?

  23. Britain Elects [email protected] · 59 secs59 seconds ago
    Latest YouGov Poll (19 – 20 Jan):
    CON – 32% (-)
    LAB – 30% (-2)
    UKIP – 15% (-)
    GRN – 10% (+3)
    LDEM – 8% (-)

    Well I thought it was going to be a UKIP collapse after the private health insurance story… guess not.

  24. @ Catmanjeff,

    I went Labour to Green entirely by reading the respective manifestos cover to cover, and picking the one I supported the most.

    Yeah, but you were a Green before it was cool. Not like these hipsters jumping on the bandwagon. :p

  25. @rosieanddaisie

    “…when many thousands change at the same time I genuinely believe it is part of a herd instinct rather than the result of serious analysis of different parties’ distinctive policies.”

    Will all respect, I think you are quite wrong on this particular point. Voters may not be able to quote chapter and verse on policies, but they have strong ideological preferences that they associate with sepcific parties over time. As long as they are locked in a two party choice, the diversity of those preferences is masked.

    Voters are pretty smart, and certainly smart enough to understand the simple math of FPTP system. This keeps their ideological preferences masked until they see an opportunity for realignment and then they jump on it. That is why pollsters have real power, for better or worse.

    Simultaneous breakout parties on the right and left give people whose ideologies are not represented by Con or Lab their best chance in a generation to endorse their preferred party. Con and Lab will try to squeeze this by fear as they always do. They may be successful but if no party is over 30% in the polls by May then as many as 20% of constituencies will be three way lotteries that no one can predict. So it will be pretty hard for the traditional parties to squeeze.

    Frustrating times for those in the prediction business, I agree. But no need to assume people are stupid to explain it.

  26. I think when tonight’s poll comes out it should cheer poor ole P Haines up and get us away fae the monotonous Scottish stuff.

    https://twitter.com/sun_politics

  27. Swing back is back, possibly.

    Just a bit of fun, possibly.

    Night All.

  28. @ChrisLane1945

    “I smell disaster, that is why.”

    On Bournemouth beach? What’s happened? Is Poole aflame?

  29. Sun Politics @Sun_Politics · 1m 1 minute ago
    YouGov/Sun poll tonight: Labour at 5 year low, Greens at a new high. CON 32%, LAB 30%, UKIP 15%, GRN 10%, LDEM 8%

  30. Is it the “S” word ? :-)

  31. CATMANJEFF
    However, we have fixed terms Parliaments now, so this is entirely hypothetical.

    Hypocritical more like. Fixed Parliaments could be suspended in 24 hours if a Government wished to. Isn’t that the “beauty” of an unwritten constitution?

    Not that I expect it to be any more than the apologia we’re all expecting, BTW

  32. my my,
    Where is Howard to espouse the need for 4 in a row?

  33. Richard..

    Do you prefer your polls to be landscape or portrait?

  34. Dreadful figures for both the main two parties again, particularly for Labour in this one.

    Shouts from Tory HQ: “Come on you Greens and SNP!”

    Shouts from Labour HQ: “Come on the UKIP!”

    The flight from the two main parties continues apace. I’ve called this election to come a relegation battle, and the evidence for my view gathers as each poll points to ever lower combined figures for Labour and the Tories. Only Populus still flying the flag now for anything approaching 70%.

  35. jim jam,

    look everyone knows about margin of error etc. but labour scoring their lowest score on yougov in 5 years is significant, i am sorry.

    The fracturing of the left with the surge of the greens is a fascinating scenario now. I still think labour will edge largest number of seats, but they will not get a ringing mandate. I think a weak centre-left government propped up by the snp, greens and lib dems would destroy the left for years….Bring it on!

  36. “Will all respect, I think you are quite wrong on this particular point.”

    I am not convinced that so many people reached the same conclusion independently – that’s all.

    There is an element of herd instinct in lots of reactions to political events. You very often hear the public, in interviews, simply regurgitating lines that have already been aired in the media.

    I am quite certain that there are many now saying they would vote UKIP or Green who would be strongly against some of either party’s main policies if they were shown them without links.

    I can’t prove I’m right and you can’t prove I’m wrong.

  37. JIM JAM

    I thought he argued for three ?

    Anyway-Wed’s Poll is going to be breathlessly anticipated. :-)

  38. Pthiers
    “Voters are pretty smart, and certainly smart enough to understand the simple math of FPTP system.”

    Interested voters perhaps but I’m not convinced that you would get much agreement if you asked a random selection of the electorate if they understood the impact of FPTP on their constituency.

    Maybe one of the telephone pollsters will ask those polled one day. Online pollsters probably already have a higher incidence of politically aware people.

  39. James Peel,

    If you can’t be polite, why say anything at all?

    This is obviously very good for the Greens, but I do wonder (as was mentioned up thread) how much is down to policies (I suspect, like UKIP, that few voters of any party know much about their policies) and how much is down to coverage/perception of strength.

    We’ll see, but I’d be interesting to work out what their potential cap is in terms of support.

  40. I heard a tory strategist, obviously biased, saying he didn’t think labour would get more than about 30% in the general election….doesn’t look that implausible now, i have say!

    the funny thing is that Con 32 Lab 30 represents a 2.75% swing C to Lab, which gets labour most seats on a UNS basis….

  41. MrN

    This was one of the reasons I asked AW about the report button earlier. :/

  42. The charge within the left to the Greens is like all the animals in London and and Edinburgh Zoo’s charging Noah’s ark.

  43. mr n

    My feeling is that all of this is pre-match skirmishing: sound and fury, signifying nothing.

    Still find it weird that people start seriously analysing the GE result based on a 32/30 split taken from one poll.

    But then, people are I s’pose.

  44. Posted too soon.

    The charge within the left to the Greens is like all the animals in London and Edinburgh Zoo’s charging Noah’s ark.

    Labour are losing the left in Alba and England.

    COLIN..Notice I never used the “S” word?

  45. @Allan

    Not fussed, whoever tweets first. Prefer Sun Politics as that is from the source, but they were too slow tonight.

    @R&D
    I think its the fact that most people ignore politics most of the time, unless something interesting comes along. Once they start noticing a lot of stories ‘x’ is surging, that becomes interesting, they start paying attention again, read a bit more and then apparently change their mind en masse.

  46. Now the left wing vote is being split. Almost as if Cameron had some sort of agenda by talking up the greens.

  47. If you think that “Parties always do what is in their best interests” you a) have paid little attention to the recent history of Northern Ireland, b) have paid little attention to general history everywhere else. Political Parties tend to do what they want to do, and sometimes that coincides with what is best for the party.

    Going to lay this out as carefully as I can with regards to Northern Ireland. NI Sinn Féin and the DUP are sectarian parties on the Nationalist and Unionist sides of the dispute. While the SDLP and Alliance are formally opposed to sectarianism. This means the SDLP and Alliance can caucus with Labour and Liberal Democrats respectively, because that’s not seen as giving power to Sectarian-Unionist or Sectarian-Nationalists. The same can not be said for SF or the DUP.

    Now, Sinn Féin don’t take up their parliamentary seats, and have been iron clad on that. There’s been no suggestion they would ever take the oath, and there’s been no suggestion that parliament will be reformed to not require it. So supposing that they ‘might’ at some point, and then extrapolating on from this that they ‘might’ support a Labour Government, in order to justify the supposition that the DUP could support a Conservative one, is simply making stuff up to suit your argument. It’s not only a jump to assume that they might at some point drop their commitment to not taking an oath, or not be required to do so, but it’s another jump to say they would support any Westminster government.

    I know you really really want to construct scenarios that provide more paths for the Conservatives back into government, but a coalition with the DUP would be a desperate act that would risk the stability of Northern Ireland by putting Sectarian Unionists back on the top of the heap. It would be insane.

  48. R&D is of course right that momentum and media (including social media)

    Lots of our new members know each other, and have clearly been talking to friends about joining.

    But even if they are joining (or saying they will vote Green) because we are flavour of the month, that really doesn’t mean that they are unaware of Green policies or values. Far more likely is that they are part of the substantial section of the country that has always been Green-minded, but have not voted for them – Green policies have polled ahead of the party for years.

    The other point is that Green party is determined by its membership, which is now dramatically different from what it was 3 months ago. There’s not going to be a national conference before the election, but future Green policy is as likely to be that of the new joiners as the older ones.

  49. New thread

  50. Also, viewing Sinn Féin solely through the view of how many NI MPs they return, betrays a complete misunderstanding of Sinn Féin. NI Sinn Féin and Republic of Ireland Sinn Féin are the *same political party*. So it’s absurd to make the argument that it’s in their “best interests” to put Westminster politics above Irish Republic ones.

1 2 3 4