Tony Twyman, who died last year, was the man behind much of the mechanics of TV and radio viewing figures, most notably as technical advisor for BARB viewing figures. In broader market research he is more widely known for coining Twyman’s Law – “Any figure that looks interesting or different is usually wrong”. The point is, of course, that strange and unusual things in a single poll are more likely the result of sample variation or error than some amazing shift in public opinion, and you should be cautious of them before getting excited (My colleague Joe Twyman likes quoting it without attribution in the hope people will jump to conclusions… not so fast!).

Anyway, today we have a classic case. Two polls that look interesting when compared to recent averages, but which are both probably no more than the result of normal sample error.

Today’s twice weekly Populus poll had figures of CON 32%, LAB 37%, LDEM 10%, UKIP 13%, GRN 4% (tabs). The five point Labour lead is the biggest Populus have shown since November, their 37% share the largest any company have shown since November. Labour resurgence?

Lord Ashcroft’s weekly poll however had figures of CON 34%, LAB 28%, LDEM 8%, UKIP 16%, GRN 8% (tabs.) A six point Conservative lead, by far the best poll for the Tories from any company for several years (the largest Tory leads up to now were the last two MORI polls, which had them three points up). Tory surge?

Of course the actual answer is that there is probably neither a Labour nor a Tory surge, that both of these changes are probably just down to sample error and that people should watch the overall trend across multiple polls, not get overexcited about individual polls. If the figures in one poll look strange or unusual, it’s probably wrong.

In some ways it’s quite nice they come on the same day, as it should stop people getting too excited over an outlier in just one direction. On the other hand, it does tend to produce lots of confused comments about how polls can be accurate when they are showing both a five point Labour lead and a six point Tory lead. Bottom line for those who are confused, part of it is down to pollsters using slightly different methods (in this case, the way Populus weight their polls tends to produce a bigger share of the vote for the main two paries than does Ashcroft). A bigger chunk will be simple margin of error – polls are not precision instruments and no one who understands them would claim they are. They are randomish samples of about 1000 or so people. The quoted margins of error are about plus or minus 3% (though given response rates, weighting effects and that polls are not pure random samples, that’s a bit of a polite fiction). That means if the real position was Labour and Conservative tied on 33%, you would expect to see the Conservatives ranging from 30% to 36% and Labour from 30% to 36%, and while the results would tend to be clustered around the middle of that range, random variation could reasonably vary between a 6 point Tory lead and a 6 point Labour lead. Taken alone and in isolation, it does mean an individual voting intention poll isn’t that useful… which is why you shouldn’t look at them alone and in isolation – watch the trend.


200 Responses to “Contrasting Populus and Ashcroft polls”

1 2 3 4
  1. My yougov prediction based on nothing at all:
    Con 33%
    Lab 32%
    UKIP 18%
    Lib 8%
    Grn 8%

  2. @Statgeek

    My hunch on those figures would be a Tory/Ukip/DUP coalition.

  3. What time should t’yougov poll be out?

  4. @Statgeek

    Sorry i meant a Tory/LD/DUP coalition.

  5. Bramley

    Comres polling on ‘issues’ for ITV:

    The National Health Service has overtaken immigration as the top concern for voters, the latest ComRes poll for ITV News has revealed.

    Interesting list.

    I don’t think we can just shrug our shoulders over this one and say “ComRes”. Because last week’s YouGov tracker found something very similar. On the “most important issues facing the country at this time” question, Health went up 13 points from 33% to 46%:

    https://d25d2506sfb94s.cloudfront.net/cumulus_uploads/document/9qu9ky90oi/YG-Archive-Pol-Sun-results-060115.pdf#page=2

    It didn’t top the list – Immigration and the Economy tied on 52% – but it’s a massive jump.

  6. @Jasper22: You are Pressman and I claim my £5.

  7. Tonights Yougov

    CON…36%

    LAB…31%

    UKIP…14%

    LIB…7%

    GRN..5%

    OTHERS..7%

  8. The YouGov is probably another outlier – there’s no reason for a sudden Tory surge, is there?

  9. It;s okay you can come out from hiding…My last post was my prediction.

  10. Where is that from Allan?

    Your guess?

  11. france?

  12. Who are the others on 7%?

  13. @ Alan

    Phew – you had me worried there! Could have made that clearer!

    I see no reason to expect anything other than a very boring YG poll.

  14. I think the trend is toward strange polling results.

    If that Yougov is correct and not a prediction, then “wow”

  15. okay, I retract my “wow”

  16. DAVID

    I think tonight’s poll will be neck and neck or a small lead for someone.

  17. @CMJ

    Fingers on the buttons. Are you ready?

  18. The Sun are tweeting it’s a sensational poll tonight.

    The little teasers!

  19. Come on Allan. For those of us who are left leaning and have high BP that was a bit too much to take.

  20. I’m hearing rumours. If true, then it’s unbelievable.

  21. YouGov/Sun poll tonight – Labour lead by one: CON 32%, LAB 33%, LD 6%, UKIP 17%, GRN 6%

  22. YouGov/Sun poll tonight – Labour lead by one: CON 32%, LAB 33%, LD 6%, UKIP 17%, GRN 6%

    Yawn

  23. CON 32%, LAB 33%, LD 6%, UKIP 17%, GRN 6%
    https://twitter.com/sun_politics

  24. Get in!

  25. PETER BELL

    Sorry about that I should had said that was my prediction lol

  26. What a poll! You first, CMJ, or me?

  27. Heh, three people posted.

    Rather boring. UKIP quite high though.

  28. Lib Dem figure seems high, but the polls have been strange.

    Relief for Labs.

  29. Is that it? Night night nothing to see.

  30. Well if it is unbelievable we should follow Anthony’s advice and wait until it is
    confirmed by following polls.

  31. Joking apart, it would appear that Populus and Ashcroft have delivered a couple of right old post-Christmas turkeys, both on the same day too.

    Labour win 2-1 on the day, though!

    :-)

  32. Blimey! I have just taken a Valium pill – now I needn’t have bothered.!

  33. Why do people bother predicting something that has such a large MOE? (Most of us knew BTW Allan!)
    So net Lab v Con if pre average of 3 is a dead heat.

  34. So that is the unbelievable poll?Some people have a rather sad sense of
    humour.

  35. YouGov obviously a rogue poll. :p

  36. Crossbat

    “Joking apart, it would appear that Populus and Ashcroft have delivered a couple of right old post-Christmas turkeys, both on the same day too.”

    Isn’t that really Populus and Populus (or populous if preferred)?

  37. UKIP still high, so it confirms that Sunday’s 18% wasn’t a random blip.

    It will be interesting to see how long this lasts, and whether there is any long-term residual.

  38. CROSSBAT11
    Joking apart, it would appear that Populus and Ashcroft have delivered a couple of right old post-Christmas turkeys, both on the same day too.
    Labour win 2-1 on the day, though!
    ______

    Well the aggregate over the last 3 polls will have labour leading by 0.5% but that may change when rounding.

    STATGEEK??

  39. Last comment before an early-ish night. UKIP on 17% in tonight’s poll and maintaining their post Paris momentum. It’s interesting to see how easily they pick up support and how resilient their 10-12% core is. They really are going to be the mother and father of all spoilers in May and if the two main parties carry on campaigning in the way that they’ve just started, then I think their balloon will just keep inflating. As will the Greens and SNP.

    32v32 Tory/Labour relegation battle all the way to the line is looking the likeliest bet to me.

  40. JIM JAM

    “(Most of us knew BTW Allan!)”
    ___

    Hmm but I feel for those who have taken the jump lol

  41. Stat geek,

    “I can’t believe I used valuable seconds of my life and Internet bandwidth for that link.”

    I can.. You like Stats and your a geek!

    Peter.

  42. We are as near to Polling Day as to 21st Septemberlast year – ie start of Labour conference.

  43. Three poll average shows a Con / Lab tie on 32.66r:

    http://www.statgeek.co.uk/wp-content/uploads/2015/01/three-day-average.png

    It doesn’t get closer.

  44. One good for Labour, one good for Tories, one sodding boring. I suspect the sodding boring may be right on the money. Off to do some court reporting in the morning so I’ll be off to bed.

  45. The Sun clearly thinks it’s “unbelievable” that Labour are still ahead by a whisker. ;-)

  46. STATGEEK

    You couldn’t get a fag paper between the two.

  47. The big 3 on 71%.

    That’s low……2.5 party politics is certainly looking poorly. Come the GE it may need a defibrillator.

  48. Ukip maintaing their post-France bounce, just as they were starting to fall too.

  49. Guess which poll is reported in the Telegraph?

    A rare turbulent day of polling, eventually settling back to somewhere around ‘no change’. YG continue with the trend of showing a slight tightening, after a slight widening, so take your pick from the other two.

    There might just be two tiny bits of cheer for Labour in recent polling though, although I wouldn’t stake my mortgage on this.

    If the NHS really is moving to the top of the concerns agenda, that would probably be good news for Labour. The other possibly positive factor for them is the detection of the merest drift back from the SNP in Scotland. I haven’t checked the numbers, but I recall a day or so back various people on here suggesting a 1% swing back.

    If this is a real phenomena (which is, I would say at this stage a pretty big if) it sounds like bad news for Jim. However, 1% a month for 4 months would see the lead cut by 8%, with Labour perhaps getting into low 30’s and SNP the high 30’s.

    Still a very good result for the SNP, but surely this would save a small bundle of seats for Labour.

    Very early days on this, but both the issues polling and the Scottish stuff look like they will be worth watching.

  50. If the Tories and Labour keep being neck-and-neck, an age-gate may have to be added to this site.

1 2 3 4