Two new polls out today, both good for Labour. Populus this morning had toplines of CON 31%, LAB 38%, LDEM 9%, UKIP 14% (tabs here). Lord Ashcroft’s weekly poll has topline figures of CON 27%, LAB 34%, LDEM 11%, UKIP 15% (tabs here).

The Ashcroft poll comes after a poll last week that showed the Conservatives 2 points ahead, and has naturally provoked some comment about volatility. In one sense it’s fair comment – Ashcroft polling has been volatile. In another sense it’s not – Ashcroft’s polling hasn’t necessarily been any more volatile than you should expect, it’s just that we sometimes have slightly unrealistic expectations of how accurate a poll of 1000 people should be!

The standard margin of error on a poll of 1000 people is plus or minus 3 points. However, voting intention figures aren’t based on the whole sample, only on those who give a voting intention – in a phone sample of 1000 that’s typically 500 or so people, giving a margin of error of plus or minus 4 points. I should add that the margin of error is based upon what the margin would be in a pure random sample. This is very much a polite fiction – no voting intention polls are actual pure random samples. Many are from internet panels, even quasi-random phone polls aren’t actually random because of low response rates. Weighting effects would also change the actual margin of error.

Looking at Ashcroft’s nine regular polls to date, the average level of Labour support has been 33%, and all nine polls have been within 2 points of this. The average Lib Dem support has been 8.5%, and all nine polls have been within 2.5% of this. What’s made them look erratic is the level of Tory support, which has averaged 29%, but has varied between 25% and 34% – two of Ashcroft’s Tory scores have differed from the average by 4 points, one by 5 points. This assumes that there hasn’t been any genuine movement in Tory support, when it’s possible there has. Ashcroft’s highest Tory score came in his first poll in mid-May, at a time when ICM also showed a Tory lead and YouGov a neck-and-neck. Ashcroft’s lowest Tory score came just after the European results when UKIP had a post-European election boost.

Bottom line is that while Ashcroft’s polls look erratic, they probably aren’t much more erratic than we should expect from topline figures based on 500 people. There isn’t anything strange about their methodology, nothing odd going on, it’s just the normal limits of how precise polling with a given sample size can be. And it’s a useful reminder of why we shouldn’t read too much into individual polls, and it’s the underlying trend and average that count.


542 Responses to “On volatility in polling”

1 9 10 11
  1. Today’s YouGov:

    Lab 37
    Con 34
    UKIP 12
    LDem 8
    Green 5
    SNP/PCY 3
    BNP 0
    Others 1

    Approval -21

    All of the positive statements (what kind of country do you want to see?, etc) in Labour’s favour; as opposed to the questions asked about who would deal with particular issues best the other day, where I think the Conservatives were ahead on every issue apart from the NHS.

    Report comment

  2. @ LeftyLampton

    Glad you put the Cocteau Twins first- in the words of some journalist “if you die and don’t hear this playing you’ve probably gone to the wrong place”. You missed out Altered Images and Orange Juice though.

    Report comment

  3. @ Lefty Lampton

    … And I just missed out the Shop Assistants too!

    Report comment

  4. SHEVII

    I did miss them. It was late. And, unforgivably, I missed The Associates too.

    Report comment

  5. Shevii,

    A person of my heart the Cocteau Twins are uniquely supreme and along with Durutti Column staples in my car.

    I recall hearing that one of the Tories Celebrity supporters was Elizabeth Fraser which surprised and saddened me until it became apparent that the great ‘Liz’ shared her name with a kids TV presenter.

    Yes early Orange Juice as well what a band before they went a bit souley.

    Report comment

  6. I believe not everyone gets the same adverts at the top of this page but I have one currently that says, “Water Secret takes Fluoride out” Why would you want to take fluoride out of the water unless you were a loony conspiracy theorist?

    Report comment

  7. @COUPER2802

    “But my reasoning is that it doesn’t make sense to match the samples to the make-up of a previous govt election because the elections are too different. It makes more sense to use a sample matching the general population and then apply a turnout model which I think Mori does – though I dont think they apply a turnout model other than likelyhood to vote. The Problem with the turnout model is that it very difficult to predict turnout of different groups and the wrong model will give the wrong result. So my conclusion is that the polls are not going to be very accurate.”

    You do realise this is pretty much exactly what Republicans were saying about all the polls leading up to the last presidential election?

    The result? The polls were right all along and Obama was re-elected as expected.

    Report comment

  8. @C2802

    Surely You Gov’s weighting to 2010 votes doesn’t intrinsically overweight Labour voters in the current referendum VI: it just matches the political spread of the sample to a known data point? Presumably if they get an accurate sample based on 2010 voting, they should then pick up where people have moved since 2010 – notably in this case, Labour voters then intending to vote YES now.

    The more turbulent the churn since then, the more scope for error, but I can’t see why weighting to 2010 would be expected necessarily to overrepresent Labour supporters in current referendum intentions? (Though I can see it would if they are picking up the “wrong” sub-sample of 2010 Labour.)

    Report comment

  9. You tell us Norbold.

    Report comment

  10. @: Lefty Lampton

    I’ll take your Associates and raise you a The Fire Engines. I can keep this up all day you know!

    @ Jim Jam

    Wow- you meet all sorts on here but I didn’t expect anyone on UKPR to have even heard of Durutti Column. I guess we’re all getting on a bit so it’s maybe it shouldn’t be a surprise that some of us share a common 1980’s music thing. I’m the proud owner of about 150 live recordings of the Cocteaus and a spattering of Durutti Column as well- a bit excessive but you never know when you’re trading what is going to turn out to be a classic gig.

    Anyway seems like we have enough people for a UKPR 1980’s indie disco and I’m sure I read somewhere that Anthony likes the Smiths so can always throw in a few of those for him. They’re the only ones that get anyone dancing anyway…

    Report comment

  11. I notice we had a visitation from OldNat and Couper in the watches of the night. Very Macbethian if I may say so.

    Report comment

  12. NORBOLD
    “Why would you want to take fluoride out of the water unless you were a loony conspiracy theorist?”
    Read on, including the Alzheimers ads – lengthy stuff until you get to it – to discover the connection with an attack on Obamacare, socialised medicine, immunisation and other things neolibberish and Tea Party.

    Report comment

  13. @DRMINBLES

    What election was that? At the last presidential election the polls were WRONG giving the republicans false hope. You are re-writing history. In fact it was the ‘turnout model’ that the pollsters applied which caused much of the problem.

    Report comment

  14. @Muddy Waterd

    I think Survation’s argument was that the sub-sample 2010 Lab 2011 SNP is causing the problem and is probably small, in their case 16, and up weight considerably causing the issue. YouGov don’t give details of that weighting but it is likely to be problematic.

    In any case we will soon see who is correct when the next few YouGovs are done. If they start to move towards Survation’s results. If not we will have to wait till 19th Sept

    Report comment

  15. @Ewan Lightfoot

    Please don’t make this personal there is enough of that on Twittet

    Report comment

  16. Weighting correctly, accounting for false recall, shy voters and the rest is hard enough but in Scotland with split ticket voting between GEs and Holyrood Elections and other specifics the task for pollsters imo is ultra-tough.

    Luck rather than skill or judgement might determine who is closest.

    Report comment

  17. @COUPER

    “What election was that? At the last presidential election the polls were WRONG giving the republicans false hope. You are re-writing history. In fact it was the ‘turnout model’ that the pollsters applied which caused much of the problem.”

    No, all the mainstream polls were correct except for Rasmussed which leaned too far Republican.

    The final average of polls exactly predicted the result and correctly predicted all states too, quite remarkable really.

    It was internal Republican polling which told them Mitt was going to win, because they changed the weighting as they were convincing the weighting applied to the other polls was too ‘pro Obama’ (for the same reasons you think YouGov are pro Labour).

    Report comment

  18. Shevii – I think Vini Reilly was the main collaborator with Morrissey on his early solo stuff so AW may well have some Durutti Column stuff (kind of).
    Did not know Fire Engines were Scottish – now Candy Skin there’s a classic.

    Report comment

  19. “Why would you want to take fluoride out of the water unless you were a loony conspiracy theorist?”

    ———

    Maybe if you’re a dentist?

    (Fluorine is also quite useful in using Thorium, though dunno how I know that…)

    Report comment

  20. A decent write up on the survation poll here:

    http://www.betternation.org/2014/07/exclusive-july-holyrood-poll-by-survation/

    Last para:

    “If you’re still not sure who to back in the battle of the pollsters, here’s a wee graph from @bgreysk on Twitter (precedes this month’s Survation result). The trend lines are the best guide, and from that YouGov look like the complete outlier. On this evidence, I think Ladbrokes would be easy to take to the cleaners given they’re offering 7/4 on Survation to be closest, but any bets are of course to be made at your own risk.”

    It looks like the Greens are probably set to do better, whatever happens.

    Report comment

  21. @ALLAN CHRISTIE

    No one likes Westminster (possibly not even Westminster). It’s an argument for devolving power from the centre, not for breaking up the UK, though. And it’s not as if Scotland hasn’t had its own share of child abuse issues.

    Report comment

  22. Couper – actually the polls at the last Presidential election in the US were pretty damn good – hence the ability of those aggregating poll data like Nate Silver to make such good projections.

    A couple of pollsters did badly, most notably Gallup – and because they are so high profile and famous it got a lot of coverage (it was indeed probably their turnout model). Most others did very well.

    Report comment

  23. On the living wage thing, and the hope that closing in on full employment may also raise wages…

    … Trouble is, that this can simply mean an opportunity for purveyors of essential goods to raise prices, soaking up the extra wages.

    Wage rises can be inflationary, thus action on prices of essentials can be rather important. And it can also make our firms more competitive when lower prices on essentials reduces wage pressures and hence costs…

    Report comment

  24. New Populous: Labour 36(-2) Con 34(+3) LD 8(-1) UKIP 12(-2) OTHERS 10(+2)

    Report comment

  25. @CARFREW

    ‘“Why would you want to take fluoride out of the water unless you were a loony conspiracy theorist?”

    ———

    Maybe if you’re a dentist?’

    What, a loony dentist?

    @JOHN PILGRIM

    That makes loony sense now. Thank you.

    Report comment

  26. Couper. ” ‘Twas a rough night “.

    Fair enough, although OldNat’s previous post had been a straightforward ad hom on me, accusing me of “speed reading” and “antipathy “.When all I wanted was an explanation of why he had compared Peter Kellner with the fraudulent statistician Sir Cyril Burt.

    Report comment

  27. @Norbold

    “What, a loony dentist?”

    ———-

    Especially so in that case, one would think…

    (Fluorine can be used not only to provide the molten salt in the reactor, and also to extract Uranium 233 from the Thorium, since you asked…

    …Well, ok, maybe you didn’t ask, but you were wondering, weren’t you? And if you weren’t wondering, think what abundant energy could mean for the voters of Tendring!!…)

    Report comment

  28. @Ewen

    You mean Oldnat didn’t surface to praise the commendable knowledge of Scottish music on this board, esp. Lefty et al.? Thought it might happinate Scots given the frequent complaint that we don’t know as much as them about all the details of their Scottishly lives…

    Report comment

  29. @Couper: “In any case we will soon see who is correct when the next few YouGovs are done. If they start to move towards Survation’s results. If not we will have to wait till 19th Sept”

    Indeed. Nevertheless, I think Kellner’s “red nat” hypothesis” has at least provoked some kind of proper technical discussion and identified a potential source of the divergent house effects in referendum polling – in place of both sides’ tendency to point vaguely to flawed methodology or partisan motivations.

    Report comment

  30. rogerh

    @ALLAN CHRISTIE

    No one likes Westminster (possibly not even Westminster). It’s an argument for devolving power from the centre, not for breaking up the UK, though. And it’s not as if Scotland hasn’t had its own share of child abuse issues
    __________

    Of course Scotland has its own abuse problems but I’m on about the so called mother of all parliaments. It’s a national embarrassment to us all.

    And yes the indy ref is not about child abuse at Westminster, I never said it was but for me personally that has hardened my own case for voting yes.

    I never speak for others unless I post fact.

    Report comment

  31. @Muddy Waters

    The flaw in Kelner’s is assuming that the default is Labour. It is perfectly possible that SNP folk lend their vote to Labour in Westminster to prevent a Tory government. Not the opposite way round as in Kelner’s hypothesis. It
    Because we don’t have the weighings of the two groups in the SNP sample we can’t tell what effect weighing them separately had on the result.

    I am a Labour voter all elections but am leaning Yes. So it is not particular bias I am just interested in why the polls diverge. In fact l think at the moment the polls are irrelevant as the real campaign will start August 5th with the Salmond-Darling debate.

    Report comment

  32. @Ewan & Carfrew etc. on Scottish musicians

    What none of you seem to have commented on (though my look through the thread has been cursory and apologies if I’m wrong) is the effect the music you were talking about (and other artists around at the same time) had on the continuing rejuvenation of Scottish culture and self awareness.
    “It is noticeable that those Scots who are too old to have been involved in the ‘scene’ in the late 1970s, the ’80s and ’90s are those least likely to vote Yes”. Discuss.

    @RogerH
    You mean proposals such as those made by Murdo Fraser?
    (see below)

    @JimJam
    Yes. The idea that interpretation of Scottish Indy VIs ought to be based on GE 2010 results alone is sheer nonsense. Far too much water has passed unser the bridge since then. And this is particularly true of Labour voters, who are in a real bind: EM and EB steadfastly refuse to move away from New Labour positions on anything that is going to cause the London right wing press to kick up a fuss – i.e. anything that really matters; but there is a hope that a new government in Westminster will produce something other than more economic misery; on the other hand, as one questioner put it at a public meeting I was at last week, “I don’t understand Scottish Labour’s inability to see that voting Yes will give them real power, instead of being at London’s beck and call.”

    ‘No’ will win – for now. But unless Murdo Fraser and his ilk are listened to, then we’ll be going through all this again within the next ten years.

    Report comment

  33. under the bridge, of course

    Report comment

  34. Carfrew

    I see no one has mentioned the brilliant guitar ‘bagpipes’ music of Big Country, a personal favourite of mine from the early Eighties, which I still play in the car from time to time.

    Report comment

  35. New thread.

    Report comment

  36. Couper: “The flaw in Kelner’s is assuming that the default is Labour.”

    I’m not sure. I think this is a bit of a red herring in terms of the polling. I don’t think it matters much which way round you express their allegiances. If Kellner’s VI data are right there is a statistically significant difference in indy ref VI between 2010-labour-to-2011-SNP voters and other 2011 SNP voters. There may be an emotional or political significance in understanding their baseline allegiance; but in terms of indyref polling, the important thing is which of the companies has the correct size/ weighting for this group.

    As you suggested before, the jury is still out on this.

    Report comment

  37. Did anybody mention Loyd Cole, the Bluebells or Bronski Beat/the Communards?
    And Annie Lennox of the Eurythmics was also gigging around that time – in the Tourists.
    Lefty did mention my personal favourites: The Jesus & Mary Chain (featuring Bobby Gillespie of Primal Scream).

    Report comment

  38. @Amber

    I had the Lloyd Cole album, the one with “Perfect Skin”.

    @John B
    We probably took the effect of music on culture as read, as that is what it tends to do, and some of us have noted the rather salient correlation of oil with the Independence thing. (I might be guessing, but I think Scottish music predates the oil thing…)

    @Ewen
    Yep, and I’m amazed John B didn’t mention Lulu… I mean she even had a hit in 1993 with a song called “Independence”…

    Report comment

  39. pre-dates, not predates, (though having said that…)

    Report comment

  40. @ Carfew

    Well if you are going to move the goalposts into the 1990’s then we’ve got Belle and Sebastien and Camera Obscura.

    Just Old Nat left to post I think and hoping he’s going to go for the Exploited! I saw them once and it was a very “powerful” gig and one of the best punk bands “musically” but difficult to justify their existence with the mindless violence outlook however tongue in cheek/getting things out of your system it was meant to be.

    Report comment

  41. @Shevii

    Well, Lulu was active in the Eighties, and even her Sixties output clearly betrays an awareness of the Independence issue. Songs like “Try to Understand”, “Let’s Pretend”, “Best of Both Worlds”, “Make Believe World”, carry within an instinctive grasp of the movement for change, and in this context, songs like “Stealing my Love from me”, “Leave a Little Love”, “Don’t take Love For Granted”, and “Love is the Answer” take on a new complexion when one considers “Love” to be a euphemism for “oil”.

    Of course it’s possible that by 2002 and the album “Together”, Lulu’s stance on the matter may have further developed…

    Report comment

1 9 10 11