Two new polls out today, both good for Labour. Populus this morning had toplines of CON 31%, LAB 38%, LDEM 9%, UKIP 14% (tabs here). Lord Ashcroft’s weekly poll has topline figures of CON 27%, LAB 34%, LDEM 11%, UKIP 15% (tabs here).

The Ashcroft poll comes after a poll last week that showed the Conservatives 2 points ahead, and has naturally provoked some comment about volatility. In one sense it’s fair comment – Ashcroft polling has been volatile. In another sense it’s not – Ashcroft’s polling hasn’t necessarily been any more volatile than you should expect, it’s just that we sometimes have slightly unrealistic expectations of how accurate a poll of 1000 people should be!

The standard margin of error on a poll of 1000 people is plus or minus 3 points. However, voting intention figures aren’t based on the whole sample, only on those who give a voting intention – in a phone sample of 1000 that’s typically 500 or so people, giving a margin of error of plus or minus 4 points. I should add that the margin of error is based upon what the margin would be in a pure random sample. This is very much a polite fiction – no voting intention polls are actual pure random samples. Many are from internet panels, even quasi-random phone polls aren’t actually random because of low response rates. Weighting effects would also change the actual margin of error.

Looking at Ashcroft’s nine regular polls to date, the average level of Labour support has been 33%, and all nine polls have been within 2 points of this. The average Lib Dem support has been 8.5%, and all nine polls have been within 2.5% of this. What’s made them look erratic is the level of Tory support, which has averaged 29%, but has varied between 25% and 34% – two of Ashcroft’s Tory scores have differed from the average by 4 points, one by 5 points. This assumes that there hasn’t been any genuine movement in Tory support, when it’s possible there has. Ashcroft’s highest Tory score came in his first poll in mid-May, at a time when ICM also showed a Tory lead and YouGov a neck-and-neck. Ashcroft’s lowest Tory score came just after the European results when UKIP had a post-European election boost.

Bottom line is that while Ashcroft’s polls look erratic, they probably aren’t much more erratic than we should expect from topline figures based on 500 people. There isn’t anything strange about their methodology, nothing odd going on, it’s just the normal limits of how precise polling with a given sample size can be. And it’s a useful reminder of why we shouldn’t read too much into individual polls, and it’s the underlying trend and average that count.

542 Responses to “On volatility in polling”

1 8 9 10 11
  1. Colin
    It’s not that I don’t understand what you’re saying about what was on the telly today, it’s just that I don’t give it anywhere near the salience you appear to.
    Again, smiley things all round !

  2. AMBER

    @”I’d say: Definitely.”

    I’d say it depends whether you’re going to get it, or you’re going to pay for it.

  3. EWEN

    Thanks-you may well be correct on salience.

  4. @ Colin

    I’d say it depends whether you’re going to get it, or you’re going to pay for it.
    It’s not as simple as that. If you or a member of your family can expect to go from minimum to living wage then I’d say that would definitely be salient & impact your voting intentions.

    People who wouldn’t be directly effected would simply think of it as a general, less salient, issue.

  5. It seems unlikely that Maude et al with succeed in making the teachers, Fire Service and public and civil servants into the “enemy within” trying to bring down the elected Government a la Scargill in the eighties.

    It’s likely to look more like stirring up industrial unrest for the sake of it.

    We’ll see from the polls. I’m not at all sure that people should be reminded of the eighties at all, just about now.

  6. @ Colin

    I actually think you made a pretty valid point about public sector pay and the possible effect on voting intention. I suspect though this is more likely to be a potential fault line in a 2015 Lab government than it will be in the run up to the election.

    In a tight election and with a pretty lethargic electorate a clear manifesto promise on public sector pay would probably make a lot more voters get out and vote for Labour who might not be bothered otherwise.

    However with Labour at risk from accusations of spending too much they won’t want to make a commitment and will rely on those voters just turning up. Equally I can’t see the Tories pushing Labour on what they will do because that will highlight to that very same group of voters that the Tories aren’t going to offer then anything.

    In the next parliament I think Labour will be greatly at risk if public sector workers are not keeping up with average pay rises.


    “The laws are currently on the statute book and have been since a 2006 EU retention of data directive (which followed terror attacks in UK and Madrid)”

    It’s an EU Directive, not statute (introduced largely at the instigation of the UK under its presidency at a session chaired by the Home Secretary). It’s up to individual countries how they implement directives.

    ” the ECJ have ruled against an EU directive.”

    Which is one of their functions. Be pretty pointless if they never overruled any of them. And one of its criticisms was that there was nothing to prevent the data from being passed to countries outside the EU.

    “It remains free for all EU nations to enact whatever anti terror surveillance legislation they want.”

    Subject to other laws, including the Human Rights Act.

    “That’s what we are doing now, reinstating the EU 2006 law”

    There was no 2006 law. The government is rushing through an entirely new law to legitimise the parts of the directive that the ECJ ruled incompatible with Article 7 of the Charter of Fundamental Rights.

  8. As people interested in polling , what do all you think about only people being allowed to enact any policy if they have the support of 50% of the total electorate?

  9. Good question, BFIELD.

    I think only those politicians who say that unions must have over 50% support for a strike should have to get over 50% of their electorate before they are allowed to take part in voting on legislation.

  10. ewen

    “Again, smiley things all round ! ”

    If there is ever a world shortage of “smiley things” [wottever the bleedin’ hell they are] we will know who to blame.

  11. re a living wage

    “I’d say it depends whether you’re going to get it, or you’re going to pay for it.”

    So you don’t feel that people should be moved by the principle of it Colin?

    I realise there is a “real” world out there and it contains selfish people and skivers but the opposite of a living wage seems to be to accept that people in the UK, in the 21st century, should expect to work for a wage that they CAN’T live on.

    Which then leaves it to those of us paying taxes to fill the shortfall in some complicated and stupidly expensive redistributive system.

    I imagine a sensible wage system would mean more UK people taking on work also.

  12. SHEVII

    @”I suspect though this is more likely to be a potential fault line in a 2015 Lab government than it will be in the run up to the election.”

    If that is the case-ie there is in fact a disparity between TUC expectations, and EM/EB intentions-then I think Cons will not be slow in trying to raise its salience before the GE.

    It will all depend on Labour’s actual; manifesto-and the noises off from Publ;ic Sector Union bosses.

  13. By the way forget “it’s the economy stupid,” I think it could well be “it’s the NHS stupid.” In 2015.

  14. PAUL-I wasn’t talking about the rights or wrongs of Public Sector pay policy-we aren’t allowed to do so.

    I was interested in the polling effect / GE effect of the scenario set out by SHEVII above.

  15. @BFIELD: “As people interested in polling , what do all you think about only people being allowed to enact any policy if they have the support of 50% of the total electorate?”

    As long as everyone has a fair opportunity to vote there should be no threshold. If you can’t be arsed to vote against something then it’s only right that your support should be taken for granted.

  16. @BFIELD: “By the way forget ‘it’s the economy stupid’, I think it could well be ‘it’s the NHS stupid’.”

    It (or its equivalent) always was up there with it:

    “In 2015.Mr. Clinton and his aides have said from the very beginning that those interests can be summed up in three phrases, which are on a sign in the campaign’s headquarters:

    • Change vs. more of the same
    • The economy, stupid.
    • Don’t forget health care

  17. Sorry. The ‘In 2015’ slipped down the page.

    ““Mr. Clinton and his aides…”

  18. RnD
    Re smiley things , they are what Colin puts at the end of some of his posts, so not wanting to be thought a grumpy BAST*RD and not wanting to fart about with bits of sideways punctuation, I writ smiley things instead.
    Smiley things all round . Except for Paul. (Wouldn’t want to deprive the puppoes).

  19. colin

    Yes, I realised that you weren’t offering your own value judgement but I don’t think the choice is as black and white as you paint.

    There are lots of things that, in my view, are socially necessary, in the public good etc and, although I am by no means wealthy I am happy that my taxes are used, in part at least, for such things.

    My point [if I had one] is that loads more people think like that and, ironically, it is quite often those who receive from the state [viz you and me, which is what I believe Thathcher’s “no such thing as society” was trying to point out] who complain when they see others doing the same.

    So taking Lady T as a role model I feel we should be encouraging MORE people to see things in a moral perspective rather than accept the status quo.

    And then we do have a “society” as we begin to feel [to quote another hero of mine] “we’re all in it together.”

    [No LOL ‘cos I mean it.]

  20. @John Pilgrim
    Don’t think it was me, but I agree with your post

  21. @R&D

    I’m afraid world shortages of anything, like balance of payments crises and the M3 measure of the money supply, are just sooo 1980s

  22. Regarding Lord Ashcroft’s defence of the volatility in his polls:

    I don’t think that his defence stacks up at all, because it doesn’t address the issue of volatility – that is, the issue of large differences between consecutive polls. Instead he tries to muddy the waters by looking at the overall range of polling over a long period.

    So to properly test this, rather than rely on Ashcroft’s flawed apprach, here’s an analysis of the mean modular variation between successive polls, for Ashcroft, the Monday Populus and the YG Sunday Times (i.e. the YG Sunday Times compared to the next YG Sunday Times, treating all differences as positive). I took the sum of the 8 differences between the 9 polls over the past two months, and divided by 8.

    Mean Difference in Conservative VI
    Ashcroft 3.125
    Populus 1.5
    YouGov 0.875

    Mean Difference in Labour VI
    Ashcroft 2.75
    Populus 1.0
    YouGov 1.0

    So while typically you can expect a change of around + or – 1% in next Sunday’s YouGov poll for both Con or Lab compared to the last YouGov, you can expect a change of around + or – 3% for both Con or Lab in successive Ashcroft polls.

  23. Alec

    If I were a young and impressionable Muslim, I would be deeply puzzled by the UK establishment’s stance on such matters. We are now locking up young British men returning from fighting Assad in Syria – just a few short months since our PM wanted to sent young British men to fight Assad in Syria.

    No, no, you are completely wrong. We have always been at war with Eastasia.

    I’ve nothing but the deepest contempt for those who encourage these gullible kids to go off on this ‘adventure’ to ‘defend their Muslim brothers’, but almost as much contempt for those that prosecute the fools that do so[1]. Apart from anything else that hardly encourages them to realise it is a very bad idea and come back home quickly. It also helps reinforce the idea among young Muslims that they are and will always be seen as the ‘other’.

    Still as long as it means that the securocrats can continue pumping up their budgets and politicians and the media can go round shouting “Scary brown people”, everyone will be happy. Except them of course and the people who have to pay for it all.

    I’ve read somewhere that all you actually need to do to avoid this is to be paid for going off to fight. Then you magically stop being a terrorist and become a valuable part of our services export market as a mercenary. So perhaps all the Islamic State of Whatever needs to do is to set up ISI Security Services in the BVI and everything is OK. Perhaps they can even apply for outsourced government contracts

    [1] Obviously we’re not talking about anyone who has done anything bad, but we seem to have ended up with a situation when anyone even thinking about going on such a silly jaunt appears to be deemed a ‘terrorist’.

  24. BFIELD

    By the way forget “it’s the economy stupid,” I think it could well be “it’s the NHS stupid.” In 2015.

    And increasingly the public seem to agree with you. If you look at yesterday’s YouGov tracker questions:

    Health jumped from 35% to 40% as one of “the most important issues facing the country at this time”. At the same time ‘The economy’ went below 50% for the first time in this Parliament. Immigration remains the most chosen issue with 54%.

    When asked about “the most important issues facing you and your family” health was already in second position behind the economy, but 39% is a new high for it and 45% a new low for the latter.

    It’s particularly noticeable that these figures are not prompted by any big national scandal and occurring in Summer rather than at the time we normally see scare stories.

  25. YouGov/Sun poll tonight – Labour lead drops one to three points: CON 34%, LAB 37%, LD 8%, UKIP 12%

  26. @ Colin and Shev II,

    I think it’s irrelevant in the lead up to the election, because everyone already knows that public sector workers will get a better deal under Labour than the Conservatives. People in favour of this (Paul, Frances O’Grady) are already on Team Red, people opposed to it (Colin) are already on Team Blue. Labour don’t dare to bang this drum too much because of “fiscal credibility”, but the Tories with their constant union-bashing and their ambitions to take away the right to strike and so forth are happy to bang it for them.

    Shev II is right that it’s a massive potential stumbling block for the next Labour government, because as soon as they get in the question will go from “Are you better for us than the Tories?” to “Are you doing enough for us?”, a question to which the answer is always no. (Although current relations are so bad that Labour can probably rack up a bunch of points just by booting out Gove and repealing the NHS competition clauses, without spending a penny.)

  27. Called it!

    “Expect tonight’s YouGov to revert to normal (something along the lines of 34/37/8/12).” ~ Me, July 9th, 2014 at 11:27 am

    I was just off by a day. :p

  28. @Pressman

    Emerging economies may superficially look like they are doing well, but I largely observe they have a small number getting very rich, and a whole load more people remaining very poor.

    Neo-liberalism drives inequality, and that’s not the model of economy we should follow in my view.

    Aping the emerging economies is a race to the bottom.

    In terms of VI, the anti-union stuff from the Government will help them little in my view. I think the strong anti-union sentiment lies in the heart of true blue Britain who would never vote Labour anyway.

    I think it will make the Conservatives less attractive to ordinary Labour voters, that maybe wavering, and certainly won’t swing the votes to blue that the Conservatives need.

  29. @ Catmanjeff and others interested in the Data Retention and Investigation Powers Bill,

    Looks like Miliband has forced the Government to put in a sunset clause for 2016.

    So, it could be worse.

    (Of course, whatever the outcome of the general election it’s hard to imagine it producing a government that will defend civil liberties, but at least it will pass its terrible legislation on a normal timetable with proper scrutiny.)

  30. @Catmanjeff

    I can confidently predict that anti-union rhetoric will do the government no good at all because even those who would sympathise with such rhetoric just don’t consider it a priority. Hence it lacks salience and may well lack popularity too.

  31. @Spearmint

    Labour’s approach to civil liberties in power will probably be rather mixed. Sadiq Khan is a civil libertarian by instinct, but Yvette Cooper is unreconstructed New Labour.

  32. @Spearmint

    Thanks, I actually smiled to myself when it was discussed on PM tonight.

    I was interested in Norman Baker’s spin on it.

    Essentially, the Lib Dems are supporting this to open the debate on ‘snooping’……..

    So they are so concerned about civil liberties, they will support an anti-civil liberties law to open the debate.

    No comment.

  33. @Roger M

    Hang on, last I remember we were at war with Eurasia, not Eastasia. Has the final, indispensable, healing change happened?

  34. @ Drunkenscouser,

    I have a lot of Sadiq Khan. But he’s not the one with the Home Office brief.

  35. A lot of faith in, that should have been.

  36. @Phil Haines

    My own take on Ashcroft’s volatility is on page 8 of this thread (3:17) and gives a similar result to yours – that Ashcrofts polls are most volatile of the 3 – by a very different method.

    The suspicion is that he’s chosen the one way of assessing volatility that suits him. I find this very puzzling, as he tries very hard to be reputable in his polls.


    Going abroad to fight for a cause doesn’t necessarily make someone a terrorist or a threat to the UK.

  38. I think the days of pay increases are over for both the public and private sector. I know both my wife and myself have had less than 2% pay rises since 2008 working in the private sector and that is the norm amongst our social group.

    The simple fact is that UK workers are very expensive. Our companies can and are replacing us UK workers at 40% of the wage with equally qualified people offshore in India, Panama, Brazil, Malaysia, Russia, Ukraine, Morocco. Increasing UK wages will just accelerate that process.

    Our next government needs to look at the expenses side to improve living standards.

    Some low hanging fruit
    – school holidays – most schools break up at the same time of the year. Stagger school holidays by county and holiday prices should drop for everyone (and it will also help fill up those empty Cornwall hotels during term time)
    – house prices – most of the price of a house is land due to our planning policies. Having the highest house prices in the world means we need to have higher wages to support those. Reform the planning system so the price of land drops to international levels.
    – Childcare – we have the most expensive childcare in the world. Learn lessons from other countries on how to reduce the cost.

    Basically take the basket of spending done by a typical household and benchmark to other western economies. Where UK costs are significantly more than international norms, find out why and fix the underlying causes.

    Easy really….

  39. EL
    Not Boot of the Beast?

  40. Lol pressman, you’ve been lurking in the background. In actual fact, the living wage need not be burdensome if it is rolled out by the state and to contracts with the state. If all procurement is living wage, tender bids would reflect this. The policy would also prevent immigrant workers from undercutting British workers. For private companies, this could be done on an incentive basis ( as Labour has suggested) and also only apply to to companies with a larger workforce.
    It really only effects the lowest paid- in Islington it meant KPMG had to pay their cleaners the London Living Wage to have a contract with Islington Council.

  41. Re the “living wage”, the whole system is bonkers.

    Most jobs seem to be graded by how intellectually or technically difficult they are and no account taken of how crap they are to do – looked at in a negative way.

    But people doing menial jobs are still doing something very important, whether its cleaning hotels, working in MacDonalds or whatever.

    Ask yourself how much money you would need to do such jobs well and what would happen if nobody was prepared to do them.

    The very word “living” screams out that it is an essential component of a “wage”.

  42. Repeating what I’m seeing on Twitter. Don’t shoot the messenger:

    “POLL: Highest #voteYes support in a @Daily_Record/@Survation poll so far – #Yes 41%, No 46%. Excluding undecideds, Yes 47%, No 53%. #indyref”

  43. I see they wheeled out a 1980s Scottish pop star onto QT tonight.

    See me? In the 1980s, I was awestruck by Scottish pop/rock music. For a country half the size of Manhattan to produce such spellbinding wonders as Cocteau Twins, Blue Nile, Jesus and Mary Chain, Primal Scream, Love and Money, early Aztec Camera, pre-stadium/stentorian era Simple Minds, Momus, Josef K…well, it just wasn’t NATURAL. There was something special about Scotland.

    And then there was Ricky Ross. A man whose songs could command the weather. No matter how sunny it was, put on a Deacon Blue record and within minutes, the whole atmosphere was a turgid mizzle.

    Where The Blue Nile would take the grey mundanity of a grey city on a grey day in winter and strip the facade away to reveal aching, epic tragedies of love lost, clung onto or pined for, every twist picked out with melodies and rhythms that picked you out of the gutter and flew you over the city to SEE where it was all happening, Deacon Blue set it all up in primary colours both lyrically and melodically. And then they bleached all the colours out of THAT and left it a black/white bad man/good man story with never a chord change that came within a million miles of raising the hairs on the neck. And it became popular because it had the VENEER of caring about it. They produced songs which rammed how much they really, really, really cared down your throat. And at the back end of Thatcherism, there was a market for that. It was vacuous. It was utterly lacking any real soul or any real intellectual content. But it had a veneer.

    So anyway. QT was good wasn’t it. Not much has changed these past 25 years.


    Yes there has been a bit of trailing of that tonight. I think YouGov is wrong, I think their sample is too pro-Labour.

    But my reasoning is that it doesn’t make sense to match the samples to the make-up of a previous govt election because the elections are too different. It makes more sense to use a sample matching the general population and then apply a turnout model which I think Mori does – though I dont think they apply a turnout model other than likelyhood to vote. The Problem with the turnout model is that it very difficult to predict turnout of different groups and the wrong model will give the wrong result. So my conclusion is that the polls are not going to be very accurate.


    I’ve never said the YES side will win but the final result would be closer than many had expected but it might just be people around me but people are talking about the complete revulsion they have towards Westminster over the pe#o cover up.

    The more I see stuff on the news regarding British politics the bigger the YES becomes for me.

  46. Statgeek

    Average of the latest polls from the 6 regular indyref pollsters (excluding undecideds) is No 56% Yes 44%. Back in April, it was No 58%, Yes 42%.

    The shift is small, and probably confirms the view of the likes of Allan, myself, and others on here, that a No victory seems likely, but that the margin of their victory may be small enough to persuade even a reluctant Westminster to decide on a serious revisiting of the UK’s internal constitutional arrangements, to avoid the risk of a future Yes vote – though I wouldn’t hold my breath for that!

    Some pro-independence enthusiasts will prefer pollsters showing a higher Yes vote, and some pro-dependency enthusiasts will look to the pollsters showing a higher No vote, I doubt that any of the methodologies can measure opinion very accurately on this issue, so I prefer to look at the average of them all – which could equally be completely wrong!

    “When asked about “the most important issues facing you and your family” health was already in second position behind the economy, but 39% is a new high for it and 45% a new low for the latter.”

    While these are valid indicators, “health” has a more generally understood meaning (the NHS) than “the economy”.
    The latter is consistently reported on in terms of the deficit, growth of GDP, manufacturing, inflation, while income and employment or the financing or access to public services seen as subject to austerity in the interests of recovery. Definitely not apples and apples.

  48. Lefty
    Great panegyric.
    Couldn’t they get one of the Bay City Rollers for QT ?

  49. I’ll be meeting Les McKeown soon. I’ll ask him….

1 8 9 10 11