On margins of error

Last night’s YouGov poll had topline figures of CON 35%, LAB 41%, LDEM 9%, UKIP 7%. The Labour lead of 6 points is unusually low, but as ever, this in itself doesn’t actually mean anything. A series of low Labour leads is meaningful, it would show a narrowing of the gap. Just one or two can be explained through normal sample variation.

I say this repeatedly – yet anytime there is a poll that is an outlier from the average I see Twitter filled with otherwise quite sensible people speculating on what might have caused the movement. The answer is almost always “normal sample error”.

I think part of the problem is people forget the degree of normal volatility we should expect from polls. The quoted margin of error for polls is normally plus or minus 3 percent. This is actually quite tenuous – the 3 point margin of error refers to a genuine random sample, when actual polls are never purely random (most involve some elements of stratification or quota sampling, and even an attempt at a random sample wouldn’t be random because of non-response), there are design and weighting factors, actual voting intentions are based on smaller samples once won’t votes are excluded and so on. However, taking plus or minus 3 points as the margin of error is a good enough estimate for our purposes.

If we look at YouGov’s fourteen polls so far this month, the average Conservative score is 33. All fourteen polls have been within 2 points of this. Eleven have been within 1 point of this.

The average Labour score has been 43. All fourteen polls have been within 3 points of this, and thirteen of them have been within 2 points of this. Eight have been within 1 point.

The average Lib Dem score has been 9 points. All fourteen polls have been within 2 points of this, thirteen have been within one point of it.

In other words, while there have been polls showing twelve point leads and polls showing six point leads, all of the polls have actually been within the normal margin of error of CON 33, LAB 43, LDEM 9 and the distribution around that has been very much what you would expect from normal sample variation.


115 Responses to “On margins of error”

1 2 3
  1. Mike N:

    The media/electorate will want to know, one by one, whether each policy is a “red-line” one. How can they possibly say “no”? On the other hand, how can they say “yes” ?

    They are buggered.

  2. @Billy Bob – I noted those reports and a number of commentators (mainly on the left, it must be said) are now saying the incident is about truth and honesty, rather than short tempers and colourful language.

    While it is still a terribly minor issue in the scheme of things, if it becomes categorised in these terms it could yet spell trouble for Mitchell’s career.

  3. Poll Alert, apparently.

    http://www.standard.co.uk/news/politics/david-cameron-pays-the-price-for-calm-down-dear-jibe-as-women-shun-tories-8163210.html

    The Standard are reporting an Ipsos Mori poll showing a big fall (since the GE) in support of Cameron from women, and a ‘surge’ in support for Labour.

    No details on the poll itself or the timings, or how the numbers compare to any polling data in the intervening two and a half years.

  4. PaulCroft
    “They are buggered.”

    You make think that but I couldn’t possibly comment.

    …………………

  5. I meant snookered of course. I know nothing of public school life.

  6. @Alec

    There are the bystanders who were present during the outrage, and were visibly shocked… and then there is the “source” who gave precise quotes.

    Either the source has a creative flair for dialogue, or remarkable powers of recall. Another possibility is that a transcript might exist somewhere.

  7. Wonder what Cameron considers a resigning issue ?

    I bet this differs from what the public think if they were asked questions in polling.

    No doubt there will be polling done on the Mitchell outburst over the weekend.

  8. Alec – it’s just the MORI poll from earlier this week. While there appears to be a sensible questions there two (changes in a question on which party people think better represents women), some of it seems to be just stuff on cross breaks. Some people may recall (he said through gritted teeth) some earlier media hoo-hahs based on gender cross breaks in individual polls…

  9. …oh, and the Standard headline is a cringingly awful example of assuming a causal effect.

    Hell, it’s a change from 2010. Before assuming there was even any meaningful change in the women question I’d want to see the same comparison for looking after the interests of lots of other groups to make sure it wasn’t just the falling tide of the Conservatives generally being significantly less popular than in 2010.

  10. @AW – I thought is sounded like poor reporting.

  11. R Huckle –

    It depends how you asked the question. If you asked outside the context of any specific scandal, I expect you’d find the public were quite tolerant of misbehaviour on many fronts (though there are some, like MPs expenses, where they would be anything but) – being rude to a policeman would certainly not be up there.

    If you ask it in the context of a specific scandal the results become very different – the public overwhelmingly think ministers should resign for having “done something silly when being a member of a different political party” while are very forgiving of ministers who have “done something silly when a member of the party I support”

  12. AW

    @” the public overwhelmingly think ministers should resign for having “done something silly when being a member of a different political party” while are very forgiving of ministers who have “done something silly when a member of the party I support””

    :-) :-) :-)

  13. nteresting that if a ‘posh bloke’ calls a policeman something it’s reprehensible, but anyone else can make fun of them.

    Casting my mind back to my time in the Met if I was addressed in the manner alleged by the Sun and now I understand confirmed by the officer on duty ,I would have failed to see any amusement value and would have arrested them whether they were a “posh bloke” or the usual pond life

  14. The real problem is the “know your place” comment which turns the episode from insulting a police officer to insulting a large section of the population. At least that’s what got my attention. I wonder if their is a recording and whether we will get the musical version. I can hardly wait!

  15. STEVE

    @”Casting my mind back to my time in the Met if I was addressed in the manner alleged by the Sun and now I understand confirmed by the officer on duty ,I would have failed to see any amusement value and would have arrested them ”

    Were you in the mounted police at the Met?

    :-)

    http://www.bbc.co.uk/news/uk-politics-18084081

  16. Steve – were you around to be called one of Maggie Thatcher’s Boot Boys?
    I confess to the occassional use of it in my Student days.

    At least it was based on a kind of truth if somewhat exaggerated.

  17. JIM JAM

    Sadly I am old enough and spent many a dismal week in some frozen Pit yard, Huddling round a tin can fire with an equally morose group of strikers who I had every sympathy with, never a word of abuse from any of us incidentally.

    In my defence I never voted for the Iron Maiden

  18. I wonder if Mitchell was really taking part in the consultation on Section 5 ?

    http://www.parliament.uk/briefing-papers/SN05760

  19. A trope or meme (or, preferably, a myth) that keeps surfacing here: no modern political party has increased its share of the vote while in government.. Harold Wilson did it twice (1966 and 1974).

  20. I don’t think the Mitchel issue will have much traction . The police are well known for their petty bullying jobsworth behaviour and many will have symapthy with him .

  21. @ Paul Croft

    I meant snookered of course. I know nothing of public school life.
    —————–
    I believe they play billiards, snooker is for oiks ;-)

  22. CHASGLAS

    Mitchell will be gone by Sunday IMHO.

  23. @Colin

    AW
    @” the public overwhelmingly think ministers should resign for having “done something silly when being a member of a different political party” while are very forgiving of ministers who have “done something silly when a member of the party I support””

    You can’t possibly be referring to all those people who thought Mandelson and Jacqui Smith should have been hounded out of public life but who think Hunt and Laws are essentially decent fellows being unfairly pilloried?

    lol

  24. Corkscrew ,

    I agree . The Police Federation are milking this for everything .

    After Tomlinson , Hillsborough and god knows how many other abuses they need some good PR

  25. Was I the only person to be amused by Yvette Cooper’s rather pious comment on what I suppose we’ll have to call Gategate:

    Downing Street clearly have a lot of questions to answer. They must make clear exactly what happened, including what was said and whether the deeply offensive language reported was used. How can a chief whip hope to do his job or instil respect when he behaves like this and can’t even keep his cool?

    Actually that’s exactly how I would expect a Chief Whip to behave – it’s part of the job.

    Admittedly if you care which gate you through to get into or out of Downing Street, it rather suggests you are a little over-obsessed with status. Think of the explosion if he’d been sent round to the tradesman’s entrance.

    That said, you’d have thought that if the Met wanted to get rid of their reputation as the armed wing of News International, it would have been sensible to leak the information to the Mirror or elsewhere.

  26. Let me make myself clear on this. Hillsborough has exposed widespread corruption and systemic dishonesty in the South Yorkshire Police in the 80s and 90s, and, at a similar time, great miscarriages of justice were also routinely perpetrated by the West Midlands Crime Squad. They betrayed the very public they were meant to serve and their actions were a disgrace to their profession. There have been, and still are, bad people in the police. There have been, and still are, very many good people in the police who give great service to their communities. That said, the Tomlinson case was a scandal, and there are other cases of people dying in police custody that need re-investigating. We could go on, but we won’t.

    Why does any of this, though, mean that we can’t admire the many good things the police do and grieve for the two policewomen, and their families, who were gunned down in Manchester last Monday? And, while we’re at it, why can’t we also side with a bobby doing his job, maybe over-officiously, who is then boorishly abused by a member of the British Cabinet?

  27. Ann in Wales

    “Cameron “Has a real problem here ”

    Yes. I feel sorry for him as I did for Harold Macmillan. He has other little tasks to fill his day without this.

    Unusually opponents have not yet tried to attack the
    PM’s judgement in appointing the disgraced minister.

    He has to work with what he’s got.

    We don’t have have enough MP’s to form a government which excludes liabilities like Mitchell.

    Isn’t that what they mean by FPTP giving “strong” government? A bigger majority allows a PM a better chance of appointing people who won’t disgrace themselves and the party.

  28. CROSSBAT11

    Hunt & Laws ARE decent fellows-but they probably deserved their pillorying to some degree.

    Actually Prescott’s the chap-got away with abusing public servants , and duffing up members of the public.

  29. Crossblat , you very politely describe the officer as over officious but his behaviour is very typical of the boorish petty bullying manner used by many officers regularly to deal with very basic non criminal situations.

  30. CHASGLAS

    @”The Police Federation are milking this for everything ”

    Yep-smells of another ambush to me. They were on the telly pronto.

    He was leaving Downing Street-not coming in. So what did he say……..can I get through the gate , presumably.

    And what did they say-that’s the question no one is asking.

    No doubt their note books will reliably record their response……

    Unlike Theresa May, who showed considerable dignity at her ambush by he Fuzz ( albeit of the jaw clenched variety), seems like Mitchell walked into it.

    You don’t get to be Chief Whip by your skill at dignified silence.

  31. @ Roger Mexico

    ……..on what I suppose we’ll have to call Gategate
    ————–
    I’ve been looking forward ti a Gategate for simply ages. :-)

    Regarding Yvette Cooper, she is over-egging the pudding with bells on (ok, I’m mixing metaphors but still…)!

  32. @Colin

    “Actually Prescott’s the chap-got away with abusing public servants , and duffing up members of the public.”

    Did his popularity no end of good too, if I remember rightly, when, as most of us would have done, he gave that pillock a fairly good left-hander when he launched an egg at his face from four feet. The public reaction was mainly; “Good on yer, John”! As for abusing public servants, I thought his secretary quite enjoyed their little liaison, didn’t she?

    “Yep-smells of another ambush to me. They were on the telly pronto.”

    Whatever happened to the halcyon days when the Police were hand maidens to Tory Governments and the sure-fire way for a Tory Home Secretary to get a standing ovation at a Party Conference was to open the speech with the words; “I’d like to start by paying a tribute to our wonderful police who…………………….”

    The blue-rinsed brigade in the audience used to swoon as one. lol

  33. @ Crossbat

    Let me make myself clear on this…
    ——————–
    My post was targetted at politicians & police spokesmen in general, not at you. But I really enjoyed your ‘rebuttal’ so if ’twas me who unintentionally provoked it, I’m glad I did.
    8-)

  34. CROSSBAT11

    @”a fairly good left-hander”

    Oh yes-he was an expert at the left hander-inside & outside the Office.

    But I try not to think about it too often.

  35. @Colin – ” …can I get through the gate”

    The answer was in all probability yes… dismount and use the pedestrian gate. It seems there had been the exact same altercation on the previous day between Mitchell and diplomatic protection officers. He was informed that there are rules about opening the main gate. Choreography for the movement of cars in and out of Downing Street is strictly coordinated.

    It would be interesting to find out if Steve Hilton had the gates opened for him. Pictures I can find show, Laws, Hammond, Letwin, Gove etc all opening the pedestrian gate (and having some trouble doing it) for themselves. There is also a picture of Osborne having the gate held open for him by an armed officer, but then he lives there.

  36. Blimey, I spend the evening at the Queen’s Hotel in St. Ives, having a lovely meal and listening to a very good folk trio and I come back and there’s sod-all to respond to here.

    Gate-gate just sounds daft – “Pleb-Gate” surely?

    Mitchell should spend a night in the cells and then be sacked.

    Firm but harsh was my ole school motto.

  37. BILLY BOB

    ” (and having some trouble doing it) ”

    For once, the “gate” suffix may be appropriate –

    Nixon’s presidential aide was called to ask him to open a pill bottle for him – “the cap had teeth marks on it where Nixon had apparently tried to gnaw it open.”

  38. @Old Nat – “Nixon had apparently tried to gnaw it [the pill bottle] open.”

    Now that is something you would be more likely to expect from Hunter S Thompson… who followed Nixon on the campaign trail, and had many interesting things to say about him (none of them complimentary). Sadly a great opportunity was lost when the two of them shared a car to the airport and talked football instead of swapping pills.

  39. I found the reaction of general acceptance from the world of finance to the growing realisation that Osborne will miss his deficit reduction targets interesting.

    One reading of this – perhaps the most obvious one – is that Osborne was highly misleading when he claimed bond markets would implode unless the deficit was slashed rapidly.

    However, I suspect it’s more complex than that. In as much as markets have minds, they seem to want assurance that their concerns are being addressed by governments. I always felt Osborne was wrong in his doomsday claims, not because they couldn’t happen, but more because any viable plan would have kept markets happy. Whether this meant halving the deficit within 5 years or eliminating the structural deficit within 4 I don’t think mattered – they just wanted a plan.

    The sanguine reaction today may also reflect a general belief that Osborne will do what is necessary, so perhaps credit needs to go to Osborne for giving markets sufficient confidence to ride through the borrowing level disappointments. However, If Darling’s plan had been adopted, that confidence would probably still be there, and there would have been less risk that a more lenient target would have been broken.

    I guess the key factor is what happens next. Unless we get decent growth in the next couple of years, I can see further slippage on the targets and a more strained relationship with the markets. The big danger would be if markets decide that Plan A is suppressing growth to such an extent that it is in itself the problem. If that happens, any credibility Osborne has in the markets evaporates and we end up in real trouble.

  40. @MIKE N

    “Aye, so you’re one of those “tempted to think” to whom I was referring?”

    You mean, “happy to state the obvious when I see it?”, of course. :)

    The laugh in all this is that this has gone to the level of media, and people are calling for his sacking for (allegedly) calling someone a bad name.

    Harriet Harman and ginger rodents springs to mind.

  41. @Colin

    “Hunt & Laws ARE decent fellows”

    Really Colin, these are the sort of blatantly partisan comments that set off a firestorm and end with AW having to wield his moderation axe.

  42. @Statgeek – I think you’ve rather missed the point. Unacceptable rudeness isn’t something that really should be tolerated in government ministers and officials, especially when there are no excusing factors. Plenty of Tories are saying precisely this (including Boris) so this isn’t a partisan game. Several MPs have privately told journalists this is not the first time Mitchell has been somewhat foul mouthed and several of them have called for him to go.

  43. Surely it’s not just swearing/rudeness that is the issue with Plebgategate. It’s the (whatever the precise words) “I’m important, don’t you know who I am (i.e. better than you) get out of my way you oik” attitude.

  44. @Billy Bob

    You said “…Now that is something you would be more likely to expect from Hunter S Thompson…”

    Although he’s more widely known for the “Fear and Loathing…” articles, his coverage of Nixon and his prodigious drug usage, I prefer “Generation of Swine”, a series of articles written for the San Francisco Examiner in the 1980’s. Tied to the discipline of a regular column gave his work shape, limited his usual segues (Nixon, drugs) and made it accessible. If you remember the tail-end of Reagan and the era of the first Bush, when Thatcherism was winding down and an ideas vacuum was beginning to form, it’s a nice companion piece and some of its imagery (“Bush will crawl miles over broken glass to…”) sticks in the mind, like a javelin thru a nun. PJ O’Rourke ripped him off shamelessly and undeservedly: the Denis Leary to his Bill Hicks (oh, ask your mother…). The political junkies on this board (i.e. all of us) will yelp with recognition at the minor courtiers of Reagan’s throne (Ed Meese! Don Regan!) You can probably get it cheap at Amazon. (ISBN 978-0-671-66147-2, see http://en.wikipedia.org/wiki/Generation_of_Swine )

    Regards, Martyn

  45. BILLY BOB

    Naw. It just comes from Woodward & Bernstein who actually talked to Special Assistant to President Nixon, Steve Bull.

    Maybe it’s just my generation who remembers the details of the original “gate”.

  46. @ SwampMongrel (from the previous thread)

    “I like your analysis but I think you’re over complicating it. The only states worth watching are OH, PA and FL. If Obama wins one of them… he wins IMHO. Romney has to win all three.

    I understand that different regions and states will move in different directions over a 4 year period but that’s already factored into the polling data we’ve got. Looking at the state of play there is no way that Obama would lose Michigan and/or Pennsylvania unless Romney is also so well placed he picks up OH, PA and FL.

    There’s also no way that Romney wins if Obama picks up NC.

    I thought your comments on the sunbelt states are interesting. I didn’t realise that Obama is outperforming his ’08 position with Latino voters. I thought a lot of his Latino support would have drifted away once the novelty of a ‘minority’ president had worn off.

    I don’t think those states in the South West are electorally that important at the moment (not enough EV’s) but it should be a major cause for concern for the Republicans. Especially, if it starts to look like Lationo’s are locked in to voting democrat.”

    I agree with you that Romney must win OH, PA, and FL to win the election. I do think (although it’s not highly likely), Obama could win North Carolina and still lose the election. It’s a possibility and not just a mathematical one.

    The states in the Southwest are VERY important. Bush won every single one of those states and needed them to win narrowly twice (well he lost New Mexico by a few hundred votes in 2000). Without those states, the GOP has to look elsewhere.

    Now as for the Latino vote. Latinos and Blacks do not see themselves as one in the same. There has been plenty of discord and division between those two communities over the years. Politically, it’s been very difficult for candidates to build coalitions with both sets of voters. This is true of black candidates seeking Latino voters and Latino candidates seeking black voters. Obama initially struggled to win over Latino voters in 08′. Some of it was racism. Some of it was trepidation over the fact that some Latinos worried over a black President who would ignore them. And some of it was due to the fact that John McCain had a good relationship with the Latino community and had been a proponent of the Dream Act. Obama was trounced by Hillary Clinton among Latino voters in every single primary in 2008. And his performance among them in the general election was solid but not overwhelming. There was no novelty vote for him.

    Over the course of his Presidency, Obama has become more trusted and more liked within that community. Furthermore, the Republican embrace of anti-Latino racism has driven Latinos into the arms of Obama. Romney’s whole ‘self deportation’ comments don’t help him.

  47. @ Crossbat11

    “However, if he is guilty as charged, then it does show him in a particularly poor light. I don’t quite go along with Jim Murphy’s twitter reaction, but Mitchell’s behaviour does play into the image of boorish arrogance that is often associated with the Tory Party. To some extent it’s the stuff of Cameron’s nightmares considering the assiduous work he’s done to try and detoxify his party’s brand.”

    You follow his Twitter feed too? I feel like the man is slowly building up a caucus of UKPR followers. I enjoy his attempts to give sports and fashion advice.

    This Andrew Mitchell dude sounds like the type who would fit in at a Romney fundraiser.

    @ Billy Bob

    “Mitchell denies swearing and denies using the word “pleb”.”

    I think it’s funny that that’s the outrageous and offending word here. It’s like Robert Moses’s 23 page response to a 1000 plus page book about him that largely trashes him and accuses him of all these awful, awful things including a multitude of racist epiteths where the only thing he categorically denies is calling Fiorello La Guardia ‘Rigoletto.’

    Can I just say something about these radical crusading right wing politicians who go after public sector workers and attack the working poor and middle class all in the name of giving rich people more money? They are giving a really bad name to rich people and country club/yacht club/city club members.

  48. @ Alec

    “In recent weeks there has been a rash of articles and comment along the lines of ‘what is the point of David Cameron’, and with one or two notable policy area exceptions, there is a clear sense of a government that has run out of ideas. I hasten to add that this analysis is not a partisan point on my part – I’m simply regurgitating comments that have come mainly from friends of the government and internal Tory party commentators. ”

    Is it that or is it that the government came in and made massive cuts to government services that were unpopular but initially supported by a majority of the public as a neccessary sacrifice for the greater economic good. Now that all these cuts have happenned and the economy is back in recession, people are left wondering what they sacrificed for. So is it that the government is out of ideas or is it that their ideas have proven bad ones, the public is unhappy, and they don’t know where to go from here?

    @ Billy Bob

    I’m trying to watch this full Romney video and I’m having to take breaks because it’s just that awful. It’s not just him, it’s all these people who were at this thing. Makes my skin crawl.

    It was nice of Romney to make a racially tinged remark about Elizabeth Warren. That was nice. You know, the idea that somehow he’d have an easier time of winning if he was actually Mexican is really insulting. I remember how difficult it was to convince the voters of Los Angeles to elect a Mexican American to be the god damn mayor and what a battle it was to overcome the racism among a bunch of supposed progressives. Oh yeah, it would just be so much easier if he was Mexican. F**king prick.

  49. Why do Tories keep going on about mid-term blues and reclaiming protest votes come the election? Eh?? What mid-term blues? What Protest vote? They’re only down about 3% on the election. It’s kinda hard to suffer a significant protest vote when you’re not far off core vote to begin with. Even if they recover all their “protest vote”, they’re still only back to 36% anyway.

    Now the Lib-Dems, boy do they have a protest vote…

1 2 3