The Guardian’s front page story on the NHS reports findings of a voodoo poll in their front page story:

“More than 90% of those who voted in a British Medical Journal poll believed the planned health reforms should be scrapped. Of 2,947 votes cast on bmj.com over the last week, 2,706 said the reforms should be dropped while 241 said they should stay”

The story does, at least, not claim this is specifically representative of anything, but the very fact it is reported carries the implication that it is in some way meaningful or representative of BMJ readers or people involved in the medical profession (in that sense the Guardian’s report is less bad than the PA copy, which presented the figures as being representative of BMJ readers). This was not, however, a poll in any meaningful sense, but an open access click button question on their website.

As ever, such open access polls are not properly weighted or sampled and are very easily fixed by people distributing the link to others to encourage them to vote… such as, erm, Guardian star-columnist Polly Toynbee here.

If you are a journalist reading this I again implore you to read this guidance from the British Polling Council on how journalists should report polls, particularly Q.13 on how to tell whether a poll is worth taking seriously or not.

In this particular circumstance the finding isn’t grossly misleading as there is good evidence to suggest NHS employees do indeed oppose the reforms (see, for example, this YouGov poll of NHS employees for 38 Degrees), but in a way that makes it even worse – reporting worthless findings when there are properly conducted ones out there.


68 Responses to “Voodoo polling corner again”

1 2
  1. Rumour has it that our ‘Arry has been acquitted.

    I am going to seek to paid my wages gross into a Swiss account in my cat’s name.

  2. RE: last two YG polls = well one or the other must be an outlier and I’ll guess we’ll find out over the coming week.

    The mood music and the news agenda “feels” more like a small Labour lead. But we’ll see.

  3. @Anthony

    Polly Toynbee isn’t the only one to solicit votes to skew a poll: Daniel Hannan did it here, and I’m sure I could find other examples if I looked.

    Regards, Martyn

  4. Martyn – lots do, it’s why the wretched things shouldn’t be reported.

  5. NickP

    Perhaps you should ask your bosses to “gift” a sum into your secret account, coincidently equivalent to what you would have earned had it been a “salary”.

    If it was a salary paid into your secret account I suspect if the authorities caught onto you then you would in fact be guilty of tax evasion.

    *This is intended to be humourous and not actually meant to form advice in any form. Please pay your taxes kids!

  6. Upside I suppose is that at least he can be the next England manager now !

  7. It does show how difficult collecting taxes from the rich is. I’m reminded of Ken Dodd’s successful plea that he was afraid of civil uprising which is why he insisted on always being paid in cash.

    How will this play against the backdrop of cuts everywhere? Wonder if the “nice guy” image will last or whether the public will resent tax dodging at this time? Although now he has been acquitted does that mean there is no tax due on the “gift/investment”?

    Might be difficult to phrase a polling question. Do you agree or disagree that Harry Redknapp has got away with murder?

  8. I dont understand why hed bother for 20 or 30 grand, thats about 3 days work… surely its more likely it was just forgotten about

  9. MARTYN

    @”Daniel Hannan did it here, ”

    Daniel Hannan and Douglas Alexander on that occasion.

  10. joe

    £189,000 in total, I understand.

  11. I can’t see harry taking the England manager’s job. He’d miss the wheeling and dealing, wouldn’t he?

  12. They just can’t help themselves can they? It’s the lure of spurious exactitude. This particular example would be OK if the said something along the lines of “the overwhelming majority of doctors” or something like that. But no they see those precise figures and they have to publish them.

  13. As for ol’ ‘Arry’s acquital, there wasn’t a jury in the land that was going to find him guilty. The cheeky-chappy persona sealed it from day one and, knowing most people’s natural antipathy to the taxman, he was never going to be convicted by his peers. Lester Pigott, who was convicted of tax evasion, cut a less sympathetic figure and probably had the misfortune of coming up against a jury who had backed a few of his less successful mounts!

    Now, if there had been a few more Arsenal fans on ‘Arry’s jury……………………….

  14. The Harry trial was clearly a fix IMO!

    Strange how the public seem to hate/vilify bankers and businessmen and deem them massively overpaid (rightly so IMO), but everyone still loved tax evading Harry Rednapp even when he looked guilty. The same goes with the massively overpaid footballers, singers and film stars – how many people complain that their idol shouldn’t be rich, isn’t paying enough tax or is massively overpaid???

  15. @AmbivalentSupporter

    I’ve just had a ten-minute conversation with a Redknapp supporter, and, my goodness, there was every logical contortion possible… :-(

    It’s tribalism: if the person involved is “us” (for whatever definition of “us”) then that person will be defended, if the person involved is “them” (for whatever definition of “them”) then that person will be attacked. Guilt or innocence doesn’t come into it.

    Regards, Martyn

  16. @Colin, @Anthony Wells

    Thank you.

    Regards, Martyn

  17. @AmbivalentSupporter

    You’re right, but then again we’re a nation of humbugs in all sorts of ways. I’ve been party to conversations in pubs where someone has boasted brazenly about swindling an insurance company by making a fraudulent claim, and been roundly congratulated for doing so by his mates, only for the very same people to lay in to “dole scroungers” and “benefit cheats”.

    Moral turpitude moves in strange and very surprising circles.

  18. @CrossBat11, Martyn

    You are both quite right!

  19. “There may be trouble ahead…” so the lyric says.

    Well, according to a well-known publisher…”the group Liberal Left said it hoped to become a rallying point for members opposed to the coalition and those who see the party as a centre-left organisation seeking common cause with Labour, Greens and others on the centre left.”

    And “The first Liberal Democrat group openly opposed to the coalition is to be launched at the party’s spring conference in Gateshead next month with a warning that the coalition has been a political disaster for the party, as well as a denial of its radical roots.”

    The only surprising thing is that it has taken so long to reach this point.

    I wonder what NC can offer the LDs to placate this threat andf hold the party together.

  20. @Hooded Man (FPT) et al.
    “FWIW Lefty and I agreed last night it’s a sample that favours Labour…..I’ve got a fiver that says Phil would agree as well…..”
    ____________

    Indeed I do. (But who’s paying?)

    The impact of these sample variations can be a bit overdone. Firstly, not all of the missing 2010 Cons are yet decided and of those that are, a few (not many) have switched. And simply adding them in to what is otherwise a balanced sample would then leave it a bit top heavy with Conservative identifiers, Daily Mail and Telegraph readers (etc.), requiring a bit more reweighting in the other direction….etc, etc.

    So without access to the full mathematical black box, it’s a bit of a judgement call, but taking the above into account I’d guess that if 2010 past vote were an additional weighting factor, then last night’s Labour lead would shrink by 1%-2% rather than the 3% the 2010 Cons are short in the sample.

  21. It s only ‘Voodoo’ if its presented as if it is representative of anything other than the views of those who took part in the survey. As far as I can see this was not the case.

    If people reading it are stupid enough to extrapolate these views to the whole population that’s where the ‘Voodoo’ bit comes in.

    You can’t blame the Guardian for this. Blame the UIK population.

  22. AW
    At the risk of opening an old chestnut, the guidance of the BPC is not enough when the Guardian is concerned.

    Q12 of the BPC Q&A guidance states: “I have seen polls conducted by different, well-regarded, companies on the same issue produce very different results. How come?” Answers a,b,c and d come nowhere near explaining the differences we so often discuss on this site.

    There should be an e as well, namely:
    “e. Some polling companies assume that people will not do as they say they will when reporting results. So differences may be the result of variations in the way different polling companies choose to interpret similar sets of responses.”

    Whether such adjustments are correct or not is beside the point. What I find difficult to accept is for massively adjusted results to be reported in a newspaper as verbatim responses, at least two days before the commissioned polling company gets around to publishing the detailed data.

  23. Why on earth would anyone report results from online polls? Is it lazy journalism or pure ignorance? Or a combination of both?

  24. Phil,

    ” (But who’s paying?)”

    ‘Twas a notional Lady at Billy Hill’s….terrible odds though, you were a racing certainty….

    I’d agree with your points, and one really does need the full stats, and over a long period, to make sense of it…..
    I’d have a couple more points to add around retention of vote probably being replicated in ‘missing’ 2010 Cons, and the additional impact on all three parties of LDs being oversampled, given their 2010 vote has been sliced up and shared out but it’s not for here…….and MOE just straddles the +1 Tory lead and +5 Labour lead anyway.

  25. I agree that people’s hypocrisy is beyond measure.

    If someone is so filthy rich we can barely comprehend it, that makes it alright for them to accidentally fail to pay tax on a few hundred thousand quid they mislaid in a foreign bank account.

    I wonder what would have happened if Stephen Hestor had admitted to the existence of a similar account?

    These things are not new, though. In Child Protection work it is very difficult to secure convictions against teachers for offences against the children in their charge.

    People may be willing to believe the allegations of children. But schoolchildren? Everyone knows they are wicked, anti-social, conspiratorial liars.

  26. And with Redknapp free…Capello quits!

  27. The NHS issue must be hotting up…Nearly 3000 signatures for `Drop the Bill` e-petition today…Maybe even Ed might get a boost

  28. Oh b*llocks. Looks like my beloved Spurs may be about to lose their messiah…. :(

  29. Nick P, great newsflash.

    What an extraordinary day for Redknapp. He must be a shoo-in now.

    Daniel Levy’s “Arry’s free ! ” champagne has just lost all its bubbles……..

  30. I think Redknapp will stay at Spurs but agree to caretake England through the Euros. No conflict of interest at all…

  31. NEIL A

    After almost 40 years in teaching, I’ll let you into a secret.

    Children are like the rest of humanity. Some are indeed, ” wicked, anti-social, conspiratorial liars”. Most tell some of the truth as they see it, or as they want to see it, or as they think others want them to see it.

    They are just like adults – but being less experienced in the adult skills of lying, disinformation and general wickedness, they aren’t as good at it.

  32. NickP,

    Can’t see him getting away with that…….or the FA seeing it like that…..

    …..although the FA may still want to complete their due diligence on him. Rosie47 utterly reeks, but the brown envelope trail always looked a more inviting path to follow……..

  33. @Neil A

    “Oh b*llocks. Looks like my beloved Spurs may be about to lose their messiah…”

    I didn’t know Gareth Bale was leaving.

  34. NEILA

    “I agree that people’s hypocrisy is beyond measure.

    If someone is so filthy rich we can barely comprehend it, that makes it alright for them to accidentally fail to pay tax on a few hundred thousand quid they mislaid in a foreign bank account.

    I wonder what would have happened if Stephen Hestor had admitted to the existence of a similar account?”

    We all know the answer to that.

    Well put Neil-agree totally.

  35. @OldNat,

    That’s certainly true. But it is bizarre the way people are much less likely to believe a group of children within a school context than without it. Given that all children are schoolchildren.

    5 boys at a Scout Group = incontrovertible evidence of guilt

    5 boys at a secondary school = incontrovertible evidence that the witnesses got their heads together behind the bike sheds to get back at “sir” for some perceived slight.

  36. NEIL A

    I’m not sure that the difference is that bizarre. Those kids who join the Scouts have voluntarily done so. If that organization breaks their trust that is different from an organization that they are required by law to attend.

    We both know that some people who find children/young people sexually attractive can be found among Scoutmasters/teachers/any other career that brings them into contact with young people – and as long as that remains totally within their heads, need cause no problems.

    Scouts can avoid the problem by leaving the organisation. Pupils can’t leave school in the same way.

  37. “Ken Livingstone said some Tory MPs denounce homosexuality, while secretly “indulging””.

    “Discussing the issue of privacy during an interview in the New Statesmen magazine, Mr Livingstone said: “[The public] should be allowed to know everything, except the nature of private relationships – unless there is hypocrisy, like some Tory MP denouncing homosexuality while they are indulging in it.”

    Asked why he referred particularly to Conservative MPs, he added: “Well, the Labour ones have all come out.

    “As soon as [former Prime Minister Tony] Blair got in, if you came out as lesbian or gay you immediately got a job. It was wonderful.

    “You just knew the Tory party was riddled with it like everywhere else is.”

    BBC

    Don’t yah just love him?

    Not offensive of course-or homophobic. Couldn’t be.

    :-) :-) :-)

  38. For once, I’ll defend Ken. I don’t think he’s remotely homophobic. And I don’t think his remarks about gay MPs were particularly offensive. At worst he can be accused of putting an unnecessarily partisan slant onto the issue.

    I actually found what he said about Thatch more offensive.

    But I think politicians should be allowed to be offensive.

  39. There’s no chance at all of Ken saying sorry.

  40. I agree Neil A. Maybe Ken shouldn’t have named a party but otherwise I agree with his views on this.

  41. NEILA

    Thanks.

    I would like to have heard KL’s reaction to Boris saying that Labour MPs are “secretly indulging in homosexuality” when every one knows that “the Labour party is riddled with it”

    We would then know if KL believes all politicians should be allowed to be offensive-or just him.

    You would also know whether the “people ” whose hypocrisy is “beyond measure” , includes Ken Livingstone :-)

  42. I think if Ken had made his remarks to me, I’d have responded with two words.

    “Ron” and “Davies”.

  43. LIZH

    Do you think “indulging in homosexuality” , rather than -say”practising homosexuality” , carries any connotation which might be seen as offensive?

    Do you think that “riddled with homosexuality” is a reasonable way of saying “many people are homosexual in that particular group” ?

    I only ask-not being homosexual , I find it difficult to judge.

  44. Colin

    The language does seem odd. However, “practising” suggests adolescent fumbling – as in “not knowing how to do it properly yet”.

  45. OLDNAT.

    Clearly I had in mind :-

    “habitual or customary performance of” or “. habit & custom:”

    rather than

    ” repeated performance or systematic exercise for the purpose of acquiring skill or proficiency”

  46. @Colin

    I suppose different people are sensitive to different ways of saying things. I seem to remember you were not offended by “calm down dear” which many women would find offensive.

  47. LIZH

    I suppose they are.

    :-)

  48. Colin

    In matters of sex, there may be little difference between your two definitions. :-)

  49. OLD NAT

    I don’t think so at all.

    The latter implies a desire to perfect.

    The former does not

    :-)

  50. Colin

    :-)

1 2