You may remember my blog post about the Observer reporting a voodoo poll as if were representative of members of the Royal College of Physicians a couple of weeks ago. Back then an open access poll hosted on a website campaigning against the government’s NHS bill found 92.5% of respondents wanted the RCP to “publicly call for the withdrawal of the Health and Social Care Bill”, and this was reported as being representative of the RCP’s membership.

I dutifully wrote a letter to the Observer’s readers’ editor, Stephen Pritchard, and he addresses the report in his column for the Observer today. Mr Prichard writes “we know opposition among hospital doctors is extremely high, but readers have a right to expect that things that we proclaim to be polls are properly conducted, using scientifically weighted samples of a population or group” and, as I have before, points journalists to Peter Kellner’s British Polling Council guide for journalists on how to report opinion polls. Full marks to the Observer for addressing the matter seriously.

Meanwhile, the RCP has since commissioned a ballot of its whole membership, professionally carried out by Electoral Reform Services. The ballot managed a 35% response rate. It found that 69% of members were opposed to the bill, but that only 49% thought the RCP should seek the withdrawal of the bill, with 46% saying the College should work constructively with the government to try and improve it.

That’s 49% who wanted the RCP to call for the Bill to be withdrawn, not 92.5%. That, dear readers, is an example of why voodoo polls are bunkum.

(Nigel Hawkes at StraightStatistics also has a post welcoming the RCP conducting a proper survey of their membership, rather than touting voodoo polls here)


13 Responses to “Voodoo polling, an update”

  1. Thanks for the update Anthony.

    It puts the rhetoric from the voodoo poll into context.

    Will be interesting to see if the Observer features the results of the RCP ballot.

    Report comment

  2. Colin – Guardian already reported it yesterday

    Report comment

  3. @AW

    “That’s 49% who wanted the RCP to call for the Bill to be withdrawn, not 92.5%”

    69% opposed to the bill, 6% in favour.

    Excluding DKs and those who were pro some bits and against others, that makes, er, 92%.

    Not arguing that voodoo polls aren’t dangerous, but they aren’t necessarily wrong either. The voodoo poll was a yes/no question, the RCP poll had more nuanced questions. The two aren’t inconsistent.

    Report comment

  4. Anthony

    I agree with Robin

    I think you need to be a bit careful with this argument you are using here as it is inconsistent with the one that is used for elections or opinion polls where DK or non-voters are excluded.

    Using your analysis the Coalition received 39% of the vote at the 2010 election and the Tories on their own only 23.5%. Less than 1 in 4 of the electorate voted fro David Cameron’s party

    This approach is frequently used by the right-wing press when discussing TU votes.

    A little bit more consistency please

    Report comment

  5. Apologies Anthony

    I have read your article properly – I was just skimming and read it wrong.

    I still agree with Robin’s general point though

    Report comment

  6. Robin – the voodoo poll didn’t ask if people supported or opposed the bill, it asked if the RCP should “publicly call for the withdrawal of the Health and Social Care Bill”. The correct comparison is with RCP ballot question asking if the RCP should call for the withdrawal of the Bill.

    Neither question had a don’t know option – the voodoo one was a straight yes/no, the RCP ballot was a straight choice between two options (though 4% declined to answer)

    Report comment

  7. Well before praising the RCP too much for conducting the survey, we should remember that they had to be forced into it “in response to a request for an Extraordinary General Meeting called by 20 fellows of the RCP and attended by 189 fellows”.

    Two things strike me as a bit odd about the RCP survey. The first is the comparatively low response rate of 35%. Their records may not be completely up to date (the fact they don’t even have e-mails for all their members is significant) and it also has to remembered that their membership includes retired doctors, which might bias things again.

    The second point is the options for the supplementary question. There was no way to indicate support for the Bill. Thus ‘Continue to engage critically on further improving the Bill’ could be portrayed as being in opposition to rejection. For many doctors it must have been the middle option – especially given that only 6% felt the Bill should be accepted.

    In reality the timing meant that the only way to obtain further changes to the Bill would be to stop it in its tracks. I get the impression that the RCP leadership were more concerned to use the survey to justify their past (in)actions rather than find out what doctors really thought.

    Report comment

  8. It matters to Anthony professionally but it doesn’t really matter, politically, what the margin against is.

    69% against is still a clear majority of respondents. Lansley & the PM said they had the backing of health professionals. They don’t.
    8-)

    Report comment

  9. Just looking at the YouGuv/Sunday Times poll there seems have been a dramatic fall in Ed Milibands support on how well he is doing as leader. I wonder if this is to do with the fact that health reforms have not been so high profile this week. Now neck and neck with Nick again.
    Any thoughts?

    Report comment

  10. AMBERSTAR
    ‘It matters to Anthony professionally but it doesn’t really matter, politically, what the margin against is.’

    All parts of the medical profession are highly regarded. Therefore this voodoo poll could have significant repercussions politically. I suppoprt the Coalition and am also aware of the need for continuous improvement in the NHS, both to address weaknesses in current practice and to meet increasing demands on the NHS budget. However, when the voodoo poll was originally published in the Observer, the fact that 92.5% of members appeared to be asking for the bill to be withdrawn cast serious doubt in my mind about the bill. The fact that the RCP’s own poll identified that less than half of those who responded supported the withdrawal, not only gave me some comfort but also confirmed how careful one must be against media (in this case left wing) manipulation of the truth for political ends.

    It appears that almost two thirds of members did not feel the issue significant enough to vote, and that of the one third who did, less than half supported pressing the Govt to withdraw the bill.

    Report comment

  11. @ Henry

    You are interested in politics & your own Party is split down the middle on the NHS bill. You might as well flip a coin as be influenced by a single newspaper article about a single poll or the balloting of a single organisation.

    Get the 49% private patients out of the facilities which belong to the tax-payer & retain parliamentary accountability for the NHS, then there might, just might, be a reason to say the LDs have won changes which make the bill more acceptable.
    8-)

    Report comment

  12. AMBERSTAR

    ‘You are interested in politics & your own Party is split down the middle on the NHS bill. You might as well flip a coin as be influenced by a single newspaper article about a single poll or the balloting of a single organisation.’

    Thank you for your useful advice. The trouble is I have tried tossing a coin but keep coming up with the wrong answer. Perhaps I have been more influenced by Shirley Williams (and of course our revered leader) than the observer’s voodoo poll. Despite the fact that Shirley is ex Labour and still socialist I hold her in high esteem (as I do a number of socialists who still support Labour). In fact Shirley was once my MP and I have a signed photo of her on my study wall between the pictures of David Steel and the wild man from borneo. I’m with Shirley on this one.

    Report comment

  13. Henry

    The comparatively low turn-out probably has more to do with poor record keeping by the RCP, the very short timescale for the ballot, a feeling that it was a bit late to do anything and a related feeling that the RCP leadership were a pretty use bunch, so what was the point of replying. There may also have been the feeling that the response was would be so overwhelmingly one-way that there was no point in replying.

    And the response was overwhelmingly one-way. Only 6% felt that the College should accept the Bill. Given that the Bill was already at ping-pong[1] stage, it was leaving it late to say “Hey let’s negotiate” though no doubt many would prefer that in an ideal world and of course there will be some things in any Bill which will be useful.

    So it may be that, given the straight choice between accepting and rejecting the Bill 90-odd percent would have rejected it and the voodoo and genuine polls would have matched, even if mainly due to luck rather than good management.

    [1] Rather delightfully this appears to be the official term.

    Report comment