Voodoo polling corner

The Press Association are reporting that “The majority of people from across the political spectrum believe Scotland should be responsible for raising most of the money it spends, according to research from an independent think-tank.”

Because it is on the Press Association feed, this is then repeated verbatim by various other newspaper websites here, here, here, etc, etc, all labouring under the misapprehension that because the Press Association reports a poll it is meaningful. They are wrong.

The “poll” was conducted by Reform Scotland, a think tank that published proposals for devolution plus earlier this year. The “sample” was drawn from people on Reform Scotland’s mailing list or following them on twitter. Needless to say, this is not a method likely to provide a representative sample of the Scottish public as a whole.

I hate to write as if addressing morons, but sadly it sometimes appears as if it is necessary. People who have signed up to follow a think tank that has proposed a devolution plus plan are, firstly, far more likely to be interested in politics (the vast majority of normal people are not on the distribution lists for think tanks!) and secondly, likely to be pre-disposed towards further devolution of power towards Scotland (for what it’s worth, the poll is also three-quarters male, only 10% over 65+ and has more Tory identifiers than Labour ones).

To give them some credit, Reform Scotland themselves haven’t claimed it is a representative poll, saying “We do not claim that this poll is totally scientific as it was self selecting. However, the responses, particularly those broken down by party affiliation, are very interesting, in particular”. Alas, the reality is that these caveats never get picked up by journalists, and such surveys inevitably end up being misreported as representative meaningful polls. For the record, the party breakdowns are not of any meaning either, since in the same way the poll overall will be grossly biased towards people with an interest in Scottish politics and a predisposition towards greater devolution, so will each of the party crossbreaks (i.e. the Labour voters in the sample will be more political and more in favour of further devolution than the average Labour voter, ditto other parties. They are also grossly demographically skewed towards younger men, and apart from the SNP have sample sizes under 100).

Over on the British Polling Council’s website there is an article written by Peter Kellner several years ago titled “A Journalist’s Guide To Opinion Polls”. Amongst other things, it gives guidance to journalists on when to take a poll seriously, and when to bin it. It is still flawless advice today:

“If the poll purports to be of the public as a whole (or a significant group of the public), has the polling company employed one of the methods outlined in points 2,3 and 4 above [quasi-random or quota sampling]? If the poll was self-selecting — such as readers of a newspaper or magazine, or television viewers writing, telephoning, emailing or texting in — then it should NEVER be presented as a representative survey.

201 Responses to “Voodoo polling corner”

1 2 3 5
  1. I loved their admission that it isn’t TOTALLY scientific.

    “At all” might have been better that “totally”.

  2. The MORI December poll showed support for “more laws and duties and all tax-raising powers” for Holyrood by party support as

    Ans, Con, Lab, LD, SNP, Oth
    Yes, 41%, 60%, 57%, 81%, 73%
    No, 56%, 33%, 38%, 16%, 20%

    So, apart from the Tories, there is actually a ” “The majority of people from across the political spectrum believe Scotland should be responsible for raising most of the money it spends” – and the Noes among the Tories, don’t have that big a majority.

  3. Yes this is not a scientific poll but it rather confirms the trend of the scientific ones.

    A very big majority (around 70%) want all revenue raising powers ie far in advance of the Scotland Bill proposals while a growing minority (around 36%) support outright independence.

    Why the new leader of Scottish Labour thinks anything other than they are totally in the soup is beyond me.

    The Times newspaper have just announced Salmond as “Briton” of the year. It may be that 2012 belongs to Scotland.

  4. @OLDNAT

    Strangely then, 16% of SNP supporters seem to be supporting you for reasons other than your core belief…


    Any thoughts on the poll reported by the Guardian on the views of Doctors regarding NHS (I assume E&W rather than Scotland, before you comment OLDNAT) and cuts in service. I’m naturally suspicious that people are more likely to respond if they agree with the statement…

  5. The Sheep

    It’s not strange at all. People vote for a party of government based on a number of factors.

    It would be just as sensible to suggest that it would be strange that 41% of Tories, 60% of Labour intend to vote against their parties core belief.

    Most people aren’t sheep.

  6. That poll reported in the Guardian is another voodoo poll.

    “Doctors.net.uk, a professional networking site to which almost all British doctors belong, asked medics: “Have cuts to staff and/or services affected patient care in your department, area or surgery during the last 12 months?” Of the 664 doctors who responded, 527 (79%) said yes and 137 (21%) said no.”

    (The Sheep will note that the question invited doctors across the UK to respond – though no doubt, the questioners had England in mind, rather than the other 3 Health Services in the UK).

  7. Well done yet again Anthony!

    I tried to post a comment on UKPR very early this morning when i first saw this PA piece, but, not entirely unusually, the UKPR site was unavailable. Yours is the blog which I have far and away the most trouble with, irrespective of which browser, operative system or connection I use. I strongly suspect that it is something to do with the design of the site, or perhaps your adverts. It is just a waste of time trying to access on a smartphone.

    For what its worth, I think that a clear maj of Scots voters would back “Devo Max”. Certainly a far larger number than would back the Calman/Scotland Bill mess, or would back the status quo, or would back abolition of the Scottish parliament.

    I’d gusstimate support for Dev Max at 70%, Independence at 45%, Calman/Scotland Bill at 25%, status quo at 15%, and abolition of the SP at 10%. All figures exclude DK/WNV, and there is (obviously) “overlap”.

  8. According to a poll in the Aberdeen Press and Journal 38% of Aberdeen football fans have had a relationship with a sheep. ;)

  9. Good article, Anthony.

    In my esteemed opinion, the very worst offenders in terms of “voodoo polls” are the Daily Express phone-in surveys.

    Not only do they ask leading questions (of the “Aren’t you sick to the back teeth that political correctness has gone mad?” variety) but as they only poll Express readers it means the average IQ of respondents is about 40 less than that of the general public.

  10. Now now OLDNAT a bit of season’s merriment please…

    There is a slight difference between the position of Labour, Tories and that of the SNP in that you are a party that has one overriding theme. Labour and Tories are not defined by their stance on Scotland…

    As for the Guardian poll, I think it’s a little trite simply to describe polls of experts through trade bodies in quite the same way as those of think tanks. I was interested in Anthony’s opinion on where on the spectrum of Voodoo they sit, from a bit under the weather to full night of the living dead Zombie.

  11. @Anthony Wells

    “Phil – don’t resorting to silly hyperbole. That is not censorship, it is normal (and correct) practice for newspapers to report their pollsters topline figures (i.e. the figures that the pollster says best represent public opinion)”

    Did I state that the Guardian should not have reported those top line ICM figures? No. What is at issue is the fact that the Guardian chooses repeatedly to publish these results alone, with no acknowledgement ever that the results are significantly different as a result of ICM adding in their own inferences of hidden voting allegiances rather than relying only on what respondents state explicitly as their intentions. It matters more in this case because they chose to make some waves through using the poll as their lead Boxing Day story and with an editorial critical of Miliband to boot.

    Is it censorship? Opinions here are split over whether ICM’s approach is the correct one or not, but people should have the opportunity at least to form their own judgement. And, in marked contrast to practise by YouGov, the full data is only released a full 2 days after the publication of the newspaper, so for that period we are stuck with only the data the Guardian chooses to publish and the interpretation it puts on it, correct or not. So, whatever the motive, yes I do think that is censorship.

  12. @Allan Christie,

    “According to a poll in the Aberdeen Press and Journal 38% of Aberdeen football fans have had a relationship with a sheep.”

    I’m surprised that figure isn’t higher.

  13. The Times newspaper have just announced Salmond as “Briton” of the year. It may be that 2012 belongs to Scotland.
    :-) ROFLOL :-)

    Does nobody else think that’s LOL funny? Alex Salmond happy to accept an award for being a Brit.

  14. Robin

    I must protest at your snide remarks about people of lower IQ, I doubt very much that low IQ folk read the express, they are far too intelligent for that!! In my experience high IQ and stupidity go hand in hand often combined with petty mindedness, some folk are far too clever to see what’s in front of their nose. :lol:

  15. I know that the Jamaican Times had terrible problems a few years ago when they commissioned research into whether people belived in Black Magic.

    The results kept being dismissed as voodoo polling.

  16. Terribly sorry – that was dreadful. But here’s a better one;

    A Higgs Boson particle walks into a church on Christmas Eve and the priest says ‘I’m so glad you’ve come – we couldn’t have Mass without you’.

  17. Robin Hood – I love the Express voodoo polls. The majority of them are 95%+ yes (or 95%+ no, as appropriate)

    Others, I think I’ve looked at doctors.net.uk polls in the past and they do make an effort to conduct kosher market research of doctors.

    Rubbish question though.

  18. Alec


  19. AmbivalentSupporter
    @Allan Christie,

    “According to a poll in the Aberdeen Press and Journal 38% of Aberdeen football fans have had a relationship with a sheep.”

    I’m surprised that figure isn’t higher

    Must had been a “Voodoo” Poll ;)

  20. I don’t know about Voodoo polls, but if it has anything to do with dolls to inflict pain, perhaps the Daily Mail could be employed in the same way. Today for example we have one story that the government are planning to close the borders to stop evacuees from the EU after it dies a death, as well as a sterling going up in value as it is used as a safe haven currency. Then there is a story about how solar rays will knock out communications etc.

    What is the current polling on whether the world will end sometime during the next two years ?

  21. AmberStar
    The Times newspaper have just announced Salmond as “Briton” of the year. It may be that 2012 belongs to Scotland.

    Does nobody else think that’s LOL funny? Alex Salmond happy to accept an award for being a Brit

    I don’t see the funny side?? Okay he wants Scotland to be independent and presumably hold a Scottish Passport as to a British one and many Scottish Sports personalities would possibly like to represent an independent Scotland but at the same time would be happy to represent GB in the Olympics.

    Now if Ed Miliband were to win this award then that would really be funny..LOL LOL ;)


    I imagine that Salmond thought it very amusing himself!

    That the Times couldn’t find any Brit more impressive than him, even more so.

    Personally, I think it’s a Unionist plot to damage his reputation for modesty. :-)

  23. Good Evening all from a lovely Bournemouth beach.
    If I may I will try to respond to earlier threads.
    HANNAH. I will send you the recent link from fb. You probably know the Old Testament story of Hannah, and that today’s reading was about King Herod.


    I would never vote for a party calling itself Christian anything! (LOL)

    To colleagues here who defended Nick Clegg and his choice of schooling. Lib Dem policy has for a long time been to close the church schools- for the understandable secular argument. So does Labour, which did not stop Harriet.
    In terms of atheism I think I was referring to the Labour leadership.

  24. CHRISLANE1945

    ” Lib Dem policy has for a long time been to close the church schools- for the understandable secular argument. So does Labour”

    For the avoidance of confusion – you are referring only to England.

  25. @ChrisLane1945:
    Yes, I do no that story- perhaps unsurprisingly, 1 Samuel 2 is one of my favourite Bible passages.


    I thought people might be interested in this from Labour List:

    The psephological geek/vulture in me really wants this to happen.

  26. Gah, that should be know, not no :-S.

  27. Hannah

    If there were to be a by-election there, then it looks interesting – given that UKIP took almost all its vote from Lab when they first stood in 2005.

    That very circumstance means that i have no chance of understanding how voters there will react!

  28. HANNAH.
    Thank you for this reminder about 1 Samuel.

    You will know probably, therefore, that the Mary stories in Luke are written to mirror the stories of Hannah and of Ruth among others.

    Best wishes for the season.


    “According to a poll in the Aberdeen Press and Journal 38% of Aberdeen football fans have had a relationship with a sheep.”

    Too many variables. The most interesting being which fact people would admit to. Having a relationship with a sheep or being an Aberdeen supporter. :D

  30. OLD NAT.
    English and Welsh church schools are the target of Lib Dem manifesto pledges.

    Clegg’s offsrping of course is on the way to the most selective church school in London.

  31. ALEC.
    The particle story is awful too, sorry, LOL as kids say.

    Because when the people gather the Particle is there immediately anyway, and his son becomes present as well.

  32. Thesheep @ OLDNAT

    “Strangely then, 16% of SNP supporters seem to be supporting you for reasons other than your core belief…”

    Nothing strange about it for they have clearly said what it was that led them to choose the SNP – “competence”.

    I see no reason to doubt that identifying SNP pro-independence supporters also think that competence is something they rather like.

    Consider what the alternative is, and that (at least relatively) the other parties do not appear to make a comparable offer.

    Some may think that we could get the benefit of this novel USP over reserved matters as well if Scotland were independent, and not only vote SNP but are pro-independence for that reason.

    I take that position myself, but credit the Founding Principles and the standards and procedures of the parliament rather than the SNP or AS personally.

    Reform of the Westminster parliament to a sufficient extent is not at all likely in my lifetime so I’ll vote for independence though it isn’t the ideal way to govern these islands.

    We have leader net approval data, but we don’t have leader or party competence polling.

    Partisans here will assure you that their party has the right values and policies. They will castigate their opponents for both and still do so if they diametrically change. They will condemn them for their taste in ties and biscuits.

    Even so, they would hardly dare credit the leaders of their chosen elite with “competence” for fear of ridicule.

    Surely “competence” shouldn’t be beyond these highly paid, educated people? It should be the norm, the average, a given for serious politicians.

    For public schoolboys playing at Oxbridge debating games that’s different. Competence in government is irrelevant and left to the [civil] servants.

  33. Amended post (posted two links)

    John B Dick
    “I take that position myself, but credit the Founding Principles and the standards and procedures of the parliament rather than the SNP or AS personally.”
    Yet the view in the English Press paints a very different picture. It’s all Alex’s doing! They do seem to be adulatory about Great Leaders!

    h ttp://wingsland.podgamer.com/wp-content/uploads/2011/12/salmondbriton2.jpg

    Fortunately, Nicola and others in the Cabinet will stop him from becoming too smug!

  34. Celtic have over turned the 15 point lead of Rangers, so hope for Scot Lab and eng Lab?

    ‘Our day will come’ as they say.

    And OLD NAT.
    scotland will soon be ‘free’.

  35. CHRISLANE1945

    Aberdeen drew with Hearts. There is as much political significance in that result.

    What strange concepts you have of Scotland from your vantage point on Bournemouth beach! :-)

  36. And don’t anyone tell me the Hibs – Caley Thistle score! The entire game is being shown now on BBC Alba.

  37. The Times newspaper have just announced Salmond as “Briton” of the year. It may be that 2012 belongs to Scotland.

    Kate Higgins has an interesting view on this at ther blog.


  38. Oldnat

    Is that the excuse?. There is no Unionist in England or Scotland that is his match, so Scotland breaks away by default?

    I don’t even accept that he is the only potential leader in his party, or even the best debater, and he certainly isn’t the main vote-gatherer if you look at the map.

    What his “secret” is, I think, the ability to pick out good people and let them get on with the “competence”.

    He does the FMQ, though I think it is too easy at times, and he supports his ministers. I don’t think that’s particularly difficult, because hi ministers are very committed as they are beginning tobe confident that they will be in the team which delivers independence.

    That’s such a big prize for people who joined the SNP since it hardly had a councillor and independence was a dream never to be realised in the lifetime of most mem that other considerations are secondary, so they are all working very hard and I expect it will cost one or more minister damage to health or relationship.

    Certainly a Dr Fox in their number would be at risk of actual assault.


    That seems to be what they are saying.

    It’s along the lines of “It wisnae me. A big boy done it and didnae run away”. :-)

  40. Cannock Chase would indeed be a fascinating by election.

    If say, 40% of the LD vote defected to Labour, and 10% to Conservative, we would have a change from the 2010 GE as follows:

    Con 41.8% (40.1+1.7)
    Lab 39.9% (33.1+6.8)
    LD 8.5% (17- 8.5)

    Prior to the 2011 local elections, the make up of the council was LD 17, Lab 14, Con 10… now it stands at Lab 17 (+3), Con 13 (+3), LD 11 (-6). Of the seats contested this time round LDs held only two of the seats and lost six, Lab held six and gained three, Con held one and gained three. With Con and Lab making equal gains from LD, an alternative scenario could be:

    Con 43.1%
    Lab 36.1%
    LD 11%

    However: votes cast in 2011: Con 9475 (15 wards – ave 632), Lab 10269 (14 wards – ave 734), LD 4484 (in 13 wards – ave 344).

  41. Sofa government it is not. Perhaps it is easier to avoid that trap if the cabinet is small.

    And another thing,….

    How is it that AS is able to overcome the disability of being an older wee fat balding guy with a wife who isn’t part of the PR show?

    Is it BECAUSE he is a older wee fat balding guy that he is taken for a serious politician and not a PR creation, and he is the only party leader around (E&W Greens apart) who is?

    He does what he needs to do, and he does it well, but the SNP’s strength is in the committment of its ministerial team. That’s where the “competence” is and which justifies the second and almost certainly the third term, independence or not.

  42. oldnat @ JOHN B DICK

    “That seems to be what they are saying.

    It’s along the lines of “It wisnae me. A big boy done it and didnae run away”.”

    You mean a big boy took it [Scotland] and ran away.

  43. John B Dick

    :-) for both posts.

    Partially, I think, it’s the normal process whereby those living outwith a political system, don’t know the team surrounding the leader, so everything is assumed to be down to him/her.

    It’s not surprising that even political geeks like me don’t know that much about English Ministers – because they have no impact here. Saying “Cameron” can be just shorthand for “the Coalition Govt” or “the UK Tories”.

    What does seem to be different about the lauding of Salmond in the English Press is that these articles are supposed to be from professional political commentators who might be expected to do some minimal research.

    The other aspect that I suspect is happening, is a deep disappointment among English commentators with the quality of their own political leaders. That is then projected into a “Wouldn’t it be wonderful if we had any leader with Salmond’s ability?” thinking.

    They haven’t analysed why he is successful (and I agree with your analysis), so they assume the cult of personality must apply.

  44. Oldnat

    I wonder if we should draw socal’s attention to this paragraph from your link

    ” Successive election campaigns have boiled down to one essential theme: that there is no limit to Scotland’s possibilities. Indeed, “Yes, we can” was the SNP slogan in 1997, long before anyone had heard of Barack Obama.”

    I fear it might break his heart to know that the great Obama pinched his best line from Alex salmond :smile:

  45. @Billy Bob

    Bear in mind that, in a FPTP election, even if the Conservatives gained seats, it doesn’t mean they gained votes. I suspect what happened is Lib Dem votes transferred to Labour in Lib/Con marginals, resulting in the Conservative gains.

  46. Socal

    Oops. :oops:

  47. Oldnat

    The Unionist narrative at present seems to be to call for detail fom the SNP so that they can pick holes in the issues regarding currency, oil, EU, Nato, borders, etc., and add confusion over the monarchy.

    Some of these are matters for negotiation, and If the Unionists have problems with the last, they should ask the Queen of Canada, an old hand who has done the job before and in French too.

    There doesn’t appear to be any enthusiasm for putting a positive case for the Union. Is it possible that there isn’t one, or is it just that it is more fun to be negative?

    Perhaps there is an opening for the SNP here. What is the real sticking point for Lab/Con? Is it the Security Council? The English voter doesn’t care if we go, he has been told that he subsidises us with his taxes, nd he can still visit as a tourist, but the political elite do care.

    Is it because the truth is that the subsidy is actually the other way round? Is it because they will need our renewable power, or water, or fear nimbys or cost if they have to take Trident away?

    It isn’t because it would be bad for Scots, that’s certain even if it would be bad for us.

  48. RiN

    Have you seen Salmond in self mocking mode (for Children in Need)?


    1m 32 sec in “In Glenrothes, I preached ‘Yes we can’, They replied ‘Nah, we didnae’. “

  49. @oldnat

    Steve Richards has a good article.
    He’s a very good commentator (or is at least better than the others)


  50. John B Dick

    I could make a better positive case for continuing the UK Union than I’ve heard from the Unionists – but I’m not going to! :-)

1 2 3 5