Voodoo polling corner

A sudden outbreak of voodoo polling this morning, or more to the point a sudden outbreak of serious newspapers reporting a voodoo poll as being meaningful.

What’s a voodoo poll? It’s what Bob Worcester calls the little “press the red button to vote” polls on Sky News, or the little readers’ votes things on the BBC website. They are entertaining enough, but they mean nothing whatsoever, they don’t measure the opinion of a representative group of people, they only measure the opinions of people who wander past that particular website (or are directed to said website by people trying to influence the poll) and care enough about the issue to vote…often several times if they know how to delete cookies from their computer. The classic voodoo polls were the old Today programme man of the year things, which used to have flagrant, mass vote rigging to try and get John Major or Tony Blair as the man of the year.

We all know that they don’t mean anything and certainly shouldn’t be reported as representing public opinion, and yet…The Times this morning reports that “A poll by the Berwick Advertiser showed 79 per cent in favour of switching allegiance from Westminster to Holyrood”, the Telegraph claims “the results of a poll have suggested that 80 per cent of the residents agree”,The Scotsman tells us that “an online poll has suggested 80 per cent of residents are backing the suggestion”,the BBC reveals “a poll carried out by the local newspaper revealed 79% of people in the area backed reunification with Scotland.”

The poll in question? It’s on the sidebar of the Berwick Advertiser’s website here, you can vote in it now if you want. Doesn’t matter if you live in Berwick, or indeed the UK. It’s a classic voodoo poll, so no attempt to limit it to the proper universe of people, nothing to stop it being fixed, no attempt at producing a demographically representative sample. These things really shouldn’t be published by reputable newspapers as an indication of public opinion – or at least – not without heavy caveats about them being non-representative, self-selecting polls.


37 Responses to “Voodoo polling corner”

  1. It’s a good story though Anthony.

    I particularly liked the comment that if Ehglish Border Towns were allowed to choose whose public services they would prefer, the border with Scotland would soon be somewhere on the south coast of The Isle of Wight.

  2. Have to admit one does not watch Parliament much. One prefers to talk to enlighhtened commentators, hence one gate-crashes this blog.

    One does however participate in the Sky Parliamentary Question-Time poll. One is purely interested in the answer “Who won the debate at Question-Time”, so to the question one answers “None of the above”!

    ;)

  3. What a complete waste of time and space. Things like this can ‘inform’ those with no previous opinion who believe what they read or see on TV.
    Taking Colin’s point it just goes to show how utterly ridiculous it is. The grass is always greener on the other side; until you get there and look back when where you came from is now ‘the other side’!

  4. Perhaps if we can get all of England to join Scotland, then we can stop worrying about the state of the Union altogether… :)

    I can feel Peter building up to say something…

    Also I think if people could vote for were they would like to be on the basis of public services the borders might extend a good deal south of the Isle of Wight, say Morocco or Nigeria….

  5. Berwick for the Berks is what I say.

    Actually as this whole dispute forms a part of the local tourist industry I’d think they would be best just to leave it as it is and milk it for all it’s worth.

    Lovely town especially the walls. Haven’t been there for years.

    Peter

  6. We really need the polls equivalent of ‘traffic light’ warning symbols on food; everything from well designed polls from impartial organisations through well designed but from partisan groups, to non-designed like the ‘red button’ polls, to the ultimate, the deliberately biased (like using ‘nationalisation’ vs. ‘public ownership’ or ‘capitalism’ vs. ‘free enterprise’ depending on which outcome you want).

  7. I recall one example from ITV Teletext’s phone-in polls. In April 1997, 75% of people intended to vote Labour in the next election. By June 1997 75% of people intended to vote Conservative in the next election

  8. Teletext are the classic examples. I don’t remember the pre 1997 ones, but since then they almost always show around 70 or 80% support for the Tories.

  9. I once came across a Cookie that you could install from the states that automatically alerted you to any poll on the net to do with Israel.

    It had originally been designed by a pro Israel group in the US but it was of course a two edged sword as although you could alert people you couldn’t stop them voting the opposite way to the one you wanted.

    One way round this to do with electronic voting might be to give every house it’s own e-mail address in a national system added to post codes. That way it would only recognise one vote per house and could tell where it came from.

    The lack of anonymity might put people off but it would be a way to approach it.

    I think a national system of household e-mails would be a good thing for Royal Mail, as it would let you send an e-mail to a house that wasn’t on line, with it being printed at the correct sorting office and then delivered by the postman. That would save a lot of time on postage and cards a lot of which Royal mail doesn’t get.

    Peter.

  10. It lives here – you too can be instantly alerted to the opening of Israel related voodoo polls. You’ll never be bored again – http://giyus.org/

  11. I think these types of polls can have some value as long as they are mainly used to compare previous result. Thus revealing a trend in opinion and not much else.

    “Just a bit of fun” otherwise.

    But having said that.

    Polls showing 90% or so, which was the case with the DRUID affair, has got to count for something at least.

    Polls from the major polling companies have to be able to be challenged one way or another. As they have the potential to be corrupted just as much as a British general election RESULT does these days.

    This, apart from the obvious reason such as ballot rigging in the form of postal votes. It is because they are selected from people that already have reasonable stable political viewpoints and long term party affiliations. Therefore can understate public opinion on specific highly charged issues.

  12. Useless even then – you can only be confident of a trend if the two polls if you have reason to think the two samples are comparable. You never have that with voodoo polls.

  13. Great article as usual Anthony. For normal people (i.e. not ‘Atlas Shrugged’ or Mike Richardson (should that last name be in inverted commas too?)) it points up the seemingly paradoxical fact that whilst an increasing number of ‘polls’ are (literally) worse than useless, we cling to the ones that mean something…

  14. Anthony,

    When are we due to have our next REAL poll?

  15. Antony wells

    These polls are not attempting to predict general election results with any accuracy. The polls you generally refer to are at least trying. Even then they only tell us what people are thinking now not when their is an election.

    Or worse all they show is what the responder thinks they should be thinking. In order that they do not seem to be thick, ill informed, or plain nasty, by the POLLS representative.

    If the poll for example is of SKY NEWS viewers then to some extent they are consistent even though not representative of the general population. They also have far higher numbers of voters then the average Mori poll which may only be about 1,500 spread over the whole country.

  16. Atlas,

    They tell us what some people may think, but not who or where those people are or if they represent a majority or minority, if there views are consistent or changing, if they differ from place to place.

    About the only thing they tell us is;

    “I’ve answered this poll”.

    It’s like looking out your window and seeing it’s raining. It doesn’t tell when it started or will stop, if it’s raining elsewhere or if it will rain tomorrow. It’s a random snapshot with no context.

    Peter.

  17. “They also have far higher numbers of voters then the average Mori poll which may only be about 1,500 spread over the whole country.”

    The classic error. It is representativeness of the sample that matters, not (beyond reasonable minimums) its size. For the 1936 Presidential election the Literary Digest polled over 2 million people. At the same time George Gallup polled around 5000. The Digest famously got it horribly wrong, predicting a Landon victory. Gallup correctly called a crushing Roosevelt landslide. The difference was that Gallup used a representative sample.

    You can’t even draw meaningful trends. There is great variation in samples obtained even by professional pollsters, hence the need to weight them to consistent demographic targets. Unweighted, uncontrolled open-access samples will not necessarily be consistent even when from the same media source. The story will be different, the timing will, people linking to it, or encouraging supporters to vote will be different. Even if you could, it would be a trend amongst people who do Sky voodoo polls, with no reason to think it reflected amongst the wider population.

  18. Anthony wells

    So what you seem to be saying is that Polls like the ones that SKY NEWS perform have no value whatsoever even when the result is sometimes over 90%.

    Now come on be logical,and use your common sense instead of your calculator all of the time, this simply can not be the case.

    If it tells us nothing else, it tells us what 90% of people that bothered to vote on the poll think. Which simply has to mean something. Especially if the question is clear and unambiguous. Surly most of the factors you describe will tend to even each other out.

    They are often a measure of a reaction to events. a snapshot that shows the power of the mass media. Which is the main reason why we have opinion polls at all anyway.

    Even if half the people that voted changed their minds by the next day they still tell us something of value.

    For example if the media is going hell for leather on an issue then do a poll. They will get a certain result. If after a few hours when the medias tone changes or new information becomes known. The polls can change quite dramatically. This tells me something even if it tells you nothing at all.

  19. I remember the ITV teletext polls as well. You just knew the only people who bothered to waste money ringing up for them were the most disgruntled people in the country who hadn’t got anything better to do, which is why they always had whoever was in government polling about 20% and the main opposition party on about 70%, as has already been pointed out.

  20. CPC

    Yes I agree. But I think I have explained in my last post why they still have a point and can be of value. They show how easily public opinion can be manipulated and by what. The sort of conformation of which that must sometimes give chief media executives an orgasm.

  21. Hi guys
    As it happens, I lectured at a Foreign Office Chevening Fellows seminar just yesterday to 40 or so representatives from Albania to Pakistan, and used one powerpoint going back to 1979, when the then owner of the Express decided that MORI had small samples, and so a coupon in the paper would be likely to bring in thousands, maybe even 10,000 readers’ ‘votes’ and so would be that much better. The late great Charles Wintour was then editor in chief, and knew better, so commissioned MORI to carry out a poll simultaneously. They got thousands all right, some 70,000.

    Here’s the text from the slide:

    Q. “How would you vote if there were a general election tomorrow?

    Express “voodoo poll”: 70,000 said 93% would vote Tory
    MORI poll: 1,070 said 46% would vote Tory.

    Also I asked the group who was the greatest statesman of the 20th Century: noone said Attaturk. Yet a Time Magazine “voodoo poll” again of tens of thousands of “votes” said he was, as the turkiish on-line population banded together to rig the vote.

    I could go on…

    To conclude, I used a well worn slide of my ten point guide to reading the polls:

    1. When were the fieldwork dates?
    2. Was the sample representative and large enough?
    3. The more sampling points the better (face to face polls)
    4. Make sure of where the sample was taken
    5. Is it an internet, face-to-face or a telephone poll?
    6. Are the questions unbiased?
    7. How are the “Don’t knows” re-allocated? (voting intention)
    8. Are differences statistically significant?
    9. Look for full question wording, and full answer wording
    10.Who paid for the poll?

    Cheers, and thanks Anthony, for reminding your readers tthat “vodoo polls” represent no one other than those with tthe time and inclination to answer, access to the net (three people in four 65+ don’t, and neither do one in four 18-24s), and are sometimes rigged.

    Bob Worcester

  22. Atlas – even when it’s 90%, it means nothing. Voting intention is one of the few things in a poll you can check against reality, teletext and Bob Worcester’s example from the Express show how you can get overwhelming landslides in a voodoo poll that do not reflect reality at all.

    They do indeed show that x% of people who filled in that poll think something… but that’s it. They have uses other than gauging public opinion – some people put them on their websites to find out about their users, what bits of the site they like and so on and you could certainly use an open access survey at the bottom of an article to gauge reaction to the article amongst readers. On websites they serve a useful purpose in making the site a more entertaining and interactive experience for users….they just aren’t any use in measuring overall public opinion and therefore should be confused with, or misreported as, proper opinion polls.

  23. All a bit of fun indeed – but there are a lot of people who take these POLLS seriously & it could sway their actual voting intentions – especially politically naive people !

    The result in Berwick makes a lot of sense to me – who would’nt want the idyllic perfect socialist nation that Scotland is becoming with all it’s handouts from the English ! The population of Berwick like the rest of us are propping up the Scots economy – for them it is easy to pop over the border .

    It’s funny though – when Scotland introduced a smoking ban – the pubs on the Scottish side were nearly put out of business while the Berwick side without a smoking ban were booming – loyalties seem to have changed now that England too has the same draconian law !

  24. @ colin who said “It’s a good story though Anthony.”

    Sorry Colin, it’s not a good story. It’s not a story at all. There’s no basis in fact for it. It should not have been reported at all.

    The only story here is that the media are increasingly willing to post voodoo polls uncritically as fact.

  25. Over 90% reminds me of elections in Iron Curtain satellites in the “good old days” – instantly suspicious and obviously worthless. (Wonder what % will vote for Putin’s nominee)?

  26. I’d be interested to know if there were any “don’t knows” in voodoo poll results? The notion of going out of one’s way to register indecision tickles me.

  27. Lee
    It is a story-it’s been a story about Berwick many times before.No doubt the measure of opinion either way has always been inaccurate-but that wasn’t what interested me.
    I thought it particularly amusing in the light of the currently perceived welfare benefits of living under the Scottish Administration.

  28. Anthony,

    Maybe it’s the text you use but is this Voodoo Comer or Voodoo Corner?

    I can understand the latter but if it’s the former it’s not that exciting.

    Peter.

  29. Hopefully the next poll will be somewhat more plausible than this one.

  30. Anthony Wells

    Its a shame you do not read my posts as carefully as I read yours.

    No one said least of all SKY NEWS or me that these polls are supposed to tell us who is going to win a general election.

    But I have have virtually never seen a Voodo Poll that that surprises me but plenty of general election results that shock me to my inner core.

    People vote for a political party for various reasons. Some of the major ones being party loyalty their own jobs, welfare payments and most of all their own parties propaganda, when they eventually get to hear what it is.

    Many of these snap polls are taken firstly before the media and the political parties PR men get to work, and secondly before people have had time to remember their own parties past mind programming.

    So their first and gut reaction is possibly CLOSER to what they really think then the more considered opinion they think they should have had in the first place. Also very often different to what they vote for in elections

    For example is what a man does or says when DRUNK and out of control closer to what he is really like inside, then when he later becomes sober?

    Just go down the average London Pub on a Friday or Saturday night and ask a LABOUR voter what they really think about issues such as immigration and feminism after they have had a skinfull. Then compare it to what a half sober Labour MP has to say on the issue on Newsnight. If you can see any relationship whatsoever then good for you because I NEVER DO.

  31. Hi Atlas, I guess that’s why we encourage people to be sober in government.

  32. I think Atlas was implying people are MORE likely to speak the truth when drunk so we should be encouraging our politicians to be drunk. Bring back Charles Kennedy!

    I personally don’t believe people’s first “gut instinct” or what they say when drunk is closer to what they really think. Most people think with their brains not their gut.

  33. Yes. I was implying that an instinctive gut reaction is not always the way to make good policy, and in a sense that’s all these polls would be, even if they weren’t statistically useless.

    Besides, people say all kinds of rubbish in pubs and regret it the next day…. :)

  34. conspiracy.

    Though you assert this with almost every post, what you singularly fail to do is provide any evidence. Like almost all conspiracy theorists your contention is that the conspiracy is so profound and comprehensive that no evidence to prove it is available.

    Meanwhile the rest of us live in a world of fallibility and shades of grey with no hidden hand directing anything.

    It’s no more difficult to live in as soon as you accept that what a[appear to be patterns are just the random effects of billions of individual decisions.

    You are no more significant than any of the rest of us, and no amount of wanting to have some fundamental insight into the hidden truth will make it true.

    Peter.

  35. Details on tommorow’s (Monday 18 Feb) ITV1 Tonight show:

    “TV poll backs Berwick border move”

    http://news.bbc.co.uk/2/hi/uk_news/england/tyne/7248529.stm

    “The referendum for ITV1’s Tonight programme had 1,182 voters in favour of becoming part of Scotland and 775 in favour of staying in England.”

    http://news.scotsman.com/uk/Berwick-residents-vote-for-move.3786272.jp

  36. I don’t think we want them back if they are still at war with Russia.