Voodoo polling corner

Back in 2012 I wrote about the Observer reporting an open-access poll on a website campaigning against the government’s health bill as if it was representative of members of the Royal College of Physicians. I also wrote to the Observer’s readers’ editor, Stephen Pritchard, who wrote this article about it.

The Guardian today is making the same error – they have an article claiming that seven out of ten junior doctors will leave the profession if the new junior doctor’s contract goes through. The headline presents it as representative of all junior doctors and it is referred to as a poll and a survey in the first two paragraphs. Only in the final, seventeenth paragraph is it revealed that it wasn’t conducted by any reputable market research organisation, but a self conducted survey of members of a Facebook group, the Junior Doctors Contract Forum, which is campaigning against the new contract (the Telegraph had a similar article earlier this month that appears to be based on the same data).

We cannot tell if efforts were made to limit the poll to actual doctors or to make it representative of junior doctors in terms of career stage, age, region and so on – it doesn’t really matter, as it is fatally undermined by being conducted in a forum campaigning against a contract. It would be like conducting a poll on fox hunting in the Countryside Alliance’s Facebook group and presenting that as representative of the countryside’s views on foxhunting. The flaw should be screamingly obvious.

Questions along the lines of “If thing you oppose happens, will you do x?” are extremely dubious anyway. The problem is that respondents to opinion polls are not lab rats, they are human beings who seek to use polls to express their opinion, even when it’s not exactly what the question asks. From a respondent’s point of view, if you are filling in a survey about something you oppose, you’re are likely to give the answers that most effectively express your opposition. Faced with a question like this, it’s far more effective to say you might leave your job if your contract is changed than say you’d meekly accept it and carry on as usual.

We see this again and again in polls seeking to measure the impact of policies. For example, before tuition fees were increased there were lots of polls claiming to show how many young people would be put off going to university by increased fees (such as here and here). After the rise, they miraculously continued to apply anyway. Nobody wants to tell a pollster that they would just swallow the thing they oppose.

I don’t doubt that many or most junior doctors are unhappy with the new contract, but you can’t get a representative poll by surveying campaigning groups, and you shouldn’t necessarily believe people telling pollsters about the awful consequences that will happen if something they don’t like happens. It’s a lot easier to make a threat to a pollster that you’ll resign from your job than it is to actually do it.

UPDATE: While I’m here in voodoo polling corner, I should also highlight this cracking example of a voodoo poll in the Daily Mail. It claims “One in three women admit they watch porn at least once a week”… but it seems to be an open access poll of Marie Claire readers, certainly it is in no way representative of all women in terms of things like age. It contains the delightful line that “Out of the more than 3,000 women surveyed, 91 per cent of the survey’s respondents identify as female, eight per cent identify as men and one per cent is transgender.” I don’t know how to break it to them, but you probably can’t include the 8% who are men in a survey of 3000 women.

217 Responses to “Voodoo polling corner”

1 2 3 4 5
  1. Should be “the British Constitution”

  2. @John B

    I never meant to imply that EVEL applied to the Lords. What I was trying to say is that it is factually incorrect to say that, as a result of EVEL, any peer will be able to do anything a Scottish MP can’t do. As is the case now both Pete Wishart and Lord Foulkes will be able to scrutinise, amend and in extremis vote down any bill they like – the only difference is that the bill will need the consent of English MPs if it has been designated as ‘England-only’ (something that applies equally to Lords and Commons amendments).

  3. @Amber Star

    The Sewel convention does not return power to Westminster. The power remains in Holyrood, but Holyrood agrees that a *specific* UK bill shall include *specific* Scottish aspects. If the Sewel convention returned power to Westminster, then it would apply in perpetuity.

    As it says in the Scottish Government’s own ‘Key Facts’ document:

    “It should be emphasised that Legislative Consent Motions normally relate only to certain specific provisions in Westminster Bills, not to whole or even substantial parts of these Bills.”

    “The use of a Legislative Consent Motion normally has no bearing whatever on the boundaries between reserved and devolved matters as set out in the Scotland Act. The Scottish Parliament continues to enjoy full legislative competence as before.”

    For you to misrepresent a Legislative Consent Motion as returning power to Westminster can only be a case of intentional misrepresentation or simple ignorance.

  4. @Cynosarges @Amber Star

    It should also be said that the Sewel convention is not a convention that has proved at all controversial over the years – it allows for Westminster legislation to extend to devolved nations, even where a matter is reserved or explicitly devolved, where this is sensible (usually when Westminster is introducing uncontroversial provisions that the devolved legislatures would else need to replicate).

  5. The real question I have about EVEL is why? It is a gift to the SNP, and SNP MPs are on Twitter spreading the line. No doubt they will be able to use every EVEL bill to good effect.

    And it underlines the fact there’s different countries in the UK, which all have separate interests. It is an admission that the Government cannot govern for the whole country and the various national interests take precedent. Westminster is in effect, no longer the unitary parliament.

    But what do the Tories gain? They have a majority and are the government – is there any scenario where a vote passed by the whole house would be vetoed by English MPs? With only 1 Tory in Scotland and he a cabinet minister, there is no/little possibility of a rebellion of the single Scots Tory – causing a majority FOR a bill (which would NOT be a government bill) of the whole house, being vetoed by the English only committee.

    Labour if elected with either a majority or minority with Progressive support will simply retract the standing order change.

    Admittedly, it was a manifesto commitment, used to help stir up English nationalism, but the Tories don’t need that now, they won, so EVEL could have been quietly dropped.

    So why?

  6. “Simply retract” perhaps, but at some political cost in England I would think.

    I think there are tactical reasons for the policy, but I think for most Tories, and most English people, its mostly just the principle of thing.

    A lot of English people see devolution essentially as a way for the other nations to leverage special pleading into extra spending.

  7. I’ve been expecting Anthony to post on the recent YG poll for “What UK Thinks” (ie Nat Cen) on their experiment on the effect of the leaders of the 3 biggest English parties (in terms of VI) on the EUref outcome.

    Unfortunately, WUKT only envisioned one of these in a both a Remain and a Leave position [1].

    The control group (with no suggestion of a Leader stance) showed – Remain 51% : Leave 49% [2]

    For those prompted for Cameron urging Remain, there was a slight increase in the Remain position – Remain 52% : Leave 48% [2]

    Those prompted for Farage urging Leave, showed marginally fewer for Remain – Remain 50% : Leave 50%

    For those to whom it was suggested that Corbyn would support Remain, the Remain vote was higher – Remain 53% : Leave 47%

    While among those to whom it was suggested that Corbyn would urge Leave, the Remain vote dropped – Remain 49% : Leave 51%.

    All very much moe, but it does suggest that in a “damn-nearest run thing”, some undecideds will be influenced by the views of their favourite/most hated party leader.

    More details here –


    [1] Presumably they see no likelihood of Cameron ever doing anything other than urging Remain, or Farage suggesting Leaving.

  8. Neil A

    “A lot of English people see devolution essentially as a way for the other nations to leverage special pleading into extra spending.”

    You are doubtless correct – and since that is the message that their media has constantly fed them, what reason do they have to think differently?

  9. Couper,

    It makes sense. The Tories have almost nothing to lose in Scotland but much to gain. Specifically, a strong SNP lets them keep on with the Coalition of Chaos theme at election after election – and ties up considerable Labour resources fighting a party which isn’t them.

  10. @Couper

    Fair points but…

    1/ Frankly the SNP aren’t a big problem for the Tories in electoral terms. If they are to make up ground in Scotland it will almost certainly be at Labour’s expense.

    2/ The admission that there are different nations came in the late 1990s. Lots of people do think England has been left behind when they hear about Scottish devolution, and read stories about Scotland getting more per head than England, and this is an effective way to signal that they are doing something about it (although, as you say, it will make little or no practical difference this side of 2020).

    3/ If the Tories in future fall just short of a majority it could be invaluable as justification for them holding on to power. Standing orders cannot, of course, be amended on the initiative of anyone other than the government.

    4/ These things are always harder to get rid of than to get through. Labour would have to think very carefully before repealing them if they got into power but didn’t have an English majority.

  11. @Jack Sheldon

    I think you will find that the “admission” about nations in the UK predates.the late 1990s by at least one hundred years (Irish Home Rule) and that constitutional devolution to NI predates that to Scotland and Wales by several.decades.

  12. @Hireton

    Yes, I will go with that! (Though I suppose what Couper and I mean is the recognition that Britain – i.e. excluding NI – can’t be governed as essentially a unitary state).

  13. @Couper

    Re: EVEL

    Wrote a detailed reply but it disappeared. I’ll summarise.

    Labour and the Tories when in government have a paranoid obsession with each other. They try their best to keep the other as far down in the gutter as possible to the detriment of their own prospects of positive policies.

  14. Jack Sheldon

    Can I suggest a further qualification to your qualification of your original suggestion?

    People in Scotland (and in the UK Government) knew perfectly well that neither the UK, nor GB, had ever been run as a “unitary state” since its 1707 inception.

    Governance is not just about the politicians, but also the machinery of government – primarily the Civil Service and QUANGOs.

    I doubt that many in England (or nowadays in Scotland, for that matter) understand the massive administrative devolution that long predates the re-establishment of the Scottish Parliament.

    I get the strong impression, even on here, that many in Albion imagine that Whitehall’s writ ran in Scotland as it did in England (and, to a lesser extent, in Wales).

    That was why the Goeschen (and its successor, Barnett) formulae existed.

    So what actually happened in the 1990s was a public admission to the people of England that there was no unitary state.

    That they paid little attention, other than to the occasional media story was quite inevitable, since so many in England (including MPs) treated legislative devolution as “something for the Scots – oh, and the Welsh, but nothing to do with us.”

  15. @Jack Sheldon

    Yes. That’s what I meant

    @Neil A

    On spending, there is no special pleading, spending is fixed by the Barnett formula, specifically to avoid special pleading.

    Scotland actually manages it’s money better, for example spending on the NHS in Scotland has been flat in real terms but in England it has increased, with arguably better performance in Scotland. But, Scotland hasn’t had a costly top-down re-organisation and Scotland no longer has costly PFI deals.

  16. Couper

    “On spending, there is no special pleading, spending is fixed by the Barnett formula, specifically to avoid special pleading.”

    However, prior to the re-establishment of the Scottish Parliament, there was a lot of “special pleading” (usually in the form of political blackmail!) by the SoS for S!

    In many ways, it was simply a continuation of the classic 18th century role of Scottish MPs (followed by Liberals, Tories and Labour, in succession) of squeezing what they could for their clients.

    Perhaps the critical difference between Scots and Northern English MPs in that regard, was that the political representatives of the Northern English were either incompetent at that tactic, or just carpet-baggers for the London interest.

  17. @OldNat

    Also a very fair point that is often overlooked, including by people who should know better.

  18. @Oldnat – I’m in full agreement with your 9.44pm post. The English, in the main, completely fail to understand how the UK functions beyond England’s borders, both in principle and in practice. It is stark, sometimes, to listen to English people in England talk about such matters, as they are almost invariably completely ignorant of the facts.

    Were I in Labour’s position (or the SNP’s, for that matter) I wouldn’t be arguing the lines they are headlining. ‘Second class MPs’ spunds fine in Scotland, but is of no import in England. The simple fact is that devolution does create an imbalance, and although this is being cherry picked (London MPs are still going to vote on UK wide transport and policing, for example) it cuts no ice electorally down here.

    A far better approach is to accept Tory logic, and then challenge them to deliver it properly. If Scotland, Wales and NI have a proper parliament, why doesn’t England? Why shortchange English voters with a half arsed constitutional mess, instead of a proper parliament with proper powers? Constructed on a fair voting system (oh no! That shatters the Tory dream!) with Westminster the UK federal centre.

    This remains the only logical option (other than separation) but no one is talking like this, because the current proposals are based on narrow party interests, and Labour hasn’t the wit or imagination to do anything radical.

  19. Couper2802 – “The real question I have about EVEL is why? It is a gift to the SNP, and SNP MPs are on Twitter spreading the line. No doubt they will be able to use every EVEL bill to good effect.”

    I’m not sure SNP people frantically tweeting about how it’s evel they can’t influence the banning of fox hunting in England while keeping full hunting rights in Scotland, is going to motivate anyone to any effect! :-)

    More to the point nobody cares about SNP going ape about it. They get worked up about so many things so often, people just yawn and think, SNP going ape, must be Friday.

    It’s dawned on everyone that the reason the Scots joined the union was because their currency was becoming worthless and the reason they voted to stay in 2014 was because they feared going back to a worthless currency. Nothing has changed in 300 years and nothing will in the near future.

    In addition there are demographics to consider.

    In 1707, Scotland had 1 million people and England had 5 million.

    In 2011 Scotland had 5,295,000 million people but England’s population has grown to 53,012,025.

    In 1971 Scotland had 5,229,000 people – so the population had only grown by 1.3% to 2011, and according to the census that growth was down to English people moving north (who voted No).

    In 1971 England’s population was 43,460,525 – so it had grown 22% to 2011.

    Those trends will continue. Scots don’t like living in Scotland much and migrate to England in large numbers. Immigrants don’t like moving to Scotland partly because of the weather and partly because of the culture (just look at the way nationalists are trying to force JK Rowling to leave Scotland because they are angry that she’s a) English and b) successful).

    By 2050 I fully expect that England will move from being 85% of the UK to 95% of it. At which point the govt of the day will say “shouldn’t we repopulate Scotland” and give incentives for English people to move north, which should change the culture there utterly.

    England doesn’t actually have to do anything but wait things out and let the Scots do their self-destructive thing.

  20. Alec

    I agree. Party politicians will normally posture / create a narrative (delete according to preference) on any issue that comes to hand.

    I do find it interesting that “saving the UK Union” doesn’t seem to be anywhere near the top of any party’s agenda,

    That the Tories and SNP respond to EVEL in the way they do was fairly predictable [1].

    Had the LDs ever really believed in Federalism, they would have made attempts to force movement towards that through during their term in government.

    As you suggest, Labour is probably the only party that could have made the argument for a properly Federal UK in England – but their obsession with ruling England, and their wholly erroneous assumption that they needed MPs from Scotland & Wales to do so, always made that an unlikely prospect.

    [1] The SNP have been remarkably adept at taking Labour lines, and making them their own. You may recall that it was Labour (specifically Mags Curran and Gordon Brown) who created the “2nd class MP” idea.eg –


  21. It sits a little strangely for the SNP, so recently a party that swore off voting on English-only matters anyway, to now consider themselves second-class. It’s a bit like Stan/Loretta in The Life Of Brian fighting for the right to have children even though he/she couldn’t have them anyway..

  22. @Candy

    Actually the SNP will still be able to vote to keep the fox hunting ban. EVEL is a veto so even if it is English only they will still have to pass a vote of the whole house.

    Hence my point that EVEL is pointless for a practical purposes this parliament BUT it is a great political weapon for the SNP. So seems a pretty stupid move by the Tories

  23. @ CynoSarge

    The Sewel convention does allow MSPs to return power to legislate on devolved matters to Westminster. I made it clear that MSPs have to give their consent; so I don’t know why you’re flinging ill-founded accusations around.

  24. @OldNat

    LDs have believed in UK federalism for as long as I can remember.

    Labour’s position on the matter has always been a matter of convenience. Philosophically, Labour’s traditional position was that of using the machinery of UK government at all available levels to deliver the best outcomes for the those most in need of state assistance. In other words, using the economies of scale of the whole UK (where possible) for the benefit of those who needed it most (wherever they were). That was the theory. In reality that simply led to centralising power.

    But the theory made sense and is still the strongest argument against what appears the inevitable breakup of the UK. That people in Scotland, NI and to some extent Wales have totally lost faith with it is testament not to its failure as a concept but to how UK governments have failed to deliver on it.

  25. @Neil A

    I’m not sure of the SNP position on this one (in the absence of Independence). I think it’s that EVEL should not happen until a full scale UK constitutional settlement to iron out all anomolies.

    In the meantime the SNP practise EVEL by default – by choosing not to vote on English only matters.

  26. @Couper

    “I think that is the only way to save the union.”

    Is it worth it? This union is not a union in practice, rather the largest partner with smaller partners bolted on.


    Agreed. What’s more, their obsession with one another prevents the nation moving forward in any manner of directions (research, industry etc.). Divide and conquer ad nauseam.


    Those stats look ok, but the causality is more than a little wanting. The migration figures were shifting long before the SNP took power, so don’t paint it as “JK Rowling got beasted”. She has a broader follower base than the SNP on Twitter (5.76M to 123K), and is happy to play the (millionaire, Labour-funding) victim. Basically, you pciked some stats to open up your opportunity to have another pop at Scotland.

    @RAF (1.09am)

    Agreed again. The difference being that if they found it did impact Scotland in some shape or form, they could say so, and get involved / meddle (delete as appropriate). Now the speaker, never a Scottish MP again it would seem will get to decide.

    All we need now is a speaker election so that Cameron can have his best option in the big chair. Rees Mogg? Courteous, polite, proper and completely Conservative. :)

  27. Candy

    Your 11.34 pm post (as they often are) was absolutely right. My feelings exactly re the SNP and the reasons for the no vote in Scotland.


    I agree i think Rees-Mogg would make a perfect speaker of the commons.

  29. A second labour peer has resigned the whip over Corbyn’s policies.

  30. @Amber Star

    As written on the Scottish Government’s own website

    The use of a Legislative Consent Motion normally has no bearing whatever on the boundaries between reserved and devolved matters as set out in the Scotland Act. The Scottish Parliament continues to enjoy full legislative competence as before.

    That you manage to distort “no bearing whatever” into “transfer of powers” is a true example of Orwellian doublethink. Jack Sheldon summed up the purposes and scope of the Sewel convention succinctly. However, the Scottish Government’s Key facts lists exhaustively the uses of a LCM. Perhaps you could explain which of their reasons (below) you assess as a permanent return of powers to the UK?

    – Where it would be more effective to legislate on a UK basis in order to put in place a single UK-wide regime (e.g. powers for the courts to confiscate the assets of serious offenders);
    – Where there is a complex inter-relationship between reserved and devolved matters that can most effectively and efficiently be dealt with in a single Westminster Bill (e.g. the introduction of civil partnerships).
    – Where the UK Parliament is considering legislation for England and Wales which the Scottish Government believes should also be brought into effect in Scotland, but no Parliamentary time is available at Holyrood (e.g. to strengthen protection against sex offenders);
    – Where the provisions in question, although they relate to devolved matters, are minor or technical and uncontroversial (e.g. powers for Scottish Ministers to vary the functions of the Central Council for Education and Training in Social Work);
    – Where the breadth of the powers of the Scottish Parliament and/or Scottish Ministers would be enhanced in a manner that could not be achieved unilaterally through an Act of the Scottish Parliament (e.g. conferral of functions in relation to railways).

  31. Following the rant of 11.34 last night it’s obvious to anyone who lives in Scotland that Candy has not the remotest idea about what Scots think. The fact that Scots leave Scotland in large numbers is nothing to do with England – it was happening long before the union of the two countries. It’s because Scots can contribute so much more to, and thereby improve, the places to which they move. :-)

  32. And if Scotland is such a dreadful place to live (Candy 11.34 again), why do so many English folk want to come here?

  33. @ CynoSarge

    Nowhere did I say it was a “permanent” transfer of powers.

    Why not try reading what my comment actually says before getting yourself all worked up?

  34. @John B,

    Because it’s sparsely populated and property is cheap.

  35. John B- where’s your evidence that ‘so many’ English people (or, indeed, people of any nationality frankly) want to move to Scotland? The rare few that do only consider it due to what Neil said- it’s cheap.

    I also want to support Candy in her 11:34 post. The non constituency threads on this site are rendered unreadable by a cabal of chip on the shoulder, inarticulate Sturgeon bumlickers who shoehorn Scotland into every sodding conversation. It’s pitiful and Anthony should have banned these pub bores ages ago. The saner members here will know who I’m referring to.

  36. @Tristan

    Yes, the real bores are people who attempt to assess what impact an ill thought out law might have on public opinion (you know, doing what the site is for), not those who, from some kind of obsessive nationalist spite, insist on vomiting out the same garbage they’ve been recycling for years whenever a certain country is mentioned.

  37. Yawn. Try again.

  38. The point about the Sewel convention being: that (some) people believe EVEL will stop a Labour or a Labour/ SNP coalition from governing. It won’t because the bill can be structured so that elements which apply to Scotland can be woven into any bill put forward by the government/ governing coalition – MSPs permitting!

    This will make it more than likely that all bills put forward by the (hypothetical, at this point) government/ governing coalition will be passed straight to the whole house by the speaker.

    EVEL will not be understood by many people, except in a very general way. It will likely make it more difficult for the Lynton Crosby English strategy – vote Labour, get SNP – to prevail in future elections. So, from this perspective, I am okay with EVEL.

    But, as I said, the EVEL standing order makes things more difficult for people who see themselves as British just because it is another ‘messy compromise’ which will need to be worked around.

  39. Tristan

    I was quoting Candy. And (again according to Candy) the English were sufficiently large a proportion of the Scottish population to have had a big say in the outcome of the Indyref. That’s what Candy said. So it’s not a ‘rare few’ but a sizeable proportion of Scotland’s population. And it isn’t only because property is (sometimes) cheap (otherwise there are plenty of places in England which would qualify as well); perhaps quality of life counts for something as well!

    The chip, by the way, seems to be on the shoulder of those who wish there was nothing more to the world than their own Little England. :-)

  40. @Tristan

    “Yawn. Try again”

    See, one post later and you’re already bored ;)


  41. @JohnB

    Actually if you re-read what I wrote – I said the census showed that the increase of 1.3% in the Scottish population since 1971 was down to English people arriving north, and that 1.3% voted No.

    Not sure how you made the leap in your mind to 1.3% being “large”!

    My point is that this whinging about EVEL is pointless. If demographic trends continue the UK will essentially be England plus a few tiny pockets of minorities on the fringes by the end of this century, thanks to people voting with their feet.

    This is also the reason why it’s a waste of money setting up an English Parliament. Why bother when the House of Commons will be essentially be all English at the end of the day?

  42. I have come across some strange comments in today’s FT regarding dismay of some Labour moderates at Corbyn’s appointment of Seumas Milne . A quote held against the latter from 2004 is ‘killings of ”occupation troops” in Iraq pale next to the toll inflicted by the (US/UK) occupiers’. Surely what he said was self-evident. Is it seriously suggested that people living in a country that has been the victim of an unprovoked attack have no right to resist their attackers? Would these people have said the same thing about the French Resistance attacking German troops in World War 2? Hypocritical idiots!

  43. As this thread is nominally about voodoo polls, can I suggest a new one?
    Let’s survey the posters here with the following question.

    Discussions about Scotland should be restricted to threads about Scottish polls, yes or no.

    I vote yes, so that’s 100% in favour so far!

  44. P.S. I noticed that the last such thread only had 11 comments, which is the lowest I can remember.

  45. @Pete B


    Even split now – the race is too close to call!

  46. But when you adjust for turnout, Pete Bs are more likely to vote than AUs, so the Yes camp is ahead in the headline figure…

  47. Candy

    With your 11.36 post you have yet again beaten me too it! I would have posted that almost word for word. We really do not need and English parliament, just a proper version of EVEL where only English MP’s can vote on English legislation, not IMO the inadequate version just voted on.

  48. @Neil A

    That is a scandalously unjustified assumption sir. Indeed I am one of the only ten people in the UK who bothers to vote in both local and European elections

  49. @TOH

    Yeah. People are in denial about the effect of population trends.

    Interestingly the Scottish response to their population problem is similar to that of the Eastern Europeans. They too are experiencing a drop in young people due to reduced fertility and young people simply walking out (the overall effect disguised a bit by improved longevity but which should become apparent by 2050 when the big boomer and Gen X generations drop out and the population suddenly collapses) – and they too are going nationalist and trying to chase strangers out.

    And in Russia, where this trend has been going longest (their population has been declining since the 90’s), they’ve not only gone ultra-nationalist but are now resorting to conscripting a precious resource, young men, and sending them to their deaths in Ukraine and Syria in an effort to pretend they’re as powerful as ever.

    It’s odd self-destructive behaviour and it would be interesting to hear how the anthropologists explain it.

1 2 3 4 5