I hope most of my regular readers would assume a Daily Express headline about a “poll” showing 80% of people want to leave the EU was nonsense anyway, but it’s a new year, a new election campaign, and it’s probably worth writing again about why these things are worthless and misleading as measures of public opinion. If nothing else, it will give people an explanation to point rather overexcited people on Twitter towards.

The Express headline is “80% want to quit the EU, Biggest poll in 40 years boosts Daily Express crusade”. This doesn’t actually refer to a sampled and weighted opinion poll, but to a campaign run by two Tory MPs (Peter Bone and Philip Hollobone) and a Tory candidate (Thomas Pursglove) consisting of them delivering their own ballot papers to houses in their constituencies. They apparently got about 14,000 responses, which is impressive as a campaigning exercise, but doesn’t suddenly make it a meaningful measure of public opinion.

Polls are meaningful only to the extent that they are representative of the wider public – if they contain the correct proportions of people of different ages, of men and women, of different social classes and incomes and from different parts of the country as the population as a whole then we hope they should also hold the same views of the population as a whole. Just getting a lot of people to take part does not in any way guarantee that the balance of people who end up taking the poll will be representative.

I expect lots of people who aren’t familiar with how polling works will see a claim like this, see that 14,000 took part, and think it must therefore be meaningful (in the same way, a naive criticism of polls is often that they only interview 1000 people). The best example of why this doesn’t work was the polling for the 1936 Presidential election in the USA, which heralded modern polling and tested big sample sizes to destruction. Back then the most well known poll was that done by a magazine, the Literary Digest. The Literary Digest too sent out ballot papers to as many people as it could – it sent them to its subscribers, to other subscription lists, to everyone in the phone directory, to everyone with a car, etc, etc. In 1936 it sent out 10 million ballot papers and received two point four million responses. Based on these replies, they confidently predicted that the Republican candidate Alf Landon would win the election. Meanwhile the then little known George Gallup interviewed just a few thousand people, but using proper demographic quotas to get a sample that was representative of the American public. Gallup’s data predicted a landslide win for the Democrat candidate Franklin D Roosevelt. Gallup was of course right, the Literary Digest embarrassingly wrong. The reason was that the Literary Digest’s huge sample of 2.4 million was drawn from the sort of people who had telephones, cars and magazine subscriptions and, in depression era America, these people voted Republican.

Coming back to the Express’s “poll”, a campaign about leaving Europe run by three Tory election candidates in the East Midlands is likely to largely be responded to by Conservative sympathisers with strong views about Europe, hence the result. Luckily we have lots of properly conducted polls that are sampled and weighted to be representative of whole British public and they consistently show a different picture. There are some differences between different companies – YouGov ask it a couple of time a month and find support for leaving the EU varying between 37% and 44%, Survation asked a couple of months ago and found support for leaving at 47%, Opinium have shown it as high as 48%. For those still entranced by large sample sizes, Lord Ashcroft did a poll of 20,000 people on the subject of Europe last year (strangely larger than the Express’s “largest poll for 40 years”!) and found people splitting down the middle 41% stay – 41% leave.

And that’s about where we are – there’s some difference between different pollsters, but the broad picture is that the British public are NOT overwhelmingly in favour of leaving the EU, they are pretty evenly divided over whether to stay in the European Union or not.


220 Responses to “No, polling doesn’t show 80% of people want to leave the EU”

1 2 3 4 5
  1. John Curtice makes the point that “Vote SNP, get Cameron” is a hard line to sell for Labour, given that Cameron is more popular in Scotland than Miliband.

  2. Colin

    Excuse for what.

  3. Re Scotland, I’ve done a post on it. The short of it is that Panelbase were a bit naughty with a “yes” ladder, MORI’s huge lead is still slightly smaller than last time and not politically weighted (which I dislike) and Survation is nothing to write home about.

    The long version is here:

    http://www.ncpolitics.uk/2015/01/scotland-snp-lead-10-points-28-points.html/

  4. AW

    Thanks-kind of anticipated you saying that !

  5. @ Anthony @ Ben

    I spoke to the guy responsible for that poll – it isn’t scientific, even though the numbers (once you exclude the DKs) look relatively believable.

  6. The chairman of Young Independence (the UKIP Student group) has been selected to fight Sheffield Hallam for them. National news ‘cos it’s a national group!

  7. Jayblanc

    There was discussion on NHS Scotland and NHS England here recently, when everyone agreed that funding for NHS England was an important contributor to the Scottish Government Block Grant.

    Hardly seems worthwhile your making that point earlier, since there was nothing in Sturgeon’s statement that supported your wider assertions.

  8. I think I suggested this before but one possibility that might come out of the EVEL debate is for a more formal split between the Budget and Annual spending statement.

    The budget would deal with revenue raising and borrowing with Barnet deciding how much each nation gets but it would be up to each country to decide how to spend it.

    That way changes to NHS spending in England would have a knock on for other spending in England but not Scotland.

    Peter.

  9. @Mr Nameless

    Had he stood in a seat a few hundred miles further north, as chair of a group with a name like that he might have stood a fair chance.

  10. Number Cruncher

    Good post – but a few minor points.

    Re “naughty” Panelbase – shouldn’t you have described this poll as a “Yes snake” or a “No ladder”? :-)

    Re lack of political weighting by MORI – I prefer good use of political weighting too, but avoiding it altogether seems better than bad use of it, like Populus online. I imagine Susan Dalgety is already asking them to conduct a Scottish poll!

    Satisfaction ratings> – As expected, people’s views of politicians correlates strongly with their VI in most cases. However, Sturgeon and Swinney are highly (or at least neutrally) rated across the board, while attitudes to others seems to break along referendum lines.

    As for the UK leaders, only those planning to vote for their party already express satisfaction with them, and even Labour supporters only give Mklkband a net +1.

    The Murphy factor – Again, as expected, this breaks along party VI lines with existing Lab voters “more” and existing SNP voters “less” likely to vote Lab because of his election – a pretty worthless observation IMHO.

  11. @ Colin

    Largely agree with you on ECB.

    Just one (important) nuance the Bundesbank was not a lender of last resort (either formally or in practice) in West Germany or the unified Germany until the Euro.

  12. Peter Cairns

    Presumably any party that wanted to balance the budget would approve of your suggestion that English domestic affairs should be run within a fixed budget, as the devolved administrations have to do.

    Wouldn’t a similar limit have to be set on non-identifiable expenditure? The Treasury can be quite imaginative when it comes to paying for London infrastructure out of UK funds.

  13. Peter Cairns

    Each “nation ” should get an amount exactly in proportion to the population of that “nation”. This nonsense of the Barnett formula should be scrapped forthwith and this would ensure fairness across the whole nation – that is the United Kingdom of Great Britain and Northern Ireland.
    Nationalist playground economics has been with us since Mel Gibson was a lad but does none of us any good whether we live in Lands End , John O’Groats or Llanelli.

  14. UNICORN
    @ Allan Christie
    “According to the Electoral Calculus… Renfrewshire East Glasgow area NAT gain from LAB : Jim Murphy.”
    __________
    If I were you, I wouldn’t place much confidence in Electoral Calculus seat projections. They were furthest off target in predicting the last set of Ashcroft constituency polls – worse even than UNS.

    To assess their vote reallocation algorithms, take a look at their prediction for Clacton, where they judge that Ukip has a 0.6% chance of winning.

    Electionforecast – with its better recent track record – predicts a comfortable win for Labour in this seat.
    I know your were just trying to wind people up. But you need to use a better source of evidence if you want to do that effectively.
    _____

    Me wind people up? That’s not in my waffling remit lol.

    Yeah I take on board with reference to the Electoral Calculus seat projections but the Scotland votes website also forecasts a SNP victory in East Ren.

    I’m particularly interested in Jim Murphy’s constituency because its also the one in which I live and one of the few councils where the SNP and Labour share power.

    He has a strong personal vote (after all he is rather to the right and this being an ex Tory seat and all that) suits some of the blues and his support for the Iraq war goes down well with the Jewish voters who make up the largest Jewish community outside of London.

  15. JASPER22
    Each “nation ” should get an amount exactly in proportion to the population of that “nation”.

    Is that a new take on Communism? ….. From each according to his/her abilities; to each according to his/her numbers?

    And presumably HS2 would automatically become English rather than UKish under your plans, or would it?

  16. YouGov/Sun poll tonight – Labour lead by one: CON 33%, LAB 34%, LD 6%, UKIP 14%, GRN 8%

  17. YouGov/Sun poll tonight – Labour lead by one: CON 33%, LAB 34%, LD 6%, UKIP 14%, GRN 8%

  18. not even 2 in a row.

  19. Another poll and Greens ahead of the Libs. Surely a podium spot now on the debates??

  20. JIM JAM

    Remember the polls are a little bit too Laboury according to Rob Hayward.

  21. Any government with enough commonsense/simplistic enough/ courageous enough (literally or in Sir Humphrey’s sense) to replace the Barnett Formula with simple population proportionality would scrap HS2.

  22. I posted on that notion on that thread Allan but as is all too common it was surrounded by a sea of Saltire comments.

    If the polls over-estamated Labour than they did before last nights 30 and do so now with the 34 but both suggest 32ish on YG. (maybe a touch higher on average)
    The point is the 30 on YG does not appear to be anything other than MOE as the 34 will probably prove to be,

  23. Labour 4% surge alert.

    What’s happened? Have the Greens imploded?

  24. STV will be releasing further details from their MORI poll tomorrow – on reaction to Smith Commission proposals. Scots dubious about them, it seems.

    Timed to match UK’s response to them.

  25. No they haven’t – they’re still doing quite well – almost certainly we have to assume that yesterday’s polls was an oddity, unless some very rapid corroborative evidence appears, and in some quantity too.
    If Labour actually gets 34% in the election there would be virtually no way for the Tories to stay in government

  26. @AC & CMJ
    YouGov/Sun poll tonight: Labour somewhere between about 5 ahead and 3 behind.

  27. JIM JAM
    “I posted on that notion on that thread Allan but as is all too common it was surrounded by a sea of Saltire comments”
    ___

    Aye see them Brigadoons and all that flag waving. ;-)

    “If the polls over-estamated Labour than they did before last nights 30 and do so now with the 34 but both suggest 32ish on YG. (maybe a touch higher on average)
    The point is the 30 on YG does not appear to be anything other than MOE as the 34 will probably prove to be”
    ______

    Yes I agree with this.

  28. “not even 2 in a row.”

    Probly wasn’t even one in a row in reality: still those odd ones do offer some people a bit of excitement and some time with their calculators and predictions.

  29. @OldNat

    I’m not sure what value to attach to Ipsos-Mori’s Scotland polls. They do seem the most extreme by some margin.

    Who knows though? They might be right.

  30. @ RAF,

    ‘Can someone explain to me what EB adds to the Labour ticket?’

    Reaction gifs and Twitter memes, obviously. This is a key part of political campaigning in the age of social media.

    Also he bakes and plays the piano.

    @ Bill Patrick,

    I think that Ed Balls’s key contribution to Labour is that he’s an effective parliamentarian and shadow minister. I think that Labour’s problems on the economy have come more from lacking a consistent and sustainable narrative than anything else.

    Er… surely part of the Shadow Chancellor’s job is to come up with a consistent and sustainable narrative on the economy?

  31. Smith proposals ? The non flag waving majority in Wales, as those post referendum polls showed, want no truck with independence, are mightily sceptical about the Assembly’s achievements under its “closet Plaid” administration, and will sleep very easily in their beds if there is NO further devolution of Westminster’s UK powers to anyone whether in Brussels, Edinburgh, Cardiff or Manchester. But who do they vote for to get that apart from the purples ?

  32. @Bill Patrick

    “John Curtice makes the point that “Vote SNP, get Cameron” is a hard line to sell for Labour, given that Cameron is more popular in Scotland than Miliband.”

    Two points about that. Firstly, Cameron, at minus 40 is “marginally less disliked” than Miliband at -40; “more popular” is a ludicrous interpretation of those figures. Secondly, it isn’t “Vote SNP, get Cameron”, it’s “Vote SNP, get the Tories.” Big difference.

    Apart from that, I thought it was a great post.

    :-)

  33. @Batty
    I presume that was sarcasm. FYI the 8% was the Greens highest on YG before yesterday, and the Greens average (mean) over the last 5 polls is now 9%, a record high.

    @AW
    given that Survation report to one decimal place, I didn’t take that as conclusive (if it was two, I would have known immediately, I come across far too many academic papers with that unjustified precision from people who haven’t got a clue what is appropriate for a sample in the hundreds). So…
    @NumberCruncher
    Thanks for the evidence.

  34. Jasper

    “This nonsense of the Barnett formula should be scrapped forthwith and this would ensure fairness across the whole nation”

    Like it or not all the main UK parties in Better Together signed up for Barnett in “The Vow”, so it is with us for the next Parliament at least, unless one or more of them renege on the Vow.

    I can understand lots of people in England not liking it or indeed there own MP’s at Westminster and their various Parties ignoring that fact, but it’s hardly the SNP’s fault that people elected by English voters don’t seem to be representing there views.

    Allan,

    ” Largest Jewish community outside of London.”

    Where did you get that from? I found a figure for only 6,000 Jewish people in the whole of Scotland but about 30,000 in Greater Manchester alone!

    I am not sure Murphy is at risk but I don’t accept the idea that lots of Tories vote for Murphy.

    I suspect lots of SNP and LibDems vote for him to keep the Tories out. If they think the SNP can win will the SNP ones still vote tactically for Labour.

    Will LibDems who don’t like the coalition stick with Labour or go to the SNP.

    What about Tories? Will they stay true blue or will they switch to the SNP in the hope of hurting Labour.

    Peter.

  35. Sorry, that should have been Miliband at minus 42.

  36. @Ben Foley

    It was indeed sarcasm; the very lowest form of wit! A slight reaction to the over-excitement caused by yesterday’s poll that generated a dedicated thread and 150 fevered and saliva-flecked posts; all within an hour.

    Another day, another poll and yesterday’s YouGov is now today’s fish and chip paper.

    Disposable verdicts, discarded with each passing poll.

    Time for keeping calm and carrying on, I think.

  37. RAF

    Re Ipsos-MORI

    We all know that the methodologies of different pollsters produce different margins, so presuming that any one of them is “correct” would be daft.

    That all pollsters show a significant SNP lead over LiS is, however, a secure conclusion.

    There doesn’t seem to be much of a reduction in that SNP lead so far, and it will be interesting to see if the LiS rebranding as a “patriotic party” allows them to steal some SNP clothes, or whether it keeps voters focus on looking through a Scottish, rather than UK, prism – a situation that could backfire on LiS.

    We’ll wait and see!

  38. @Ben

    My initial feeling was yesterdays poll was a high point within MOE for the Greens.

    The Green VI is rising slowly, but not at a consistent 10%+ yet.

    Yesterdays poll had a small 18-24 sample, probably more prone to a high Green response, and then weighted upwards, enlarging the ‘spike’.

    I will check the tables tomorrow looking for this demographic.

  39. @ Allan Christie

    “…the Scotland votes website also forecasts a SNP victory in East Ren.”

    I could well be wrong, but the Scotland Votes ‘predictor’ looks as if it is just a straightforward UNS calculation. If so, I wouldn’t set much store by that either.

    Any local knowledge you have is probably much more valuable than models of this kind.

  40. @Spearmint

    “Er… surely part of the Shadow Chancellor’s job is to come up with a consistent and sustainable narrative on the economy?”

    I agree. It was consistent and sustainable to start with, and scored reasonably in polling. But then the “too far, too fast” narrative was dropped, after which Labour’s polling on the economy has been downhill.

  41. Welsh Borderer

    “Smith proposals ?”

    They don’t refer to Senedd powers.

    “non flag waving”

    If they don’t want to wave flags, then voting for the flag waving Kippers would be an odd choice.

  42. Unicorn

    I think Scotland Votes largely applies a Scottish UNS model – but does have some “gut feeling” variations – maybe restricted to O&S and Charlie Kennedy!

  43. Crossbatt
    ” A slight reaction to the over-excitement caused by yesterday’s poll that generated a dedicated thread and 150 fevered and saliva-flecked posts; all within an hour.”

    And I missed it. Bugger. Still it’s only a matter of time till permanent crossover happens. It will be 1992 all over again.

  44. P Cairns
    I restrained myself from immediate response about the comment on Jewish people in East Renfrewshire but appreciate your comment. In fact at most 1.5% of Jewish people in the UK live in East Renfrewshire.

  45. Batty

    Je pense nous sommes sur le same page: the reaction yesterday was really quite odd. I prefer a much broader analysis both in time and logic.

    Still, c’est la vie said the old folks.

  46. Crossbat11,

    Would you prefer “Miliband is even more unpopular in Scotland than Cameron?”

  47. @Robert Newark

    “And I missed it. Bugger. Still it’s only a matter of time till permanent crossover happens. It will be 1992 all over again.”

    Ah yes, 1992 and all that. That’s the Tories equivalent of Villa’s 1982 European Cup triumph. A great day, now all but lost in the mists of time, long gone and barely remembered by all but a few old nostalgics, still pining for the glory days.

    It’s an understandable human trait to wish for the good old days again. All our yesterdays can produce comforting thoughts, but they rarely return.

  48. @Bill P

    “Crossbat11,
    Would you prefer “Miliband is even more unpopular in Scotland than Cameron?””

    Yes, I would.

    Thank you.

  49. @ Phil Haines,

    To be fair, the collapse in Labour’s economic ratings was also correlated with the economy finally beginning to pull up. It’s hard to tell how much was due to policy incoherence and how much was due to the mounting evidence that Osborne wasn’t presiding over a permanent recession.

1 2 3 4 5