It’s been a long journey, but we’ve finally arrived at the eve-of-referendum polls. For a lot of the Scottish referendum campaign the discussion about polls was one of right or wrong – we had lots of polls showing the same trend (flatlining!), but showing different absolute figures. Companies like MORI, TNS and YouGov were showing big NO leads; companies like Panelbase and Survation were showing a tight race. Then we had a period of some companies showing a strong movement towards YES, some not, and we have ended up with everyone showing much the same figures (what was the true picture earlier in the campaign we will never know for sure – by definition you can check eve-of-election results against reality, but never mid-term ones). With one MORI poll still to come, here are the YES shares in the latest polls from each company (taking the online and telephone methodologies seperately for those companies who have done both):

Ipsos MORI (phone) 49%
ICM (phone) 49%
TNS (face to face) 49%
YouGov (online) 48%
Panelbase (online) 48%
ICM (online) 48%
Opinium (online) 48%
Survation (online) 48%
Survation (phone) 47%%

Essentially everyone is predicting the same result, the margin of error on most of the polls is around plus/minus 3%, every poll is within two percentage points of the others. This isn’t going to be a case of individual pollsters getting it right or wrong, they’ll either all be around about right or all be horribly out.

There’s a temptation when the polls are like this to say YES and NO are within the margin of error, that it’s “too close to call”. It doesn’t really work like that – these polls are showing NO ahead. The margin of error is on each individual poll, and it’s equally likely to happen in both directions. Hence if the “true” balance of public opinion in Scotland was 50/50 we’d expect to see a random scattering of results around that point, some polls showing yes, some polls showing no. We’re not seeing that. We’re seeing polls randomly scattered around the 48/52 mark, suggesting that’s most likely where public opinion is – a very small lead for the NO campaign.

It’s possible there will be a very late swing, that people will have changed their minds in the last few hours or in the polling station itself. In most polls there really aren’t that many don’t knows left to make their minds up though.

The alternative route to an upset is if the polls are wrong, if there is some systemic issue above and beyond normal random sampling error that affects polls from all the companies. I wrote yesterday about what the potential risks are – the main challenges in my view are first whether people who are on the fringes of society and normally play little part in politics don’t get picked up in polls but do vote; and secondly whether there has been an issue of differential response rate, have the obviously more enthusiastic yes voters been more willing to take part in polling that no voters are?

Personally I’m a little more worried about the latter – I think there’s more chance of the polls ending up underestimating the NO vote than the YES vote, but there comes a time when you just have to trust the data. The polls say the result will be around about YES 48%, NO 52%. We will see on Friday morning.


166 Responses to “What the final polls tell us”

1 2 3 4
  1. RogerH

    Probably a mistake to just see it in party political terms. Labour in Scotland essentially said that Westminster could be trusted to deliver, as they stood side by side with Tory & Lib Dem.

    Tory MPs NOT reneging was part of the “vow”. If they’re all going to be Clegg ……

  2. Jonathan Bartlett

    Okay, then I won’t send in the police round to search for Statgeek’s corpse. but it’s too confusing that he has a statgeek website, too. Could you change your URL to something more, ummm, obviously bio-statistical in nature to make the distinction clear. :-)

  3. Pete B: Scottish tax raised goes straight to Scotland.

    So the Scottish budget is mostly determined by the Barnett formula, and it gets its money from Westminster. But that’s going to be minue about half the income tax revenues, so the Westminster funding will give them the equivalent of if everyone paid 10% income tax at basic rate. The remaining 10% income tax is then collected directly in Scotland to make it up to the UK’s 20% basic rate – although the Scots could then set that to 15% or 5%.

    It’s an extension of the current system whereby the Scottish government has powers to boost or lower income tax by up to 3% with knock-on effects direct to their budget (a power Salmond has refused to use, despite complaining not enough is being spent on services; the SNP’s budget problems are thus partly engineered, one feels).

    I may be wrong about all that, but that’s roughly how it works if I’ve got it correctly :)

  4. “TheStatsGeek and Statgeek are not the same person.”

    We only have their word for that.

  5. I hope whatever happens with all these vows and promises, that the rest of the UK is not forgotten. There ought be something resembling an English parliament coming out of this: I personally would not be very satisfied by Westminster with a (Scotland-shaped) slice taken out.

    With a GE coming up there might be some promises of a future vision for the nation’s constitution beyond immediate (and large) economic and social problems. Certain parties have mumbled about this sort of thing before, but only done things that suit them and/or had other things to concern themselves with, eg the EU question.

    If Clacton causes a big enough earthquake, then I fear these internal changes might never happen. That would be a shame, but hardly surprising.

  6. missis minty

    “morrisey….. captured”

    I think it should say “dead or alive” on the poster.

    he is one of the few people I don’t know that I really despise.

  7. @Muddy

    His site is far more statty than mine:

    “Quantifying heterogeneity: tau^2 = 0; H = 1 [1; 1]; I^2 = 0% [0%; 0%]”

    …now that’s statty!

    My site is (pretty much) HTML5 valid though…cos I’m more geeky than statty, and I like pretty charts. :))

  8. Isn’t HTML5 just called html now? ‘cos the faint praise was getting a bit much?

  9. James Baillie,

    “the SNP’s budget problems are thus partly engineered, one feels”

    Though some would sooner believe that Westminster would abandon the Barnett formula rather than see Holyrood raise income tax and spend more on public services…

  10. Has the electoral commission made any comment on what contingencies are in place should turnout he very high later in the day?

    Regardless of which result you want, the thought of the result being so close that those who tried to vote and couldn’t being a large enough number to potentially have influenced the outcome is too ugly to contemplate.

  11. Gordon:

    “Above all, though, Brown became his father’s son. This wasn’t so much a speech as a prayer. An invocation worthy of any southern Baptist revivalist preacher. He implored. He beseeched. He cajoled. With his hands locked together, he looked heavenwards. What two world wars had brought together “let no narrow nationalisation split asunder”. With that, he fell to his knees, as if expecting a bolt of lightning to strike the roof of the community hall and for the newly crowned Queen of Scots, aka JK Rowling, to appear in a golden chariot. Brown had saved his best for last and if this was to be his political epitaph it was a hell of a way to sign off. A lot better than Donald Dewar’s for sure.”

    The poetic amongst you will recognise the subtly hidden QUADRUPLE alliteration of the Ns in this:

    “LET NO NARROW NATIONALISM SPLIT ASUNDER”

    This is genius and the turning point of the whole campaign.

    It makes Bully, Bluff and Bluster seem like a terribly tired trio trying too hard…… [see I’ve dunna quintuple with pike now………]

  12. Old habit. Used to be fussy about XHTML 1.0 and 1.1 and all the strict / transitional, so just in the habit of adding the 5.

    I worry that someone will accuse me of using 4.01, as I have a couple of pals with sites that use XHTML 1.0 Transitional and refuse to take the plunge despite it being a five minute thing to change. They still design with HTML 4 and tables for layout and so on…

  13. @James Baillie

    I rather like your counterfactual, which is not implausible though no more likely than a squillion and one other possibilities.

    I posted my own counterfactual 5 or 6 days ago. Slightly less plausible than yours, but much more “interesting”.

  14. @R&D

    It’s a pity he never turns up to the HoC and makes speeches like that, and actually earns his wages / expenses. If he (and others) did, including the Lords, the Yes side of the debate would have less ammo to use against them (Westminster).

  15. Bing is now showing 51.3% No and 48.7% Yes. That’s another swing to Yes – up by 0.6% since yesterday.

    My personal feeling is that yes will do it, by a tiny margin – I think it will be 50.2% Yes 49.8% No. Hope I’m wrong, but it really looks like momentum is going the Yes way.

  16. @Candy

    “Bing” said Zebedee… time for bed ;)

  17. My other prediction is there is no stomach for further referendums for a long time. That includes EU referendums. Judging by the SNP people, can you imagine how animated and intimidating the Kippers will get in an EU referendum?

  18. SAVVY

  19. I think we are all losing site of the postal votes. These were cast some while ago at a time when the No vote was much stronger.
    Assuming that the postal votes mirror those polls when they were cast, there is a dormant boost to the No’s in these votes already cast.
    I understand the postal vote is around 600,00, so on that basis there is a large potential boost for the NO campaign.
    Did any of the Polls take that into account?

  20. Chrishornet

    Anyone in a queue to enter the polling station at 10 pm must be allowed to vote. A member of the polling station staff will identify the end of the queue at exactly 10pm. Anyone arriving later cannot vote.

  21. Still think No to win by at least 8% points at least.

  22. So today’s the day. I’ve made some tweaks to my early results benchmarks in response to feedback. To recap, the first version of the model used the dispersion from the 1997 devolution referendum. This had the benefit of using known, real vote tallies, motivated by the uniformity of the swing between the 1979 and 1997 referenda. But disconnect between devolution and independance views led to odd results in certain areas.

    The second version used the estimates of the 2010 westminster vote shares by local authority combined with partisan yes/no splits from opinion polls. This gave much more intuitive results than the first version, but had no way of adjusting for regional patterns specific to this referendum.

    This third version includes uniform adjustments at regional level based on polling data. This has similar caveats to the second version.

    Areas where I’m particularly cautious include the islands – the regional adjustment is based on polling of the highlands and islands region but clearly the islands could be very different to the highlands and some people have indeed suggested this.

    Here is the updated dispersion, normalised to a 50-50 national tie, with the previous two methods for comparison:

    http://numbercruncheruk.blogspot.co.uk/2014/09/how-should-we-interpret-early-results-III.html#more

    As always, feedback is welcome.

    I’ll be online through the night to comment on how the results shape up as they come in.

  23. Statgeek

    Ah, the gentle poetry of variance calculations in a random-effects meta-analysis. We don’t have enough of that kind of thing here.

  24. @ Old Nat

    “Anyone in a queue to enter the polling station at 10 pm must be allowed to vote. A member of the polling station staff will identify the end of the queue at exactly 10pm. Anyone arriving later cannot vote.”

    Good. That’s how it should be.

    Are you going to be out doing GOTV today?

  25. This may give us a hint about the DKs… It looks like it was done as part of the YouGov poll (apols if someone picked up in this in an earlier thread)

    h ttp://www.buzzfeed.com/jimwaterson/scotlands-no-voters-feel-threatened-afraid-to-speak-out-and

    Buzzfeed involved in polling!

  26. @SoCalLiberal

    Yes, it should be that way, but often isn’t. Underfunding of elections isn’t something new on either side of the Atlantic….

  27. number cruncher

    “feedback is welcome”

    You are Pete Townsend and I claim my five guineas.

    [Today is the first day of things carryin’ on the way they’ve always done: dead exciting I think so will go back to sleep.]

  28. I’m a bit wary of using the Quebec polling disaster to inform a general rule about referendum polling.

    Here’s another Canadian mega polling disaster, from just a couple of years ago, with some of the same companies.

    It may be that Canadian polling is just rubbish.

    http://en.wikipedia.org/wiki/Alberta_general_election,_2012

  29. Phil:

    Maybe, maybe not…

    http://t.co/CYpy1PBFQO

  30. @Numbers

    Yes, I knew that, but the Quebec one is the one that I hear people citing.

    Great analysis of yours btw.

  31. Thanks! Now we just need some results…

  32. Concrete evidence to support the shy no hypothesis?

    http://www.independent.co.uk/news/uk/scottish-independence/scottish-independence-almost-half-of-no-voters-have-felt-personally-threatened-by-the-yes-campaign-9739038.html

    “It found that a strikingly high 46 per cent of No supporters felt personally threatened by the Yes campaign during the referendum, while 50 per cent did not. In contrast, only 24 per cent of Yes supporters felt personally threatened by the No campaign and 72 per cent did not (“don’t knows” excluded).

    Almost half of No voters (49 per cent) said they had not always felt able to speak freely about their views on the referendum, while 48 per cent felt they had. Meanwhile, 76 per cent of Yes backers believed they had always felt able to speak freely, and only 21 per cent did not.

    Opponents of independence were also more likely to have fallen out with friends, family members, colleagues and other people than were supporters of Scotland leaving the Union.”

    Given this is a survey of the voting public, rather than campaigners or actiivists it at the very least puts a lie to the yes supporters on here who have dismissed reports of such an atmosphere as a product of the biased english press from afar.
    It certainly makes hte much speculated idea that people would be wary of expressing their intention before the ballot to vote no a lot more plausible

  33. YouGov

    Those displaying posters/stickers/badges:
    Yes 47%
    No 20%

    29% of Noes who didn’t say it’s because they were worried about getting a negative reaction from others, compared to 6% of Yesses. That accounts for much of the difference.

  34. @Gattino

    Glad you found a reference to it that wasn’t buzzfeed! It feels a lot more respectable now…

  35. Gattino – not sure I’d call that concrete, but it’s very strong anecdotal evidence. The question is, how big is the shy “no” effect, and how much of it (if any) is offset by don’t-normally-vote “yes” voters not being picked up by the polls.

    My guess is that No might be a couple of points higher than the polls, but who knows…

  36. I’m sure someone said it was Anthony’s birthday on referendum day.

    So Happy Birthday Anthony but before anyone follows my lead you better check your facts out…

    Hope you got all the spreadsheets your heart desires and have a great day.

  37. I’ve found from the No attacking Yes for “splitting up Scotland” across social media that most are just shocked to see other people disagree with them.

    It’s also impossible for in such a split race that more on one side have fallen out with family and friends. Quite literally impossible.

    On shy nos? This is a poll of nos and cannot represent anyone lying on their voting intention.

  38. The poll, if I’ve not misread it, is by yougov. Buzzfeed merely commissioned it.

    If I’ve misread or misunderstood then I hold my hands up.

  39. To all those voting today, may I wish you the best of luck.

  40. I wonder if it could be rather like 1987, with the Yes campaign playing the part of the Labour Party. The Noes have had their rogue poll and ‘wobbly Thursday’ but the fundamentals remain in their favour.

    55-60 for the Noes (but as much in hope as expectation).

  41. Thanks for posting on here Mr Chruncher

    What do you (or others here) say… if we look at say Clackmannanshire and Dunfries & Galloway in the early hours, will we be able to say something meaningful about the respective strengths of Yes and No?

  42. @Number Cruncher

    I’m working on something of my own, and as part of that I have already discounted the Shetlands and Orkneys as part of that analysis, due to being extreme outliers.

    Seeing as they represent under 1 % of the voting population, I don’t think it matters.

    I will read your info later with interest.

  43. Cruncher

  44. Andy Murray is going to be boo’d to oblivion come next summer.

  45. @”Tory MPs NOT reneging was part of the “vow”.”

    Indeed-what David Aaronovitch calls “having everyone else’s cake and eating it”.

    And the cake apparently includes Barnett Formula in perpetuity-a formula which gives Scots 16% more per head of public spending than rUK; and which is now described as grossly unfair by its author. Barnett devised it 36 years ago a a temporary stop gap.

    So it isn’t surprising at all that senior members of the government , ( as opposed to the former PM who “pledged “all this stuff) are thinking about the reaction of their MPs when , at last, they get to debate & vote on it.

    And interesting straws in the wind they are too.

    Clegg & Cameron have recognised that more power for Holyrood means the West Lothian Question has to be answered.
    And it sounds as though they both favour the McKay Report conclusion-Clegg says it is a “starting point” -Cameron says it is a “good report”.

    McKay proposes English MP only voting majorities for English only laws. That neither NC nor DC have given any signal that a separate English Parliament is a prospect is hardly surprising given the antipathy shown over Regional Government, Elected Mayors & Elected Police Commissioners.

    So the game looks set-Gordon’s Pledge for McKay.

    ….and what do Labour say about this?-that it will resist all but minor changes to Westminster voting power and that they will oppose anything that creates “two classes of MP”.

    See what you’ve done Gordon?-the fun & games starts soon. Clegg says proposals will be tabled by January.

  46. @ Billy Bob – Yes, I’d say those two should tell you something about as meaningful as you’re going to get at that early stage…

    @ Catmanjeff – I’ve discounted them, or rather given up on trying to predict them! I look forward to reading yours as well.

  47. @ Colin – I can see where this McKay thing is going, soon you’ll have people in London complaining about being ruled by a Holyrood elite that they didn’t vote for :)

  48. Number Cruncher

    :-)

  49. I heard there was a last poll today, from IPos or somebody?

  50. Yes, around 10am I hear…

1 2 3 4