It’s been a long journey, but we’ve finally arrived at the eve-of-referendum polls. For a lot of the Scottish referendum campaign the discussion about polls was one of right or wrong – we had lots of polls showing the same trend (flatlining!), but showing different absolute figures. Companies like MORI, TNS and YouGov were showing big NO leads; companies like Panelbase and Survation were showing a tight race. Then we had a period of some companies showing a strong movement towards YES, some not, and we have ended up with everyone showing much the same figures (what was the true picture earlier in the campaign we will never know for sure – by definition you can check eve-of-election results against reality, but never mid-term ones). With one MORI poll still to come, here are the YES shares in the latest polls from each company (taking the online and telephone methodologies seperately for those companies who have done both):

Ipsos MORI (phone) 49%
ICM (phone) 49%
TNS (face to face) 49%
YouGov (online) 48%
Panelbase (online) 48%
ICM (online) 48%
Opinium (online) 48%
Survation (online) 48%
Survation (phone) 47%%

Essentially everyone is predicting the same result, the margin of error on most of the polls is around plus/minus 3%, every poll is within two percentage points of the others. This isn’t going to be a case of individual pollsters getting it right or wrong, they’ll either all be around about right or all be horribly out.

There’s a temptation when the polls are like this to say YES and NO are within the margin of error, that it’s “too close to call”. It doesn’t really work like that – these polls are showing NO ahead. The margin of error is on each individual poll, and it’s equally likely to happen in both directions. Hence if the “true” balance of public opinion in Scotland was 50/50 we’d expect to see a random scattering of results around that point, some polls showing yes, some polls showing no. We’re not seeing that. We’re seeing polls randomly scattered around the 48/52 mark, suggesting that’s most likely where public opinion is – a very small lead for the NO campaign.

It’s possible there will be a very late swing, that people will have changed their minds in the last few hours or in the polling station itself. In most polls there really aren’t that many don’t knows left to make their minds up though.

The alternative route to an upset is if the polls are wrong, if there is some systemic issue above and beyond normal random sampling error that affects polls from all the companies. I wrote yesterday about what the potential risks are – the main challenges in my view are first whether people who are on the fringes of society and normally play little part in politics don’t get picked up in polls but do vote; and secondly whether there has been an issue of differential response rate, have the obviously more enthusiastic yes voters been more willing to take part in polling that no voters are?

Personally I’m a little more worried about the latter – I think there’s more chance of the polls ending up underestimating the NO vote than the YES vote, but there comes a time when you just have to trust the data. The polls say the result will be around about YES 48%, NO 52%. We will see on Friday morning.


166 Responses to “What the final polls tell us”

1 2 3 4
  1. Courtesy of Nate Silver

    15 Clarifying questions for close elections (excluding not applicable questions)

    http://fivethirtyeight.com/features/15-essential-questions-to-ask-in-close/

    1. Which candidate’s supporters are more enthusiastic?
    Yes.
    2. Which candidate is liable to have the better turnout operation?
    Yes. No are reported to not have a GOTV operation
    4. Do the demographics of the undecided vote, or the weakly-attached third-party vote, favor either of the candidates?
    No – more women undecided and they favour No.
    5. Which candidate got the more favorable coverage in the morning newspaper, or on the local evening news, on the Sunday before the election?
    No – kitchen sink and all. Gordon Brown also delivered a very good last minute speech, although Survation poll up to 9pm didn’t reflect that it moved polls.
    8. Does either candidate begin with a built-in lead from early or absentee voting?
    Not clear. There was a yes surge after the second debate when postal votes started, but mainly older voters postal vote who are voting no
    10. Which candidate stands to benefit from cellphone-only voters, or other voters who may not be represented in the polls?
    Yes. See radical independence canvass of the most deprived areas.
    11. Has the polling in previous elections in the state, or in similar elections in similar states, tended systematically to underestimate the performance of either candidate’s party?
    No – referendum polls tend to overstate yes historically
    12. Which candidate drew more people to their campaign appearances in the state over the last two weeks of the campaign?
    Not sure?
    14. Which candidate has run the more positive, optimistic campaign, and will leave voters feeling better about themselves as they exit the polling place?
    Yes

    Bonus question
    16. Are there any shy voters?
    No’s are reported to be shy, but we haven’t seen real evidence of that yet.

    So that sums up 4 for yes, 4 for no

    Add that to the polls and No wins. That’s my final prediction, unfortunately, as I personally would have liked to have seen Yes win. Lets hope I am wrong like I normally am!

  2. If you want the exact margins of the No leads, see previous thread.

  3. Hope people will forgive a personal view on the referendum.

    My questionable wisdom is:

    The result will be close. It’s fantastic that the turnout will be very high.

    Both options have upsides and downsides. Both options are viable.

    Don’t trust politicians. Both sides have exaggerated the problems of the other option and ignored questions about their own plans.

    Whichever side wins will struggle to deliver on everything that they have promised. There will be disappointment both for those who lose and after a while, for those who win too. Neither option will cure all our problems.

    And most of all remember on Friday morning we will all still be living together across the island of Great Britain and will all need to get along with each other whatever the constitutional arrangement is.

    Good luck and everyone be nice to each other come Friday morning.

  4. @Richard

    Goodness you can write quickly.

  5. My prediction based on…I dunno, call me psychic….54:46 against.

    My preference, Yes to win. I started out hoping for a no (it seemed like the blindingly obvious choice), but frankly have been alienated by the Yesers and annoyed by the ‘throw money at it’ attitude of the Noers. Britain as a whole would be weaker without Scotland, but perhaps individually we’d all be better off?

  6. I was dont know. Then I heard Gordon Brown’s speech and that swung it to NO for me. I know the speech did the same for many others also.

  7. Thanks for the great blog posts Anthony. Regarding your good point about the fact that we have lots of independent poll results all giving almost the same answer, and hence we have less sampling error than the usual +/- 3% figure, I wonder why more use is not made (or maybe it is and I just haven’t seen it) of meta-analysis methods to pool results of polls.

    I did a very a very rough and ready meta-analysis of last week’s polls, results available on my blog:
    http://thestatsgeek.com/2014/09/15/meta-analysis-of-this-weeks-scottish-independence-polls/

    Obviously you can’t meta-analyse polls over a long period, since the underlying quantity being estimated is changing over time as opinion changes, but at least when multiple polls are conducted over the same short period, it seems to me to potentially be useful.

  8. @Phil

    Can’t type that quickly. AW hinted he was about to start a new thread earlier so I waited and pasted it in to avoid Lefty (and my) problem of writing essays on old threads!

    @northumbrianscot

    Agreed with everything you said!

  9. @Mactavish from previous thread re another referendum.

    I’d expect Constitutional and Electoral Reform to be amongst the powers up for discussion for Devolution under a true Devo Max system.

    Either way the SNP would not seriously contemplate another referendum for at least 2-3 Holyrood parliaments unless something significant changed in the meantime (a Yes vote to leave EU at UK level while Scotland voted to remain would be the one possible event I can think that would put it on the table sooner).

    Politically the SNP would suffer from putting another vote in a manifesto too soon and fail to get a Holyrood majority.

    However realistically if/when the SNP got a majority at Holyrood, with another vote in their manifesto, the precedent has now been set constitutionally that a vote must be held. Even if powers to hold referenda had not been permanently devolved from Westminster another temporary agreement would have to be reached.

    The British Constitution works on precedent and now it’s been established it would be politically extremely difficult for Westminster to turn down a revote, whether in 2029, 2034 or 2039.

  10. @ Richard,

    I might dispute 11 – referendums tend to overstate yes, Scottish polling tends to understate the SNP. Which bias trumps?

    (I lean towards the first myself, but the second is still a factor.)

  11. I can’t remember which poll it was, but there was a forced choice for the DK’s, and they split in favour of “No”. Apparently, Yes supporters are targeting DK’s (somehow) and will be attempting to force them to vote – I don’t know if they can know if a DK has spoilt his ballot or something in protest. Even so, this seems a counter-productive tactic.

    The police are going to busy tommorow.

  12. tark (fpt)

    Yes relying on either disastrous pollsters or a hidden swing of extraordinary proportions. The most exciting polldrums in recent memory?

    Well they wouldn’t have to be ‘disastrous’ for Yes to win or indeed No to win clearly which would be just as inaccurate; but they are all huddled together for safety. The danger is that there may be some systemic error which is affecting all of them.

    I think it’s worth pointing out that there not quite as much uniformity as appears if you look at the figures before eliminating the DKs. And we know that most of those those DKs say that they will vote (of course most non-voters tend to put the phone down of pollsters and never join panels). This (and some squeeze questions) hints that DKs may break pretty evenly with a slight tendency to No, but not as strongly as we usually see with referendums.

    So, while the referendum is still No’s to lose (but then they’ve made a pretty good job of doing that), the pollsters won’t have been really humiliated except in a big result either way. But it’s certainly exciting.

  13. I can’t remember which poll it was, but there was a forced choice for the DK’s, and they split in favour of “No”.
    ————
    Panelbase – but there were only 5% DKs.

  14. @ Muddy Waters (fpt),

    Oh, Old Nat deserves to be further up the reeducation list than that!

    Can we submit a recommendation to get him bumped higher? Who do we talk to? Blair McDougall?

  15. Northumbrianscot

    You echo my thoughts exactly.

  16. Many friends of mine that I thought were nailed-on No voters have or are actually only reluctantly voting No. This includes two very firm Tories, one a serial council candidate. They want to give Britain one more chance and see it as a sort of probation.

    If these extra powers don’t amount to anything much, and if there are repeats of the bedroom tax or the anti-foreigner invective, they’ll shift their opinion fast.

    I can well see a situation where SNP plus Green is enough for a majority again (because, let’s be honest, Lamont isn’t going to do any better against Salmond than Gray did), a large slice of the population feel they were duped with the extra powers guff, their generally heightened interest level leads them to see more that they dislike in Westminster politics, and polls start showing a clear lead for indy. The Scottish Government could then arrange a fast referendum- no need for a lengthy campaign when we’ve just had one – and Yes wins with something to spare.

    I think a lot of No voters are doing so while holding their noses, and if the UK government doesn’t deliver, they’ll feel betrayed.

  17. @Spearmint

    I based my answer on this analysis, did not think of the SNP bias.

    http://blog.whatscotlandthinks.org/2014/09/accurate-will-scottish-independence-referendum-polls/

    @KeithP

    I don’t think Yes are targeting DK’s, I think they are targeting those who don’t normally vote. If you look at the radical independence site, their canvass of those areas shows very strong support for Yes amongst those who have been failed by our political system and who don’t normally bother to vote. There is nothing wrong in ensuring that the voice of those people is also heard for once. (and a lesson for all politicians – just because people don’t normally vote, doesn’t mean you can ignore them – one day they can really upset your applecart)

  18. Prediction: No 55/45 (I still expect fear to overcome hope when it comes to wielding the fatal pencil).

    Preference: As an Englishman, NO. I don’t want permanent Tory (or, god forfend, UKIP) government down here. Were I Scot living inwith Pictland it would be a close call, but I might well lean towards yes.

  19. What if the polls havent taken into account the extra million voters that dont usually vote….I think that the one last big hope for YES

  20. Richard,
    I think you’re bang-on with your comments about those who don’t normally vote. This, together with the very high number of registrations suggests that there could yet be a surprise.

    Matthew Egan
    “I think a lot of No voters are doing so while holding their noses, and if the UK government doesn’t deliver, they’ll feel betrayed.”

    I can guarantee that the UK government won’t deliver, because the ‘vow’ was made without reference to Parliament and will also really p— off the majority of the English.

  21. Just spotted this on the ‘Wee Ginger Dug’ site:

    “A wee note about posting comments on Thursday.

    There are strict laws about publishing during the time that polling stations are open. It’s illegal to publish any exit polls or anything which might be construed as an exit poll while the ballot is going on. So because I have things to do and can’t spend all day hovering over the comments, overnight I will suspend the publication of any comments on the blog until polling stations close at 10pm on Thursday. You can still post a comment, but all comments will be held in the moderation queue and will not be published until 10pm. After 10pm commenting will return to normal and if you’ve had a previously published comment, your comments will appear instantly.”

    I wasn’t aware that a site’s readers’ comments could do that. Anthony…your take on that?

  22. I’m not sure what to make of 89.6% postal vote turnout in Edinburgh… In 2010: postal vote turnout was 83.6% across Scotland as a whole, and total turnout in Edinburgh was 6 pts above the Scottish Average:

    http://www.edinburghnews.scotsman.com/news/politics/scottish-independence-89-6-postal-vote-turnout-1-3544716#.VBoKpnr1Bmo.twitter

    So while it sounds like a huge number, and I’m sure turnout will be substantially higher than in 2010, this number on its own probably isn’t as dramatic as it looks.

  23. Pete B: “I can guarantee that the UK government won’t deliver, because the ‘vow’ was made without reference to Parliament and will also really p— off the majority of the English.”

    I can guarantee that if the UK Govt did that, vast hordes of English would join the campaign for Scottish independence.

  24. @Spearmint: “Oh, Old Nat deserves to be further up the reeducation list than that!”

    Not whilst Brian Souter and Morrissey are still at large, he doesn’t.

    But I guess we might be able to bump Noam Chomsky a bit further down the list to free up a space.

  25. Ultimately common sense will prevail, all the flag waving and rabble rousing nonsense will be seen for what it is by the majority of Scots, a nae vote is a vote for stability, if you’ve got nothing to lose, then vote aye, but if you’re a responsible citizen with a family to support, you can’t afford to follow the ludicrously romantic road to ruin.

  26. In case anyone is interested (it probably just tells you what you already know!), I have re-run the meta-analysis using data from the six polls which have come out since Sunday:

    http://thestatsgeek.com/2014/09/17/meta-analysis-of-scottish-independence-polls-update-17th-september-2014/

  27. Number Cruncher

    I’m not sure what to make of 89.6% postal vote turnout in Edinburgh… In 2010: postal vote turnout was 83.6% across Scotland as a whole, and total turnout in Edinburgh was 6 pts above the Scottish Average

    The trouble is that if you have groups that are more likely to vote anyway (postal voter, Edinburgh voters) then it doesn’t really tell you much about the groups who usually don’t vote much. It’s similar to all these polls showing likelihood to vote in the 90s. It probably won’t be that high, but the refusniks are also likely to not answer polls or even be easy to contact.

  28. @Miserable Old Git

    Living in Wales I share your sentiments

  29. Who is Jonathan Bartlett, and how did he gain control of Statgeek’s website?

  30. @Matthew Egan

    I think you had perhaps enlighten your friends that there won’t be another chance for many years.

    My own opinion is that if you have read the White Paper (versus any promises / pledges from the No side), and the Wee Blue Book (versus the mainstream media), you will be kitted out to make a decision on now or never.

    Those over 35 it will probably be never (short of a massive swing to SNP on a single issue, such as another Iraq-esc issue).

    Whatever you decide, vote positively. :))

  31. The calm before the shtorrrrrrrrrrrrrrrrrrrrm………………………..

  32. @MuddyWaters

    Hi. I didn’t gain control of anyone’s website! :-) I’m a biostatistician who setup a blog website last year called thestatsgeek.com, and have just come across this site in the last week when reading about the Scottish indy poll. I’m not the same person as Statgeek who is posting here!

    Best wishes

  33. I still stick to looking at the most objective evidence. Despite the closeness, No have a slight lead and, unless what have become the “known unknowns” that the pollsters have been discussing, then I suspect the markets to be relieved on Friday morning.

    However, the reaction of Westminster then will be critical if the UK is to remodel itself into a workable system.

    The Herald’s No position is conditional on the UK introducing a Federal-style constitution.

    Better Nation’s additional “trust” questions in last week’s Survation poll

    http://www.betternation.org/2014/09/a-natural-majority-for-yes/

    simply confirm what we have all known for a long time – that a few additional changes to the devolution settlement – especially if they are mainly in taxation and reduce the % of Scots taxes that stay/are returned to Scotland, are unlikely to settle the constitutional relationship on these islands.

    A No vote would create an opportunity for a visionary Westminster politician (for a No vote ensures that only decisions made there matter) to model a new UK, to inspire the folk in these islands to grasp that vision, and force the backwoods MPs to accept it.

    However, looking at the current crop of UK politicians, I really can’t see any of them capable of doing that, so the whole business will rankle on.

    (Of course, I hope my prediction is wrong, and that a Yes vote will force Westminster to look to the interests of “the people” and not their parties and their careers.)

  34. Roger – Absolutely, that is why I’m saying that this number should be kept in context

  35. @ Muddy Waters,

    Well, Morrissey has to be first just as a matter of public safety- we have to capture him before he releases any more music. But surely there must be some people we can bump.

    George Monbiot? He’s harmless, no one ever listens to him anyway.

  36. Statgeek,

    I was under the impression that no exit polls are planned anyway.

  37. Oldnat
    “(Of course, I hope my prediction is wrong, and that a Yes vote will force Westminster to look to the interests of “the people” and not their parties and their careers.)”

    For once I agree with you, but I fear we will both be disappointed.

  38. Statgeek

    In those far off days when BBC Scotland allowed debate on their blogs, they were always suspended during voting hours.

  39. @ Bill and Statgeek,

    Anthony said they were too expensive to commission for anything but general elections. (Although arguably the potential breakup of the UK is much more important?)

  40. People who don’t usually vote are not the malleable marginalised that they’re being cast as. Getting them registered is easy. Getting them to vote, except by post, is like herding cats. They don’t have to do anything ‘clever’ to avoid being ‘forced’ to vote. They simply say they’ve already been & voted. Unless you are going to spy on them from the moment the polls open, you are not going to know whether they’ve voted or not.

    But let’s assume that you succeed in ‘bussing’ them to the polls. They can vote as they please or spoil their ballots. It’s just insulting to assume that they’ll simply vote the way they’re told to vote.

  41. I think the number of possible outcomes is still staggeringly vast from all this (I mean, the vote has a binary outcome, but the political after-effects certainly do not).

    I mean, take this as a not entirely impossible counterfactual:

    2014 Scotland votes YES, Salmond puts together a mostly SNP team of negotiators who start talking to a combined LabLibCon UK team about independence, date set for 2017

    2015 Cameron returned as a minority government, Tim Farron’s LDs providing supply & confidence. The Tories’ GE campaign has centred on being the party to ensure the UK is not cheated/gets the best deal from the Scots and on avoiding a currency union.

    In Scotland, it rapidly becomes apparent that Salmond was in fact bluffing about currency and an intransigent EU, and the political stock of the SNP begins to fall dramatically as a revitalised Gordon Brown declares his intention to enter Holyrood and paints Salmond as a liar who has gambled Scotland’s future away.

    2016 Holyrood election is won by Labour, with LD allies.

    What the hell happens then? Do the anti-independence parties still have to keep negotiating their independence settlement for 2017?

  42. @Jonathan

    Your probably a proper statgeek. I’m just an IT geek that likes a bit of stats. For some reason, people think that I should be all statty, and never have an opinion (like what they always do – hehe).

    I bet my charts are funkier though! :))

  43. Aw, I just wrote a decent length comment for once and it’s in moderation :(

    AW: What triggers that, so I can avoid it in future?

  44. @Bill Patrick et al

    I was referring more to mischief making from supporters of either side on comments of sites, to perhaps influence opinion…and how site owners moderate said comments. It never occurred to me before.

    What of social media? Impossible task, I feel.

  45. Spearmint

    George Monbiot isn’t on the list at all. He’s down to become a practice dummy for George Galloway’s forthcoming self-defence classes – a kind of controlled experiment to discover if the pen is mightier than the head-but (not that he’ll be allowed to take a pen with him. Far too pointy. he might have poor George G’s eye out.)

    But you’re right in principle. It might be a nice gesture to give him a bit longer to scamper about in the open fields. Tell you what: I’ll give him Brian Cox for company, to free up a priority place for ON.

  46. Amber Star,

    “Unless you are going to spy on them from the moment the polls open”

    You underestimate the size of Radical Independence’s groundgame. They have already bussed 50% of Scotland into polling booths; you’re probably next. Hear a truncheon banging at the door yet?

  47. On the ‘vow’

    1) They said that the Scottish parliament will have the final say on how much is spent on the NHS. Note – not just the NHS in Scotland.

    2) They promise tax-raising powers and also maintenance of the Barnett formula. So therefore if the Scottish Parliament reduced taxes, because of the formula Scotland would receive an even greater proportion of taxes compared to their contribution. i.e. if the English pay 20% tax, and the Scots 20%, the Scots get more back per head from the total pot because of the formula. If the English pay 20% and the Scots pay 18%, the Scots would still get a bigger share per head but will have contributed less, tehreby making even more ‘profit’.

    Have I misunderstood something or is it all a total nonsense?

  48. Spearmint

    I think it’s more that they wouldn’t know where to start with an exit poll – there’s no past voting patterns to go on to know where to site a representative sample of collection points as there is with a GE.

  49. @MATTHEW EGAN

    The SNP isn’t going to find it so easy without a Tory government at Westminster, though. No use blaming Tory MPs for reneging on the deal when they’ve already been kicked out of office (as they undoubtedly will be).

  50. MUDDY WATERS

    “…Who is Jonathan Bartlett, and how did he gain control of Statgeek’s website?…”

    TheStatsGeek and Statgeek are not the same person.

1 2 3 4