In Defence of Polling

2015 is unlikely to be remembered as a high point in opinion polling.

In the months since the election I’ve spoken at various events and appeared on various panels, and at almost every one at some point there’s been a question from the audience along the lines of “Why should I ever believe what a poll says ever again?”. It’s normally from a middle aged man, who looks very pleased with himself afterwards and folds his arms with his hands sort of tucked in his armpits. It may be the same man, stalking me. I fear he may be out in my front garden right now, waiting to pounce upon me with his question when I go to take the bins out.

Anyway, this is my answer.

Following the general election we pollsters took a lot of criticism and that’s fair enough. If you get things wrong, you get criticised. The commentariat all spent months writing about hung Parliaments and SNP-Labour deals and so on, and they did it because the polls were wrong. The one bit of criticism that I recall particularly grated with me though was a tweet (I can’t find it now, but I think it was from Michael Crick) saying that journalists should have talked to more people, rather than looking at polls.

And you know what, I thought to myself, maybe if you had talked to more people, maybe if you really ramped it up and talked to not just a handful of people in vox pops, but to thousands of people a day. Maybe then you could have been as wrong as we were, because that’s exactly what we were doing.

Polling is seen as being all about prediction – however often we echo Bob Worcester’s old maxim of a poll being a snapshot, the public and the media treat them as predictors, and we pollsters as some sort of modern-day augur. It isn’t, polling isn’t about prediction, it’s about measurement. Obviously there is a skill in interpreting what you have measured, and in measuring the right things in the first place, but ultimately the irreducible core of a poll is just asking people questions, and doing it in a controlled, quantifiable, representative way.

Polling about voting intention relies on a simple belief that the best way to find out how people are going to vote at a general election is to actually go and ask those people how they will vote at the general election. In that sense we are at one with whoever it was who wrote that tweet. You want to know how people will vote? Then talk to them.

The difference is if you want make that meaningful in any sense, you need to do it in an organised and sensible fashion. There is no point talking to 1000 people at, say, the Durham Miner’s Gala, or at Henley Regetta. There is no point only talking to people you can find within five minutes of the news studio. You are not going to get a proper picture if everyone you talk to is under 40 and white, or the sort of people walking round a shopping centre on a mid-week afternoon.

If you want to actually predict a general election based on talking to people, you’re going to have to make sure that the thousand people you talk to are properly reflective of all the people in Britain, that you’ve got the right number of people of different ages, genders, races, incomes, from every part of the country. And if you find out that despite your best efforts you have too many men and too few women, or too few young people and too few old people, you need to put it right by giving more weight to the answers from the women or young people you do have.

And that’s it. Ultimately all pollsters do is ask people questions, and try and do it in the fairest, most controlled and representative way we can. Anyone saying all polls are wrong or they’ll never believe a poll again is essentially saying it is impossible to find out how people will vote by asking them. It may be harder than you’d think, it may face challenges from declining response rates, but there’s no obviously better way of finding the information out.

No pollster worth his or salt has ever claimed that polls are infallible or perfect. Self evidently they weren’t, as they’d already got it wrong in 1970 and 1992. People can and do change their minds between polls being conducted and the election (sometimes even between the final polls and the election). Lots of people don’t know how they’ll vote yet. Some people do lie, sometimes people can’t answer a question because they don’t really know themselves, or don’t really have an opinion and are just trying to be helpful.

Pollsters know the limitations of what we do. We know a poll years out can’t predict an election. We know there are things that you really can’t find out by asking people straight (don’t get me started on “will policy X make you more likely to vote for Y?”). Our day to day job is often telling people the limitations of our product, of saying what we can’t do, what we can’t tell. If anything, the pendulum had swung too far before the election – some people did put too much faith in polls when we know they are less than perfect. I obviously hope the industry will sort out and solve the problems of this May (more on that here, for those who missed it). It’s a good thing that in 2020 journalists and pundits will caveat their analysis of the polling and the election with a “if the polls are right” and mean it, rather than just assuming they will be. For all its limitations though, opinion polling – that is, asking people what they think in a controlled and representative way – is probably the best we have, and it’s a useful thing to have.

Public opinion matters because we are a democracy, because people’s opinions drive how they vote and how they vote determines who governs us. Because it matters, it’s worth measuring.

What would it be without polling? Well, I’m not going to pretend the world would come to a shuddering halt. There are people who would like to see less polling because they think it would lead to a better press, better political reporting. The argument goes that political reporting concentrates too much on the horse race and the polling figures and not enough on policies. I don’t think a lack of polls would change that, if anything it would give more room for speculation. The recent Oldham by-election gave us an example of what our elections would be without polls: still a focus on who was going to win and what the outcome might be, except informed only by what campaign insiders were saying, what people picked up on the doorstep and what “private polls” were supposedly saying. Dare I whisper it after all the opprobrium, but perhaps if Lord Ashcroft had done one of his constituency polls early on and found UKIP were, in fact, a country mile behind Labour the reporting of the whole by-election might have been a tad better.

There was a time (up to the point that the Times commissioned YouGov to do a public poll) in the Labour leadership election when it looked like it might be the same, that journalists would be reporting the campaign solely upon what campaigns claimed their figures were showing, on constituency nomination figures and on a couple of private polls that Stephen Bush had glimpsed. While it may have been amusing if the commentariat had covered the race as if it was between Burnham and Cooper, only to find Corbyn a shock winner, I doubt it would have served democracy or the Labour party well. Knowing Corbyn could win meant he was scrutinised rather than being treated as the traditional left-wing also-ran, meant Labour members could cast their votes in the knowledge of the effect it might have, and vote tactically for or against a candidate if they wished.

And these are just election polls. To some extent there are other ways of measuring party support, like claimed canvassing returns, or models based upon local by-election results or betting markets. What about when it comes to other issues – should we legalise euthanasia? Should we bomb Syria? I might make my living measuring public opinion, but for what it’s worth I’m a grumpy old Burkean who is quite happy for politicians to ignore public opinion and employ their own good judgement. However, for anyone who thinks politicians should reflect public opinion in their actions, they need to have the tools to do so. Some people (and some politicians themselves) do think politicians should reflect what their voters want, and that means they need some half decent way of measuring it. That is, unless you’d like them to measure it by who can fill in the most prewritten “write to your MP” letters, turn out the largest number of usual suspects on a march, or manipulate the most clicks on a voodoo poll.

In 2016 we will have the results of the BPC inquiry, we’ll see what different methods the pollsters adopt to address whatever problems are identified and we’ll have at least three elections (London, Scotland and Wales) and possibly a European referendum to see if they actually work. Technically they won’t actually tell us if the polls have solved their problems or not (the polling in Scotland and Wales in May was actually fine anyway, and referendum polling presents its own unique problems), but we will be judged upon them nevertheless. We shall see how it pans out. In the meantime, despite the many difficulties there are in getting a representative sample of the British public, I still think those difficulties are surmountable, and that ultimately, it’s still worth trying to find out and quantify what the public think.


91 Responses to “In Defence of Polling”

1 2
  1. As always a very informative and sane piece – thanks.

  2. “That is, unless you’d like them to measure it by who can fill in the most prewritten “write to your MP” letters, turn out the largest number of usual suspects on a march, or manipulate the most clicks on a voodoo poll.”

    Oh dear: that eloquent logic is about to be utterly ignored (over the next 4 years) by the current Labour leader and his packed NEC !!

  3. @AnthonyJWells et al

    While I am not as sanguine as others regarding the fixability of the problem, I still hope that the polling industry will find a way to fix things, and that “what went wrong…” will be followed by a believable “…and this is what we’re dong to fix it”.

    But regardless of fixability, I still hope that you and the below-the-line commentators have the best possible 2016, and that the constant thoughtful articles supplied by UKPR continue unabated. My very best for 2016 and Happy New Year!

  4. Anthony

    Thanks for all your work in maintaining this site as an island of rationality in the sea of hyperbole which is politics.

    Have a Guid New Year [1], and I look forward to reading your account of the new YG methodologies for 2016.

    [1] Similar felicitations to all contributors to, and lurkers in the brushwood of, UKPR. These felicitations are all-inclusive with the exception of electoral success by Better Together parties in Scotland in May and EU Leavers whenever the referendum is called. :-)

  5. Great piece AW, not that I’d expect anything else. Keep it up in 2016.

  6. Well we managed to make it another year without destroying ourselves. Seems worthy of celebration to me :)

    Happy New Year all, and best wishes for 2016!

  7. First in 2016?

  8. The reported conclusions of the polling inquest, that those being polled were unrepresentative of that section of the population – particularly the young – is an inevitable result of the use of online panels. The members of those panels will inevitably be more interested/engaged in politics than those who are not.

    I just hunted back for Anthony’s summary of the differences between online and phone polls during the election campaign (http://ukpollingreport.co.uk/blog/archives/9350), and lo! the phone polls show the Tories 2-3% better off compared to online polls.

    In retrospect, I wonder if the difference was even bigger than this. I recall some discussion around then, that there was a tendency for pollsters to adjust their adjustments in order to be more in line with the majority of pollsters?

  9. “…meant Labour members could cast their votes in the knowledge of the effect it might have, and vote tactically for or against a candidate if they wished.”

    This leads me to ask what effect the (published) polls have on how people vote and what effect the polls should have on how people vote?

  10. Robin – it isn’t inevitable. Imagine, for example, using a panel that was wholly recruited from, say, football websites, associated with a football brand and mostly used for asking questions about football. It would almost certainly be skewed towards football fans… but there would be no obvious reason why it would be skewed towards the politically aware. That said, it is very hard to do in practice.

    I think the phone/online difference is a bit of a red herring. Read the post you linked to. The gap wasn’t that big, and was exaggerated by other factors that co-incided with the online/phone split (Ashcroft and ICM also reallocated don’t knows, which increased the Conservative lead by 1 point, so part of the supposed difference was nothing to do with phone/online). In the final pre-election polls there was no meaningful gap between phone and online at all in terms of the Lab/Con position.

    My own guess is that the underlying problem (polls getting people who are too engaged in politics) affected both modes – people who join panels are too political, people who answer the phone and agree to take part in phone polls are too political. It may be that one mode is affected a little more or less than the other, but they both give the wrong result and the underlying problem is the same. The solutions may, of course, be different.

    It’s also worth considering that while all telephone polls will get similar sample make up (one man’s randomly dialled number should end up contacting similar people to the next man’s randomly dialled number) the quality of internet panels may vary widely depending on the effort put into recruitment: imagine a panel recruited through nakedly political sources vs the fictional football panel I mentioned above – they would likely have very different skews.

    It’s worth considering though that there WAS a significant online/phone gap in UKIP support, and we’re seeing a similar gap in online/phone polling on the EU referendum.

    Phil’s Dad – the same. Some people will cast their vote in the light of what the national picture is, who might win nationally, or what the position is in their own constituency. We may or may not wish them too, but it’s not our decision (and besides, they would probably do much the same in the absence of polling… they’d just do it based on rumour, campaign claims, Lib Dem bar-charts, betting markets, etc, etc)

  11. OLDNAT

    I am desperate for the UK to leave the EU but it does not stop me from wishing you and all the other contributors to this site, a Happy, Healthy and Peaceful New Year.

    AW

    Another excellent piece, thank you for that, and the site which always contains posts of interest. Happy New Year to you.

  12. AW – I am your stalker. Surprise!

    Now, why should I ever trust an opinion poll again?

  13. People vote for many reasons – habit, idealism, duty, fear, belief, hope, imperfect knowledge, passion and so on.

    But the qualities needed for creating and improving opinion polls to predict voting are things like intelligence, statistical skill, reason, knowledge and creativity, not to mention sanity and sheer hard work. Thank you Anthony for providing these in great measure and making life a bit better for us all. We need rationality.

    Thank you also to all the posters on here for the fascinating comments, even when I disagree. It’s like being at university again.

    Long may the site continue.

  14. Accurate polling is going to have to include ethnicity alongside class, gender and age.

  15. The most obvious example we see of a question being put to an apparently randomly chosen member of the public is when someone like Crick steps out into a shopping centre, sticks the mike under someone’s nose and asks for their view on the latest issue.

    The reply is often just empty words with the victim quite clearly trying and failing to sound like someone who has thought it through. They nearly always either respond with the non PC geezer down the pub “It’s obvious innit” answer or they waffle away hoping that they look like a “right thinking person”.

    The result is always the same. A totally bogus picture of what people are really thinking. So, no, talking to more people is not the answer. Particularly when it’s a journo just trying to back up an opinion piece they have produced in advance.

  16. I’ve read quite a lot on “what went wrong” in 2015, and I take a slightly different view. A lot of what has been written is very sensible, and identified well how the inaccuracy came about. But there is an assumption that the new methodology will apply to all future elections, as if things have changed permanently. This election under-reported the Tory vote, and over-reported Labour. All the factors were, to a greater or lesser degree, involved with issues which are affected by the specific scenario of this election; when there was a significant fear factor present. The next election may be similar, or maybe not, but it illustrates that the conditions of the election may affect the best methodology. This means there will remain a significant risk that the polls will still get it wrong. The man with the folded arms has a point (but as ever, it’s more complicated than that).

    On a different note, I’m not sure I want polls which are 100% reliable. Politics tends to be swayed by polls too much already, in my view, and really reliable polls would just exaggerate that, at the cost of moving debate ever further from policy. I am increasingly of the view that polls should be banned altogether for the month leading up to elections.

  17. Happy New Year to all, and many thanks to Anthony, from a wet and windy Bournemouth East

  18. I think the polling industry is in danger of getting the kind of unjustified kicking that (say) Gordon Brown got over the crash of 2007 (which was actually one of the few things he got right and handled well). At the end of the day, the Scottish and Welsh polls were OK – and John Curtice’s election-night bombshell was, after all, based on polling. The pattern of voting in England was obviously not what anyone had expected, but even that wasn’t as far off as the actual election result might have made us think: an overall majority on less than 37% of the vote is a very unusual result, owing a lot to one-off factors like the SNP steamroller and the collapse of the Lib Dems.

    The cliche of number-crunchers being divorced from reality was given a bit of credibility by Peter Snow in his more excitable days (“and if this result was reproduced across the country”…), but it was never really fair. The worst extrapolators from limited data are the people who don’t realise they’re doing it – the opinionators and vox pop merchants, who extrapolate from a handful of data points (and sometimes not even that).

  19. Oxford Uni/YG have published polling Qs on climate change (fieldwork 9-10 Dec).

    https://yougov.co.uk/publicopinion/archive/

    North of England sample (Yes, I know!) shows least concern about it, or belief that it is man-made.

    I wonder if repeating the poll now would show a difference?

    In other words, how much are polls affected by recent events, as opposed to longer term beliefs?

  20. The polls were out – but the real problem for pollsters is that they are perceived to have been far more wrong than they actually were. When they’re predicting a dead heat, they only need to be a few points out, and the effects are dramatic. The skewing effect of First Past the Post takes care of that. Another factor was that people are really responding not to the inaccuracy of the polls themselves, but to the inaccuracy of the interpretation of those polls. If people had trusted the raw poll data, they would have been far closer to the near-total SNP triumph than was actually the case, for example.

    I have a question that’s been in my mind for months – if it’s been covered in comments already I’ve missed it, so apologies for that.

    Yes, the polls were somewhat off of the actual result. But immediately following the election, there was a noticable jump in the polling towards the Conservatives and away from Labour. Suddenly, on May 9th, instead of both being in the mid-to-low 30s as had been before the election, the Conservatives were on 40 and Labour on 30. The positions have recovered slightly but nowhere near to where they were pre-election. If anything the polls immediately after the election slightly flattered the Conservatives by comparison to the actual result.

    So as far as I can see, one of three things (or a combination of these) has happened:

    1. The polls weren’t really that wrong at all – they are a snapshot, not a prediction – and a huge number of people only made up their mind to vote Conservative on election day. In other words the figures really did change between the last polls on 6th May and the results on the day itself. The only way to test this would be to actually poll people on election day (which I think isn’t allowed?)

    2. Large numbers of people who were not saying before the election that they were going to vote Conservative, suddenly started saying they would after the election. Maybe a Conservative election victory helps to eliminate the ‘shy Tory’ factor? Or maybe lots of people follow the herd.

    3. The polling companies immediately started adjusting to the election result (perhaps because some of their weightings are inherently based on the most recent election result). If so I think this makes analysis a lot harder, and it would be interesting to know how similar to the pre-election polls the new ones would be if they were not adjusted for that factor.

    What do you all think?

  21. Wes – it’s the third one, for exactly the reason you say “because some of their weightings are inherently based on the most recent election result”. All the pollsters except MORI are weighting their results so people’s recalled 2015 vote matches the result of the 2015 general election.

    We know the first one isn’t the case. It’s illegal to publish polls conducted on election day while polls are open, but not illegal to carry them out: YouGov and Opinium both did, and published them later, and there was no significant shift to the Tories. The other way of testing is to go back and re-interview the same people, to see if they changed their mind. Again ICM, YouGov, Opinium, Populus have all done that and found no shift. Survation found a small one, but looking at the average across all the recontact surveys there was no significant late swing.

    Ditto for the second one, it would show up in recontact surveys (in fact it would be difficult to distinguish the two!). People’s answers before the election broadly matched their answers after the election – some people switched, but they cancelled each other out.

  22. Thanks Anthony! I didn’t realise about the polls conducted on the day, that’s very helpful. (Also renders most of my comment meaningless, but I’m glad I asked).

  23. A happy new year to all.

    @Oldnat

    The differences between the regions for who thinks climate change is man-made look different (and not significant) when the MOE related to sample size is considered.

    The world’s climate is changing as a result of human activity

    Region – % – Range with MOE

    London – 65% – 55 to 75
    Rest of South – 59 – 54 to 64
    Midlands/Wales – 62 – 55 to 69
    North – 58 – 53 to 65
    Scotland – 72 – 61 to 83

    It looks to me like all the regions are really quite similar. The Scottish sample looks most out of line, but the lower end still fits into the confidence interval of the other regions.

  24. CMJ

    Thanks for that MOE analysis.

    The comment about the North of England was really just a hook to hang my questions.

    “In other words, how much are polls affected by recent events, as opposed to longer term beliefs?”

  25. @Oldnat

    You would like think that recent events might change long term thinking, but the human race has an amazing ability to hurtle towards various brick walls, yet waits until the crash has happened or is unavoidable before acting.

    Recent opinions about the recent Paris summit makes me think that many people are expecting some scientist to come up with a magic bullet, that will reverse or nullify the effects of climate change, that does not require a major adjustment to their lifestyle.

    I hope they are right, but suspect they are wrong.

  26. @Oldnat

    “In other words, how much are polls affected by recent events, as opposed to longer term beliefs?”

    Can’t think there is any way to answer that question.

    @catmanjeff

    “You would like think that recent events might change long term thinking,”….

    Why? (see below, part of Met Office statement 2014)

    “As yet, there is no definitive answer on the possible contribution of climate change to the recent storminess, rainfall amounts and the consequent flooding. This is in part due to the highly variable nature of UK weather and climate.”

  27. @Sam

    If I was a betting man (wanting security), not taking action further action to reduce the affects of flooding because event in the last few years were just random variability would seem careless.

    I insurance my home vs fire. The chance of it burning down is very slim, but if it happened I would rather have some protection against that catastrophic outcome.

    Not taking action because we aren’t entirely sure climate change is the cause is a big risk with the lives of many people. Better dafe than sorry.

  28. Correction

    @Sam

    If I was a betting man (wanting security), not taking action further action to reduce the affects of flooding because event in the last few years were just random variability would seem careless.

    I insurance my home vs fire. The chance of it burning down is very slim, but if it happened I would rather have some protection against that catastrophic outcome.

    Not taking action because we aren’t entirely sure climate change is the cause is a big risk with the lives of many people. Better safe than sorry.

  29. Yes, I know it’s effects not affects.

  30. @Anthony – Happy New Year

    – but that is not a good wish limited to Anthony but to all who use and read and contribute to this site….

    I loved this piece – engaging and informative….polls as Anthony says are a measure – and an indication of mood- they are not predicative and they do not supplant the great art of rhetorical persuasion that still lies at the heart of politics in democracy. but that does not mean we will always measure well or choose wisely what we might measure any more than democracy guarantees we will always be swayed by a better argument or the right ideas. These failures only represent our flawed reality which is why politics remains always as much an art as it is a science. Polls also can see us as voter better informed or can be used to contrary ends – but in a world where all sorts of complex realities and interests collide polls and their interpretation remains an important – if not governing- part of our political dialogue much as our political parties remain the best means we have devised of reaching an end we collectively determine upon.

    However, as we seem to have reached a world where all those who might be actively engaged in politics are very often not registered to vote let alone bothered to vote if they are registered, properly conducted polls also represent a means of engaging a wider sense of opinion in our polity…and that makes them important and makes it more important that those conducting them want them to be as as authoritative as they can be made….all our lives are doomed to hone the imperfect representation but it is better work than polishing the false reflection …..

  31. There’s actually two pieces in the Guardian from yesterday afternoon which also address the problems which Anthony discusses. One by Tom Clark[1] entitled “Polling puzzle: how response times may explain election surprise” is basically Martin Boon Has Another Of His Meltdowns. He’s been talking to ‘neuroscientists’ and claims that those scum, the public, have been lying to him because they’ve not been answering the questions quickly enough[2]. Whereas in reality it’s probably more that they haven’t been answering the phone. At all.

    The second article by Clark and Alberto Nardelli, entitled “Why opinion pollsters failed to predict overall majority for David Cameron” summarises the current responses of various pollsters (YouGov, ICM, ComRes) in advance of the BPC/MRS Inquiry:

    http://www.britishpollingcouncil.org/details-of-opinion-poll-inquiry-announced/

    which now seems to be reporting pretty soon. Some of the possible solutions seem reasonable – weighting age groups to proportion of those who vote rather than of population for example[3]. Similarly distinguishing between older age cohorts may also be effective[4]. But there is a real danger of ‘retrofitting’, of applying corrections that would make the figures work last time without any real rationale as to whether such adjustments reflect the underlying reality (as seen in social research, electoral statistics or BES surveys). There’s also the worry that the inaccuracies in May were caused by one-off factors, particularly the ‘threat’ of the SNP[5]. Even if identified correctly, altering methods for these would skew future polling.

    If there are systematic problems, however, they will mostly lie with those who are more likely to be omitted from surveys[6]. Even the BES random, knock-on-your-door-16-times one only had a response rate of 56% and there’s no way of telling if the 44% were politically the same as those who took part. If nothing else I would imagine they would be much more likely to be non-voters. If only 5% will respond to phone polls and at best a couple of million belong to internet panels, then these may have even greater discrepancies. In a way it’s a surprise that polling works at all.

    But as Anthony remarks polling has continued to be effective both in GB (for example in the Labour leadership – technically more difficult that usual VI polling) and abroad – most recently in Spain. The second Guardian article also points out what happens when polling isn’t used:

    Meanwhile, this month’s Oldham byelection showed what an election without polls look like. Pre-election expectations of a close vote were set by reportage, cosy briefings, betting markets and gut instinct. They were all wrong.

    Not that this stopped the various would-be oracles pronouncing on why they were right all along and explaining the result away due to candidates, the weather, brown people etc.

    [1] I assume the rest of the politics team had left early to attend various NYE shindigs, leaving the numerate behind to produce articles that for once aren’t about the only approved topic of Why Corbyn Will Never Win. No doubt Wintour et al believe that, following Mandelson’s piece yesterday, the entire Labour membership will have been convinced by him to dump Corbyn immediately and no more persuasion is necessary. Doesn’t everyone follow what Peter says?

    [2] Apart from anything else, for many questions you would expect and hope for a more considered response, if only to make sure that the wording had been properly read and understood.

    [3] Even this may have problems because it assumes voting patterns remain unaltered. Obama’s vote in the US was underestimated because his campaign encouraged more voters from previously under-voting groups (the young, non-whites) to register and turn out.

    [4] There may still be a big problem though that, say, the over-70s who respond to polling may be even more unrepresentative of their contemporaries than with younger age groups. Those who will answer phones to strangers or be comfortable with computers will be atypical and weighting them up may make the bias worse rather than better.

    [5] This may explain why Scotland polling was fairly accurate (though it tended to overestimate Labour a little) and Wales and London polling not as far out as the rest of England (both had Con too low but not as dramatically).

    [6] There are inclusion problems as well – those responding who are not registered (though they may think they are) or even entitled to vote.

  32. A very good report.
    I like to compare local by-election results with opinion poll results as shown on your website since they show how people have actually voted and can give variations between the types of constituency according to the respective strengths of the parties.
    My own calculations on by-elections show very little difference between the relative position of the two main political parties from opinion polls.
    By-election results gave four major disadvantages over opinion polls.
    1) They have to be based on a sample of up to fifty results spread over a wide period and, therefore, in times of change show the position in the middle rather than at the end of the period.
    2)There has to be a degree of estimating because some calculations have to be based on the position at elections held before 2015.
    3) They give an over-optimistic result for Liberal Democrats who tend to perform better in By-elections than in General Elections.
    4) Opinion Poll samples are professionally assessed whereas by-elections take place in a random manner depending on vacancies and the readiness of councils to fill those vacancies. Because of the reluctance of certain metropolitan councils to conduct by-elections, the current sample is weighted towards the shires.
    In the case of the Oldham By-election, the national opinion polls suggested a safe Labour seat. The contrary impression possibly arose from journalists talking mainly to the residents of the outer suburbs who were of European origin. As you say a poll by Lord Ashcroft would have corrected that misimpression.
    There are almost no opinion polls appearing for the Holyrood Elections in May. There have been quite a number of by-elections where the results suggest the S N P could lose its overall majority. A few reliable opinion polls of a full size sample of the electorate would be to the advantage of all three main parties.
    Basically By-election results are, in assessing the current political situation, a good back up to opinion polls.

  33. A Happy New Year to everyone while I wait for my latest post to emerge from automod (and so therefore a very Happy New Year to Anthony to keep in with him).

    There are some rather unexpected political features in the YouGov climate change poll:

    https://d25d2506sfb94s.cloudfront.net/cumulus_uploads/document/krdxq5r9hq/YouGov-results-Oxford-University-Climate-change-and-the-environment-151211.pdf

    the figures for various forms of climate change denial are roughly similar among those who voted for the Conservatives (officially believing in climate change and taking action about it) and those who voted UKIP (officially at best sceptical about its existence). You’d expect UKIP voters to be much more hostile than Tories[1].

    Exploring attitudes however, Conservatives are much more likely to agree with “Climate change is definitely happening but there is time to work out the best actions to take, and we should not rush into anything that could harm our standard of living” (38%)[2] in contrast 29% of UKIP go with “It is not yet clear whether climate change is happening or not – scientists are divided on this issue. I don’t believe climate change is happening at all – it’s simply scare-mongering and we should ignore it.”[3]. If you wanted to be cruel you’d say that (a minority of) both groups are ignoring climate change – Tories because they are selfish, UKIP-pers because they are gullible.

    There’s also a strong age-differential between those under 40 and those above (though even the over-60s think climate change is man-made). As noted there’s also the same London + Scotland v the Rest split that we see on EU polling.

    [1] Both groups are actually most likely to agree with “The world’s climate is changing as a result of human activity” – Con 48%, UKIP 50% – than any other option.

    [2] This is actually the most popular option for Tories, just beating the 34% for “It is a serious and urgent problem and radical steps must be taken NOW to prevent terrible damage being done to the planet”, which is actually the most popular for UKIP (36%) as well as Labour (60%) and Lib Dem (64%).

    [3] These two sentences look like different options to me that have been shoved together at some stage by mistake.

  34. Roger Brickell

    “There are almost no opinion polls appearing for the Holyrood Elections in May.”

    Eight over four months Sept – Dec seems a little more than “almost no”!

    I’m not sure whether all political parties in Scotland see these full-scale polls as being to their advantage, but they certainly know of them. :-)

    https://en.wikipedia.org/wiki/Scottish_Parliament_election,_2016#Opinion_Polling

  35. Anthony,

    You commented earlier that there was no significant difference between the and phone polls prior to the election. Since Comres changed their methodology there appears to be an emerging trend in their results: the online polling is resulting in larger leads for the Conservatives. I’m not familiar with the detail of the voter turnout model they use but perhaps the same correction to online and phone polls may be inappropriate. Have you got a view?

  36. “Alarmingly for the polling industry” (Grauniad)

    http://www.theguardian.com/politics/2016/jan/01/eu-referendum-polling-results-depend-methods

    In the interests of protecting journalists from themselves, can we arrange for them to be strapped to chairs under spotlights and exposed to constant readings of Anthony’s posts?

  37. @OLDNAT

    “In other words, how much are polls affected by recent events, as opposed to longer term beliefs?”

    A happy new year to all!

    Longer term beliefs, by definition, only change slowly and therefore affect (as in causing a change) polls only rarely. Much of the attention and froth must therefore come from recent events, the majority of which have little long-term effect. Most recent events that matter therefore are those that change longer term beliefs. Those seeking to capitalise on recent events must therefore shape the narrative to effect that change.
    You’ve seen that with the Syrian refugee events – there are multiple story arcs promoted by interested parties that attempt to change long-term public opinion by “explaining” the events. I expect you’ll see that with the floods soon enough.

    @CATMANJEFF

    “Not taking action because we aren’t entirely sure climate change is the cause is a big risk with the lives of many people. Better safe than sorry”

    This is not a comment on climate change per se and the need to mitigate its effects, more on the “safe than sorry” part. Some years ago, I came to realise that while the cost of protecting against any one threat could be less than the value of an item, claiming this as a justification to spend to defend on that basis can quickly lead to the total cost of defensive measures against multiple threats exceeding the cost of the item itself! Alas, it is necessary to assess the credibility of threats and ranking them before committing to defending against them.
    By this line of reckoning, if climate change is credible enough to require mitigation, perhaps one should also put a
    bit into asteroid defence too since it is a credible, inevitable threat? We have been fortunate in that the cosmos seems to have been preoccupied with flinging rocks at the Russians in the last 100 years or so but that could change. An airburst over Western Europe could be an example of a future “recent event” that could change long term beliefs about the ranking of threats pretty promptish.

  38. @rogerbrickell

    How are you extrapolating from Scottish local by election results in multi member wards using STV to the Scottish parliamentary elections under a FPTP/AMS system?

  39. @Fewmet

    By this line of reckoning, if climate change is credible enough to require mitigation, perhaps one should also put a
    bit into asteroid defence too since it is a credible, inevitable threat?

    The evidence that climate is significant exists already. It’s the level of certainty if it is man-made that , according to some people, is not quite there. The car we as a race is driving is heading toward a brick wall we think, maybe 75% likely, maybe 90% likely. Given the nature of studying climate science involves a lot of lag and long time scales, the only intelligent response to something more likely than not to be happening and we can intervene in a economically viable manner is to act.

    If we wait too long for the evidence to be conclusive enough, we may have already hit the wall and be passing through the windscreen. Not a great idea in my book.

    The interventions to climate change – cleaner energy, energy efficiency etc – are good for the planet regardless. We are burning the finite resources of our planet too hard and too fast, and slowing this down would aid us in the long run anyway.

    Are you really suggesting the threat to the planet of an asteroid strike is the same as man made climate change?

    At some point in the future – most likely thousands and thousands of years away at least – a big asteroid will strike earth, and we will be totally fubar as a planet. Not much we can do, except build spaceships by then and leave this rock.

    Climate change is affecting the lives of people now, and in a few decades will affect the lives of millions and millions of people, in the poorest parts of the world especially.

  40. @Fewmet

    I would add if we get it wrong on Earth, we’ve got nowhere else to go.

  41. My New Year resolution is to re-engage with this site which I stopped visiting when I became politically deflated in September. (No need for 3 guesses why I reckon)
    I miss, though, being informed on a range of matters and the measured way most contributors participate.

    FWIW, I think trend is the key to poll watching so if they are all going in a particular direction that tells us more than the raw numbers.

    Best Wishes to all for 2016 and thanks to Anthony.

    BTW – I think you can be Burkian in believing that MPs should make their own minds up rather than following apparent public opinion but still want to know what that apparent opinion is.
    Knowing your view may be a minority one (perhaps within your support base rather than in the Electorate as a whole) allows you to engage with those you represent better and in certain cases when and where to try to change minds.

  42. There was a poll, possibly for a TV station, a week or so before May 7th which predicted the Liberal Democrats would lose every single seat in the South West – from Cornwall to Cheltenham.

    I immediately recalled this when the Exit Poll was announced thinking that far from being a “rouge” it could in fact have been the most realistic of the campaign.

    I am sure that somewhere this poll will have been the subject of analysis and discussion, but it would be interesting to hear Anthony’s assessment (again?) as to how this this poll was undertaken and received by the pollsters and others.

  43. “(though even the over-60s think climate change is man-made). ”

    Some of the over 70s while agreeing that some climate change is occurring, think man’s impact is very small and that climate change is basically due to cyclical factors.

  44. Nice to see you back Jim Jam.

    I think the Burkean view of political representation in a pluralist democracy like ours is the appropriate one, and I agree with your comments about its implications for interaction with “public opinion”.

    Presumably the reason you thought to mention the issue is the current debate in the Labour Party over the purpose of an MP. And I would suggest that whilst , by “public opinion” you mean all the voters in ones constituency, those advocating what you describe as “following public opinion” in the Labour Party are refering to a much narrower electorate-the Membership.
    Even more narrow is the “mandate” which Anthony so pithily describes as from those who “measure it by who can fill in the most prewritten “write to your MP” letters, turn out the largest number of usual suspects on a march, or manipulate the most clicks on a voodoo poll.”

    And it is this looming change in the Labour Party, which some of us who are old enough have watched it go through before, which move Labour MPs like Chris Bryant are to write pieces like this :-

    “Now, I’ve always thought that Edmund Burke sounded rather arrogant when he told the voters of Bristol that an MP “owes you, not his industry only, but his judgment; and he betrays, instead of serving you, if he sacrifices it to your opinion”. The voters of Bristol thought so, too. They dispensed with his services at the election. But Burke’s point is even more important in an era of social media, when every email, every tweet and every posting comes replete with the demand that an MP should do precisely as his or her constituents wish (which is always precisely as the correspondent wishes).”

    New Statesman
    7/12/2015

  45. Colin – thanks.

    Not a conscious linkage but I must have been affected by the discussions.
    Certainly, I suggested to my local MP at the time of the Syria air strikes vote that she should vote as she saw fit not in line with representations from party members or even constituent correspondence even if overwhelmingly this was of the opposite view.

    In the end that representation was split but she took time to attempt to answer the questions posed by those with an alternative view and, there was no rancour at was the best LP GC I have attended for many years.

    Corbyn has many weaknesses but encouraging genuine, respectful, debate within the party on many issues is perhaps his major strength.
    In the modern era we know the media will portray differences of opinion as division and, whilst some of this is unfair, it is incumbent on MPs and the wider membership to present their differences in non-confrontational ways.

  46. @ Colin

    It seems to me that there are some issues where the average voter can hold an opinion that the MP should take into consideration and some issues which are so complex that the average voter cannot reasonably be expected to have an opinion worth taking into account.

    Generally speaking we can all form a valid opinion on whether the death penalty / abortion/ gay marriage should be permitted or whether we should allow a local planning application to build a by-pass/ housing development. But how many of us really know enough about the EU to have a valid opinion on Brexit.

    It is these latter issues where, IMHO, we elect and pay our MPs to take the time to study the complex issues and vote accordingly.

    Sadly it is the complex issues that are often the more important and which the media (social & mainstream) are most easily able to manipulate opinion and most people tend to make up our minds on the basis of simplistic arguments or political prejudices. This is why a referendum on the EU, whilst superficially “a good thing” is actually nothing of the sort because most people will cast a highly ill-informed vote based on the frequently inaccurate “facts” put out by both sides.

    Re the Labour Party I left to go to the SDP in the 80s and the tactics of the hard left were the same then as now. The strategy was to make “the members” the supreme authority in policy matters. This was superficially democratic but in fact was deliberately pitched at the only level where the left could control things.

    The hardline activists in my CLP would drone on for hours with procedural points etc and votes were always delayed till enough of the regular members had drifted off out of boredom.

    Social media now adds a new element whereby a very small mouse can appear to roar like a lion. This time I have left already to save myself the pain!

  47. @MIDDLEENGLANDER

    I do think in hindsight far too much faith was put in Ashcroft’s experimental methodology for LD seats, as well as optimistic briefing from within the party, and not enough in the national trend which was always suggesting they’d do extremely badly.

  48. Olly, the labour party has always listened to the views of its membership. An ordinary member is encouraged to contribute.
    This is how the party was born. Not shocking really.

    My partner has attended several CLP meetings and I am glad to say it was inclusive and not mired in process.

    I might even rejoin the LP soon.

  49. @ Mark W

    I wasn’t saying that the labour party didn’t listen to its membership I am saying that the hard left seeks to use that to control the party in a very organised fashion which is a different thing entirely.

    I am sure you are correct that your partner’s CLP is not being mired in process but I would also caution that Momentum have really yet to find their feet yet. I doubt its strategy will turn out to be much different from Militant Tendancy. Lets see how the Trident debate pans out before drawing any conclusions.

  50. I think you’re a bit mistaken, OllyT. I’m a Labour moderate (hovering between Open Labour and Labour First) but the constituency-level left don’t hold nearly the power or presence they did in the 1980s.

    They don’t have the organisation or knowledge of internal party workings that existed then, so they’re either nutters who turn up and one meeting and turn everyone off them, constructive left wingers happy to work with the rest, or they don’t show up at meetings at all.

    The meetings I’ve been to so far have been reasonably constructive, pretty straightforward and not dominated by the left in any way.

    Off Twitter, they’re not nearly so bad. I fear what 1980s Twitter would have been like.

1 2