In Defence of Polling

2015 is unlikely to be remembered as a high point in opinion polling.

In the months since the election I’ve spoken at various events and appeared on various panels, and at almost every one at some point there’s been a question from the audience along the lines of “Why should I ever believe what a poll says ever again?”. It’s normally from a middle aged man, who looks very pleased with himself afterwards and folds his arms with his hands sort of tucked in his armpits. It may be the same man, stalking me. I fear he may be out in my front garden right now, waiting to pounce upon me with his question when I go to take the bins out.

Anyway, this is my answer.

Following the general election we pollsters took a lot of criticism and that’s fair enough. If you get things wrong, you get criticised. The commentariat all spent months writing about hung Parliaments and SNP-Labour deals and so on, and they did it because the polls were wrong. The one bit of criticism that I recall particularly grated with me though was a tweet (I can’t find it now, but I think it was from Michael Crick) saying that journalists should have talked to more people, rather than looking at polls.

And you know what, I thought to myself, maybe if you had talked to more people, maybe if you really ramped it up and talked to not just a handful of people in vox pops, but to thousands of people a day. Maybe then you could have been as wrong as we were, because that’s exactly what we were doing.

Polling is seen as being all about prediction – however often we echo Bob Worcester’s old maxim of a poll being a snapshot, the public and the media treat them as predictors, and we pollsters as some sort of modern-day augur. It isn’t, polling isn’t about prediction, it’s about measurement. Obviously there is a skill in interpreting what you have measured, and in measuring the right things in the first place, but ultimately the irreducible core of a poll is just asking people questions, and doing it in a controlled, quantifiable, representative way.

Polling about voting intention relies on a simple belief that the best way to find out how people are going to vote at a general election is to actually go and ask those people how they will vote at the general election. In that sense we are at one with whoever it was who wrote that tweet. You want to know how people will vote? Then talk to them.

The difference is if you want make that meaningful in any sense, you need to do it in an organised and sensible fashion. There is no point talking to 1000 people at, say, the Durham Miner’s Gala, or at Henley Regetta. There is no point only talking to people you can find within five minutes of the news studio. You are not going to get a proper picture if everyone you talk to is under 40 and white, or the sort of people walking round a shopping centre on a mid-week afternoon.

If you want to actually predict a general election based on talking to people, you’re going to have to make sure that the thousand people you talk to are properly reflective of all the people in Britain, that you’ve got the right number of people of different ages, genders, races, incomes, from every part of the country. And if you find out that despite your best efforts you have too many men and too few women, or too few young people and too few old people, you need to put it right by giving more weight to the answers from the women or young people you do have.

And that’s it. Ultimately all pollsters do is ask people questions, and try and do it in the fairest, most controlled and representative way we can. Anyone saying all polls are wrong or they’ll never believe a poll again is essentially saying it is impossible to find out how people will vote by asking them. It may be harder than you’d think, it may face challenges from declining response rates, but there’s no obviously better way of finding the information out.

No pollster worth his or salt has ever claimed that polls are infallible or perfect. Self evidently they weren’t, as they’d already got it wrong in 1970 and 1992. People can and do change their minds between polls being conducted and the election (sometimes even between the final polls and the election). Lots of people don’t know how they’ll vote yet. Some people do lie, sometimes people can’t answer a question because they don’t really know themselves, or don’t really have an opinion and are just trying to be helpful.

Pollsters know the limitations of what we do. We know a poll years out can’t predict an election. We know there are things that you really can’t find out by asking people straight (don’t get me started on “will policy X make you more likely to vote for Y?”). Our day to day job is often telling people the limitations of our product, of saying what we can’t do, what we can’t tell. If anything, the pendulum had swung too far before the election – some people did put too much faith in polls when we know they are less than perfect. I obviously hope the industry will sort out and solve the problems of this May (more on that here, for those who missed it). It’s a good thing that in 2020 journalists and pundits will caveat their analysis of the polling and the election with a “if the polls are right” and mean it, rather than just assuming they will be. For all its limitations though, opinion polling – that is, asking people what they think in a controlled and representative way – is probably the best we have, and it’s a useful thing to have.

Public opinion matters because we are a democracy, because people’s opinions drive how they vote and how they vote determines who governs us. Because it matters, it’s worth measuring.

What would it be without polling? Well, I’m not going to pretend the world would come to a shuddering halt. There are people who would like to see less polling because they think it would lead to a better press, better political reporting. The argument goes that political reporting concentrates too much on the horse race and the polling figures and not enough on policies. I don’t think a lack of polls would change that, if anything it would give more room for speculation. The recent Oldham by-election gave us an example of what our elections would be without polls: still a focus on who was going to win and what the outcome might be, except informed only by what campaign insiders were saying, what people picked up on the doorstep and what “private polls” were supposedly saying. Dare I whisper it after all the opprobrium, but perhaps if Lord Ashcroft had done one of his constituency polls early on and found UKIP were, in fact, a country mile behind Labour the reporting of the whole by-election might have been a tad better.

There was a time (up to the point that the Times commissioned YouGov to do a public poll) in the Labour leadership election when it looked like it might be the same, that journalists would be reporting the campaign solely upon what campaigns claimed their figures were showing, on constituency nomination figures and on a couple of private polls that Stephen Bush had glimpsed. While it may have been amusing if the commentariat had covered the race as if it was between Burnham and Cooper, only to find Corbyn a shock winner, I doubt it would have served democracy or the Labour party well. Knowing Corbyn could win meant he was scrutinised rather than being treated as the traditional left-wing also-ran, meant Labour members could cast their votes in the knowledge of the effect it might have, and vote tactically for or against a candidate if they wished.

And these are just election polls. To some extent there are other ways of measuring party support, like claimed canvassing returns, or models based upon local by-election results or betting markets. What about when it comes to other issues – should we legalise euthanasia? Should we bomb Syria? I might make my living measuring public opinion, but for what it’s worth I’m a grumpy old Burkean who is quite happy for politicians to ignore public opinion and employ their own good judgement. However, for anyone who thinks politicians should reflect public opinion in their actions, they need to have the tools to do so. Some people (and some politicians themselves) do think politicians should reflect what their voters want, and that means they need some half decent way of measuring it. That is, unless you’d like them to measure it by who can fill in the most prewritten “write to your MP” letters, turn out the largest number of usual suspects on a march, or manipulate the most clicks on a voodoo poll.

In 2016 we will have the results of the BPC inquiry, we’ll see what different methods the pollsters adopt to address whatever problems are identified and we’ll have at least three elections (London, Scotland and Wales) and possibly a European referendum to see if they actually work. Technically they won’t actually tell us if the polls have solved their problems or not (the polling in Scotland and Wales in May was actually fine anyway, and referendum polling presents its own unique problems), but we will be judged upon them nevertheless. We shall see how it pans out. In the meantime, despite the many difficulties there are in getting a representative sample of the British public, I still think those difficulties are surmountable, and that ultimately, it’s still worth trying to find out and quantify what the public think.


91 Responses to “In Defence of Polling”

1 2
  1. @OllyT

    You make some interesting points about MPs, but there is a large ‘however’ in my view.

    It is these latter issues where, IMHO, we elect and pay our MPs to take the time to study the complex issues and vote accordingly.

    This forgets several key issues.

    MPs largely vote according to the wishes of party whips, not the pros and cons of the matter at hand. Votes are often partisan, with the key aim to embarrass opponents, or try to position them in certain way.

    Also, MPs are subject to private interests and lobbying on behalf of pet causes that can bend the way they vote.

    MPs also vote to support their manifesto (mostly), and at times all Governments have manifesto items that don’t stack up. Voting on the issue at hand would mean not voting for this policy, but MPs support it as it is a manifesto commitment.

  2. JIM JAM

    We will find out in due course whether ” genuine, respectful, debate within the party ” is the objective-or whether the “debate” is restricted to Momentum & the Membership-and MPs are merely their mouthpiece in Parliament.

    OLLYT

    I agree that the more complex issues are those where MPs particularly need to explain , educate & use their judgement .

    I’m not sure one should exclude issues like capital punishment as being “simple issues” though. This contains layers which touch on religious belief, social conscience & even the vexed business of morality. If ever there was an issue where personal judgement is called for-that is surely one ?

    Your experience of Militant Tendency in the 80s beats mine-I was merely an interested observer :-)
    For a man with a reputation for “straight speaking” it seems to me that Corbyn can obfuscate with the best of them when it comes down to the nitty gritty. At present he seems happy to mouth platitudinous rules about the conduct of his “new politics” in his party, whilst watching his “supporters” breach every one, and allowing Abbott to reprimand any MP with concerns.

    Time will tell what Corbyn is actually intent on doing within the Labour Party-and at some point he will actually have to take responsibility for it.

  3. I don’t recognise Colin’s above.
    I think we should contrast labours democratic processes with the Tories, like er they don’t have anything much.

  4. Anyone notice jeremy Corbyn get a mention in EastEnders?
    It’s true, in a recent Xmas episode martin was recounting a dream to Stacey, ” there was this bloke called corbyn banging on about trident”, he said.

    I lolled then frowned. EastEnders never mentions the real world .
    Maybe a cameo next?

  5. Happy New Year to all.

    @ OldNat

    The current Left in Liverpool and Leeds (the two places that I know well enough to form an opinion) is very different from the MT (still, that Labour is anywhere in Liverpool should be credited (partly) to the MT (crossing Scotland Road). Most Tendency members I know haven’t even rejoined Labour in Liverpool.

    And Militant wasn’t that well organised – they just captured the mood in parts of the country and could translate this to appropriate slogans and in some cases policies. As an interested outsider I could see things pretty closely). However, basing policies largely on the immediate needs of the unskilled and semi-skilled working class has so far been self defeating (in cities and countries), so it’s unlikely to work in the future.

    As to polling.

    There seem to be generic problems (bridging the self-selecting poll aka elections and the demographically restructured polls) and specific to the May elections (distortions due to LD and UKIP sometime 2011-12).

  6. As to the discussion party democracy, grass root opinion, policy making and the Labour Party above.

    I couldn’t help remembering a certain V.I.L. who, towards the end of his life could only write short notes. Of course, there was no way he would propose grassroot policy making, although he adopted the SR programme on the land which gave the Bolsheviks the majority in the night of 7th of November, and of course the Soviet system till 1918 was largely grassroot), but he wrote about the need of men (and women) who live amongst the people and without mistakes recognise their opinions, feelings, moods, desires. Obviously, focus groups had not yet been invented then.

  7. @Catmanjeff,

    Don’t disagree with much of what you have added – my main criticism was with the hard left agitating for grassroots policy making and dressing it up as the “last thing” in democracy. It has never been anything of the sort IMHO.

  8. Fewmet – “By this line of reckoning, if climate change is credible enough to require mitigation, perhaps one should also put a bit into asteroid defence too since it is a credible, inevitable threat?”

    Actually asteroid defence is already in place – it’s called the Moon, which protects us by pulling stray debris towards it (asteroids enter it’s gravitational space before they encounter the earth). That’s the reason why the Moon is so much more pock-marked than the Earth. Of course the rare one travelling very fast will get through – but the probability it will land on Britain is small – they end up in the oceans or on Russia.

    Regarding floods – we’ve been so preoccupied with our own, we haven’t noticed the flooding happening elsewhere – it’s a global event, i,e down to climate change and not the EU or grouse farming. Parts of Paraguay are underwater. The Missouri flooded last week, and now the Mississippi is cresting at record levels. The Mississippi rarely has winter floods – it tends to flood in spring thanks to snow melt.

    See

    http://www.reuters.com/article/us-usa-weather-idUSKBN0UF1R620160102

    This is happening despite them increasing flood plains in a system of “floodplain buyouts”, where the local councils are buying houses in flood prone areas where the US Corps of Engineers have reported it’s not possible to defend those properties. The buyout houses are then destroyed to increase the flood plain.

    It’s something we should think about looking at here. Here’s the buyout page of the city of Austin, Texas:

    https://www.austintexas.gov/onioncreek

    I’m sure some of our councils could create programs based on this model. (To pay for it, hold a local referendum to get permission for a flood levy on top of council tax, and then use the money raised to buy out properties to increase the flood plains).

  9. Lazlo

    I think you might have been replying to OllyT and not myself. :-)

    My only experience of the “hard left” was watching them enter (and destroy – through incompetence, not intent) Sillar’s Scottish Labour Party.

    There wasn’t much sign of Militant in the bits of LiS I was in – any “entryism” seemed to be folk like me bolstering the devolutionist wing of LiS. We were far more successful!

  10. Laszlo

    If my comment ever appears – apologies for misspelling your name, yet again!

  11. Lazlo

    Ah! Spotted the moderation [1] trigger in my response to Laszlo!

    I think you might have been reply-ing to OllyT and not myself. :-)

    My only experience of the “hard left” was watching them enter (and destroy – through incompetence, not intent) Sillar’s Scottish Labour Party.

    There wasn’t much sign of Militant in the bits of LiS I was in – any “entryism” seemed to be folk like me bolstering the devolutionist wing of LiS. We were far more successful!

    [1] It’s an interesting word, “moderate”. In both website and Labour Party usages, it often seems to be used to describe stances which are both extreme and unwilling to respond to reality. :-)

  12. Lazlo

    Ah! Spotted the moderation [1] trigger in my response to Laszlo!

    I think you might have been responding to OllyT and not myself. :-)

    My only experience of the “hard left” was watching them enter (and destroy – through incompetence, not intent) Sillar’s Scottish Labour Party.

    There wasn’t much sign of Militant in the bits of LiS I was in – any “entryism” seemed to be folk like me bolstering the devolutionist wing of LiS. We were far more successful!

    [1] It’s an interesting word, “moderate”. In both website and Labour Party usages, it often seems to be used to describe stances which are both extreme and unwilling to respond to reality. Perfectly acceptable words are demonised by moderating. :-)

  13. @ OldNat

    No worries. All my British (English and Scottish) relatives misspell my name.

    But I would be interested in your view (as always).

  14. “It’s something we should think about looking at here.”

    “Fat chance” would be my initial response. In the last 5 years, the number of new houses built in flood risk areas has increased faster than those in safe zones, and 40% of coastal local authorities don’t even have a coastal management plan.

    On top of this, it has now emerged that the government received a report in November detailing that the cuts to maintenance budgets had doubled the effective area of flood risk, and will cost taxpayers more, as flood protection assets will wear out more quickly, increasing taxpayer spending overall.

  15. Good Afternoon all from a wet, windy and sandy beach in Bournemouth East.

    I think that enthusiastic Labour members may forget that they are not always representative of the wider electorat ne.

    Jeremy may have the 18th Brumaire syndrome, which will have, imo, weak electoral impact in 2020 and 2025

  16. @ CHRISLANE45

    “I think that enthusiastic Labour members may forget that they are not always representative of the wider electorat ne. ”

    That could equally apply to any UK political party

    One of the major failings of FPTP is that it encourages this view.

  17. An early 2016 setback for David Cameron. Lost to Ted Hankey in straight sets.

  18. For the 2016 elections (as for others), exposure to arguments of the various parties will have a major effect on polling.

    Hence, the categorisation into “major” and “minor” parties can be very important – especially in non-FPTP systems.

    Perhaps the BBC should read Anthony’s article – as in their October 2015 “Electoral Landscape” briefing paper for the Trust, they are very dismissive of polling evidence – which is particularly obtuse of them, given that the London, Welsh & Scottish polling was pretty accurate.

    http://www.bbc.co.uk/bbctrust/have_your_say/election_guidelines_2016

    Current Electoral Support
    Opinion Polls
    38. Questions remain for the polling industry, following the 2015 general election result, when the strength of the Conservatives relative to Labour was, to say the least, underestimated. It’s worth remembering, however, that over a long time period, the polls did give indications regarding other parties which turned out to be more reliable: the strength of UKIP in England and Wales; the surge in support for the SNP after the referendum in Scotland; the longterm decline of the Liberal Democrats across Britain; and the relatively modest improvement of the Greens in England.

    39. In assessing relative levels of electoral support, running up to May 2016, there is a substantial amount of evidence from real votes in real elections and, for the time being at least, comparatively little useful additional information from the limited number of voting intention polls.

  19. I’m a long time lurker who has only ever commented at GE time before, so hello everyone.

    I thought I would pipe up over AW’s comment regarding Michael Crick’s “saying that journalists should have talked to more people, rather than looking at polls. Whilst the language is clumsy, perhaps Mr Crick has a point.

    For a start, it was in the heat of feeling a little embarrassed – along with all the commentariat – of being wrong on the final result. I can understand him wanting to pass the blame. What I think Crick was saying, however, is that dichotomous questioning (which party will you vote for?) may not necessarily work these days with the efficacy of the past and we need other, complimentary, ways to triangulate the findings of polls. I don’t find this overly controversial. And yes, I know supplementary questions are asked.

    With so much flux and regional variation, perhaps fewer but far far more in-depth sampled qualitative polling is needed (annual, perhaps) to understand real thoughts and fears of the electorate. I contend that current methods are totally unresponsive to a respondent who says, for example,: “I think Ed Milliband would be dreadful as PM” and lists several reasons why. Then is asked who they’ll support… “Labour”. Whilst I am not involved in polling, I cannot imagine this was an uncommon answer.

    Whilst I get the ‘man in the pub’ – and we obsessives – are unrepresentative of the population as a whole and I loath vox-pops on TV, I think we run too far away from scientific analysis of people’s wider qualitative thoughts for reasons of cost, manpower and partial intellectual snobbery against anti-positivism. The problem is: people are not homoeconomicus.

  20. CBX1985

    What I think Crick was saying, however, is that dichotomous questioning (which party will you vote for?) may not necessarily work these days with the efficacy of the past

    Thanks for commenting and welcome and you raise some interesting points. But, in the end, how else are you going to find out how people are going to vote, except by asking them how they are going to vote? There seems to be an idea that you can somehow extrapolate how people will vote from responses to other questions with added mystical knowledge, but assuming that people are lying to the pollsters (and possibly themselves) about their intentions doesn’t really make sense.

    Firstly if they are giving the ‘wrong’ answers about voting they are quite likely to be doing so about other questions as well[1]. Secondly the polls seem to have been accurate in assessing the people they actually asked. As Anthony points out re-surveying those who took part gave the same result. These people seem unlikely to have mislead the pollsters twice and different firms found the same thing. The real difficulty seems to be that the wrong mix of people were being asked.

    So there’s nothing to suggest that those currently being polled would respond any differently to a more complex approach – the solution lies in greater inclusion. Now it’s possible that those absent or under-represented people might respond better to indirect questioning. They could be more changeable, more easily swayed by emotional factors, more scared into voting certain ways by press campaigns. The lack of a coherent political view might make them both more likely to be swing voters and less likely to respond to polls. But getting them to take part is the initial trick.

    [1] This was the problem I always had with Number Cruncher’s theory based on the discrepancy between VI and other variables such as leadership and economic competence ratings compared to historic data. If both are based on the same sample, any fault in one will mean the others are tainted by the same factors. The reasons for such differences in the last five years need to be looked at, but the non-VI responses may not be any better at predicting voting behaviour. Even if you get the right result, it doesn’t mean you got it for the right reason.

  21. CBX1985

    Hi. Welcome back.

    “far more in-depth sampled qualitative polling is needed”

    Since, as you note, this is expensive, I doubt that it will replace (or even much supplement) existing polling.

    Presumably this is what focus groups are designed to do, and New Labour were reputed to rely on them – as Better Together also did.

    This use by campaigning groups, however, only involved pre-determined target groups of the population and were no substitute for the whole population.

    Ashcroft is the only pollster that I’ve seen publish summaries of the chat in focus groups, and I don’t remember these necessarily being constructed to be properly representative of the constituency – how could they be?

  22. I think there are many lessons to be learned from the polling for the 2015 election but perhaps the most obvious one, which from the pollsters point of view is also the least palatable, is that there is no reliable way of obtaining a representative sample.

    A clue could possibly be found in the results for UKIP. Some polls had them much too low, some consistently much too high. This was perhaps because UKIP supporters were simultaneously somewhat evangelical and thus keen to respond when asked but less likely to be asked by some pollsters, due to their sampling methods.

    There would also seem to have been a strong “anyone but UKIP” vote which skewed the final vote somewhat, probably towards the Conservatives who had lost a great deal of their bogey man status while in government. We’re this not so, UKIP would probably have won a number of seats, mainly at the expense of the Conservatives.

    We now have a party leader who has extremely enthusiastic, almost evangelical, supporters who are also very internet aware. Prepare for some strange polls, again dependant on sampling methods.

  23. RMJ1

    “We now have a party leader who has extremely enthusiastic, almost evangelical, supporters who are also very internet aware.”

    Surely being “very internet aware” is hardly unusual these days? Just look at the move to online shopping.

    If you mean that more people discuss politics on social media now, then that seems to be a good thing – just as people used to pack out political meetings at election times.

    I don’t see that different methodologies showing different levels of support is necessarily a problem – as long as pollsters are consistent. It’s only when journalists try to compare sausages with doughnuts that interpretation problems arise. ;-)

  24. I am not in anyway advocating replacing polling as is, and direct questions do work well for what they are (and I enjoy them!). I mealy think that the industry can be a little unimaginative – or, more likely, solely using methods which easily sells to their clients at affordable rates, for which I do not blame them – in reading the wider voter landscape.

    My point, re Crick, was that conventional polling should only be one, albeit a dominant, facet of how politics and public mood is reported. AW saw his point as an insult to his profession; I would see this as an opportunity to provide greater supplementary data which, if collated and analysed relatively impartially, would be remarkably valuable to all concerned.

    However, I appreciate the difficulty in creating other more complex systems. But I do not buy into a defeatist ‘it can’t be done’ either. I am not suggesting a panacea, just a check and balance of wider view points and how they relate to VI. This would be difficult to mathematically quantify, but not impossible; it would, however, scientifically demonstrate key themes emerging which could show flaws in the quantitative polling in real time, not just after a GE in an industry-wide panic.

    My example of the Labour voter who thought EM would be a disaster is a case in point. The visceral nature of such views would alert a competent social scientist that their intentions are highly contradictory and make them think. If repeated widely, or within a region, this would allow one to add a strong caveat into the data.

    As for a method, I do not claim to have the answer. However, I’m bored. So, for example and thinking out loud, why not break constituencies down purely to 5 regions and the nations. The cost would be high enough, but annually should be achievable. Then, from there, carry out in each a fully representative sample picked as for a conventional poll, but using 1 hour long in-depth interview techniques on a respondents rather than set yes no questions (hence the infrequency). In essence: let them talk, not the interviewer Follow this up with a normal poll at the end. The poll would be analysed normally; analyse other figures using social research methods, whilst a clever statistician could also attach a stock-market like value to the entire poll package.

    My experience of leading focus groups (not political ones) is that you get one person who swings the views of lots of others simply by speaking early. I found the whole process highly demoralising. Daniel Kainerman similarly wrote about this somewhere, but I can’t find it. I find them a pointless compromise at qualitative data. But, again, I can see why one would use it as a shortcut.

  25. CBX1985

    I take your point about the effect of the “early interventionist” in focus groups.

    One of my concerns (and it’s very easy to pick holes, while very difficult to come up with positive alternatives!) about individual interviews is the problem I’ve come across in these is managing the coding team to code the responses in the same way!

    It might be cheaper to educate the journalists out of their statistical incompetence. :-)

  26. @RM
    ” how else are you going to find out how people are going to vote, except by asking them how they are going to vote?”

    If you ask me who I will vote for, I shall tell you it is a secret ballot. So would my wife, who has no interest in reading about or taking part in polls on politics. Others might well dissemble in other ways. It may be that the presumed excess of Labour supporters in telephone polls is due to fewer of them having regard for such ‘constitutional niceties’ and so not numbering themselves among ‘don’t know’ or won’t say’.

    If you could follow my conversations, or even note the above, or other of my comments on this and other blogs, you might gain some idea of who I am likely to vote for – and perhaps a better idea of who I am very unlikely to support. For other posters the position is I submit even clearer, despite posting rules.

    How about “on a scale of 0 to 5, (0 =never) would you be prepared to vote for Party X” – then according to the results weight support for party X in other polls.
    I suspect the result would simply be to increase the size of the error bars, which is what one might do anyway, adding to the sampling uncertainties the uncertainty in voters sticking to what they have told you.

  27. Not sure that less building all round wouldn’t be a good idea. 40% of greenhouse gases in the UK come from offices so a ban on new office building especially in London is the logical step if you believe in global warming.

  28. Have people seen the following article?

    http://mattsmithetc.com/works/shape-of-the-polls/

    He plots leadership approval against voting intention – and there is a “winning zone” in the top right corner where all parties that win sit.

  29. 40% of co2 emmisions are from industry and office premises combined. Roughly half, 20%, is from private and public sector office premises.

  30. I agree with those who say that the pollsters didn’t get it that wrong. Given how wildly unrepresentative their initial polls must be I am endlessly surprised that they get it as right as they do

    Given the importance they attach to weighting it seems to me that the problem is not simply that the original sample was unrepresentative – this is bound to be the case – but that the model underlying the weighting is wrong For example, it might be the case that Labour voters on panels were more committed and likely to stick with Labour. By contrast Labour voters in the country might be less likely to stay with Labour. If this was true of Labour but not of the Conservatives it would be difficult to weight appropriately by past voting behaviour.

    Personally I would like to balance the polls with a) the occasional random sample and b) some qualitative material In contrast to my wife. myself and almost all the polls my father in law correctly predicted the result of the 1970 simply by talking to people in his Devon village. I think that ToH covered himself with glory with his prediction this time, partly because of his intuition and sense of how people work, So a bit more investigation of the springs of action might not go amiss.

    Finally I agree with Mr Nameless. I went to my first Labour meeting expecting either nobody to be there or alternatively for the place to be full of the finger jabbing representative of the new kinder politics, In both respects I was pleasantly surprised

    .

  31. I agree with those who say that the pollsters didn’t get it that wrong. Given how wildly unrepresentative their initial polls must be I am endlessly surprised that they get it as right as they do

    Given the importance they attach to weighting it seems to me that the problem is not simply that the original sample was unrepresentative – this is bound to be the case – but that the model underlying the weighting is wrong For example, it might be the case that Labour voters on panels were more committed and likely to stick with Labour. By contrast Labour voters in the country might be less likely to stay with Labour. If this was true of Labour but not of the Conservatives it would be difficult to weight appropriately by past voting behaviour.

    Personally I would like to balance the polls with a) the occasional random sample and b) some qualitative material In contrast to my wife. myself and almost all the polls my father in law correctly predicted the result of the 1970 simply by talking to people in his Devon village. I think that ToH covered himself with glory with his prediction this time, partly because of his intuition and sense of how people work, So a bit more investigation of the springs of action might not go amiss.

    Finally I agree with Mr Nameless. I went to my first Labour meeting expecting either nobody to be there or alternatively for the place to be full of the finger jabbing representative of the new kinder politics, In both respects I was pleasantly surprised

  32. Via Britain Elects
    Westminster voting intention:
    CON: 39% (-2)
    LAB: 29% (-1)
    UKIP: 17% (+1)
    LDEM: 6% (-)
    GRN: 3% (-)
    (via YouGov / 17 – 18 Dec)

    Tables via https://yougov.co.uk/publicopinion/archive/

  33. It’s increasingly hard to see why the LDs continue to be given the status of “major” party by the broadcasters.

    For Scotland 2016, BBC have included LDs as “major” : created a new(?) status for the Scottish Greens among the minor parties [1], and inexplicably promoted UKIP to the same position as the Greens!

    [1] The only other party currently represented in the Scottish Parliament is the Scottish Green Party. The audience will expect us to report their campaign more substantively than those of other smaller parties who are not represented.

  34. Oldnat –

    The BBC guidelines at the general election had the same sort of neither major-nor-minor categorisation for the Greens:

    http://downloads.bbc.co.uk/guidelines/editorialguidelines/pdfs/2015_Election_Guidelines.pdf

    In the larger party category were Con, Lab, LD, SNP, PC and UKIP

    Greens were singled out in the smaller parties and to be given “proportionate levels of coverage in output to which the larger parties contribute, and, on occasion, similar levels of coverage, if appropriate.” All the other minor parties were to get the standard BBC bare minimum.

  35. Having participated in my first ever opinion poll last Autumn, the experience did little to increase my confidence in their reliability . It was a telephone poll conducted by Populus, the results of which, as far as I know, were never published, but it was a laborious and time consuming process conducted by a clearly exasperated interviewer. The questions were rushed, sometimes inaudible, and the topics ranged from gambling to Premiership football. The political questions came towards the end of a 20 minute conversation, by which time both the interviewer and myself were losing the will to live. The interviewer also divulged to me how difficult it was to find respondents in the 18-25 age group who were prepared to participate. He told me that people in my age group, aged 55-65, were always the more likely to answer the phone and respond to the questions. It all felt desperately unscientific and rickety to me, but I guess no poll is ever published unless it conforms to strict statistical standards on sample viability. He says hopefully! :-)

    This may be an idle and wistful thought, but I wonder if the mood and temper of the 2015 election led to some of the polling inaccuracy? I found it an extraordinarily dismal and heavy hearted-campaign, devoid of excitement and optimism; fertile ground for resentful and tight-lipped voters just wanting the whole ghastly exercise to go away so that they could get on with their lives again. If my mood thermometer was reliable last May, then I could see a lot of voters playing fast and loose with pollsters. As an election, it was one of the most depressing political experiences of my lifetime.

    Voting these days seems to be more about expressing an aversion to somebody or something rather than an opportunity to build that shining citadel on a hill. In these doleful circumstances, is it any wonder that pollsters find it desperately difficult to measure how people intend to vote? Hoodwinking pollsters may just be another manifestation of the “up yours to all politicians” sentiment so rampantly pervasive in our stuttering democracy these days. Politics is a painful once in every five years dip of the toe in the water for many now.

    A cynical world, indeed.

  36. CB11

    A very dismal post, if I may say.

    We have to believe better than that, surely.

  37. The interviewer also divulged to me how difficult it was to find respondents in the 18-25 age group who were prepared to participate. He told me that people in my age group, aged 55-65, were always the more likely to answer the phone and respond to the questions.

    I am in the 40-54 section, and treat any person calling me (except people I personally know) as a hostile force to be repelled at all costs. I have a phone so I can call out, and not for other people to call in!

    Getting people to answer the phone must be getting tougher.

  38. Anthony

    Thanks for the clarification re the Greens and the BBC’s previous practice. They do love unsubstantiated assertion, don’t they?

    On what basis do YG and other pollsters continue to give LDs headline status?

    I know this is only one poll, but you found the same number of people with LD and SNP VI in this one – and suggesting that the SNP were a “major” party across GB would be an odd idea (other than in number of MPs).

    Do you have guidelines on this? Do you follow BBC and/or Ofcom rules? or is it made up as you go along?

  39. Very interesting article Anthony, I do wonder though, that in an era when hundreds of thousands of people are prepared to declare their religion as Jedi in a National Census, how much reliance can ever be placed on answers to pollsters?

    There was a time when most people could be relied upon to give honest answers to authority figures such as census officers or pollsters, but that sense of responsibility seems to be in decline.

    Perhaps the time has come to find alternative ways of checking pollster reports, such as sophisticated analysis of social media (obviously allowing for the fact that different demographics will be more or less likely to appear in such areas).

  40. @Pete B

    I’m sure AW will be delighted to be considered an authority figure ;-)

  41. @Colin

    “I’m not sure one should exclude issues like capital punishment as being “simple issues” though. This contains layers which touch on religious belief, social conscience & even the vexed business of morality. If ever there was an issue where personal judgement is called for-that is surely one ?”

    ——–

    Well, the simple, no-brainier rebuttal to the capital pumishment idea is the problem of miscarriages of justice.

    At least if someone is sent to prison, if it comes to light later they were innocent you can release them.

    There is rather less succour available for those who instead have been more permanently demised.

    Since there are many, many ways a wrong verdict may be returned – faulty evidence, poor legal representation, nobbled juries, wrong police procedures, complex and confusing cases, mistaken expert analyses, mistaken witnesses! critical evidence (or new forensic techniques) that only comes to light later etc. etc. – you cannot 100% discount miscarriages of justice.

    Unless you have miraculously discovered some new way to 100% obviate all these sources of error in verdicts, the irreversible nature of capital punishment is therefore rather problematic, and usually puts an end to the debate when one points it out.

    Happy New Year all.

1 2