ComRes have an interesting post over on their site about differences between online and telephone polling so far this year (as well as making some extremely sensible points about the polls not being all over the place). As they correctly say, telephone polls this year have been showing a tiny Conservative lead, online polls a tiny Labour one. It’s only a small difference, but it’s there and it is not new – at the start of the year I produced a chart showing house differences between the different polling companies over 2014, and even then an online vs telephone tendency was observable: the two most “Toryish” polls were Ipsos MORI and ICM, both done by telephone. The most “Laboury” polls were TNS and Opinium, both done online.

Look a little closer though, and things are not quite that cut and dried. There are many causes of variation between polls, telephone or online fieldwork is just one of them. There is variation between different online companies and between different phone companies. Last year ComRes’s telephone polls actually produced some of the more Laboury figures, the online Populus polls tended be on the Tory side of average. Below is the average for each company so far this year (given the polls have been pretty static in 2015 I haven’t worried too much about timings of different companies polls, it’s just a straight average).

phoneonlineleads

So all three companies who have been showing a Tory lead are done by phone, all the online polls have been showing an average Labour lead. But note the variation – MORI use the telephone, but they are showing a Labour lead on average. Two online polls (YouGov and Opinium) show barely any Labour lead at all, Survation, TNS and Panelbase average around a 2 point Labour lead. This is because there are plenty of other reasons for variation between pollsters too, different approaches to weighting, turnout, don’t knows and so on – I summarised lots of them here. Just looking at one can sometimes be misleading, for example, ICM and Ashcroft also reallocate don’t knows by past vote, which normally bumps up the Tory position by a point or so, so that will also be a major part of the difference between them and companies showing worse results for the Conservatives (one should also bear in mind that the monthly polling companies have only produced 3 or 4 polls this year – so a single odd poll like ICM’s this month has a large impact on the average).

I’ve no doubt that telephone vs online is one of the reasons for differences though, especially when it comes to UKIP. The graph below has even starker differences. With Labour vs Conservatives the difference between phone and online polls is a matter of a few points. With UKIP there is a vast gulf between the figures from different pollsters…

phoneonlineukip

The companies showing lower UKIP scores are all telephone. The companies showing higher UKIP scores are all online. While there is little difference between the phone company showing the highest UKIP support (Ashcroft) and the online company showing the lowest (YouGov), there is a gulf of 9 points between the highest and lowest ends of the scale. Why there should be such a difference between online and telephone polling of UKIP we cannot tell – some of it may be an interviewer effect (people being more willing to tell a computer screen they are voting for a non-mainstream party than a human interviewer), some of it may be sampling (some online samples getting too many of the sort of people who vote UKIP, or some phone samples getting too few, or both). Until the results are in we won’t really know.


566 Responses to “Phone and online differences”

1 6 7 8 9 10 12
  1. ProfHoward – “Ed Miliband sounds as though he wanted to end the superficiality that has been associated with Labour and Tory image makers in recent times.”

    You still have to sell your policies – they never sell themselves. You just have to do it in a different way. Think of the amount of time an old school politician like Abraham Lincoln spent on his speeches and promotion. He didn’t just say “lets end slavery, the idea sells itself!”

    Regarding the presentation thing – a good analogy is wearing make-up. Anyone will tell you that the natural look takes as much art and make-up as a full make-up look. Only an idiot would think they would look just as good if they didn’t bother at all :-)

  2. @LRR

    As I set out above, and others have done at length, I really don’t think there’s any point trying to talk about geographical or general trends for the LibDems any more.

    There are Lib Dem held seats and each one is its own race. In some – like Bob Russell in Colchester it would appear – they seem to be doing exceptionally well, in others very badly.

    The Ashcroft polling of individual seats in the South West proves that you can’t even call across a region or according to required swing where they might win or not.

    Nationally they will almost certainly do very badly – locally it’s all down to the incumbent and their campaign.

  3. David
    We’ve been discussing this for an hour or two. Incidentally there at present 15 LD held seats but the one they missed out was Bristol West where Labour were second last time.

    Unfortunately this poll does not help if one wanted to know which seats were actually going to be held. Ashcroft has shewn how skewed it all is in the SW.

    Regarding NC’s pronouncements and names used, seen from his point of view I can understand it.

  4. LITTLE RED ROCK.
    Good Evening to you.
    I think that the Labour Party will do well in areas where they are competing for first place with LD’s and the SNP will take the Lib Dem seats in Scotland.

  5. UK ELECT.
    Thanks for the link, exciting stuff

  6. To add to the South West picture, UKIP could take enough votes off the incumbent Tory in Weston-super-Mare to let the LibDems in. You heard it here first.

  7. @Roger Mexico

    Thanks for the summary of the Ashcroft polling.

    I’m not sure that we will see these seats visited again before polling day – but one affect that I don’t think can be discounted is that all the LibDems money and efforts (in so far as is permissible) are being concentrated in these seats.

    From all the anecdotal evidence on these threads, it seems as though these will be among some of the most contested seats on the ground this election.

  8. RM

    See my last post, I don’t think they missed out Cheltenham.

  9. Nick Robinson just said on the BBC, that the Lib Dems poll ratings have “sometimes dipped below UKIP’s”. Err…

  10. JonBoy

    Yes I see your point. I see Weston very much as some would view the Essex Kent area. We call it Weston on Mud.

  11. I don’t know whether the Birmingham Mail has any political bias or not, but I found this primer on the SNP apretty fair offering to Brummies.

    http://www.birminghammail.co.uk/news/news-opinion/who-snp-matter-england-9022165

    As to their online “pol” (Harrumph!) I clicked on it to see the views of the locals – but I suspect I was only seeing the view of all the other Scots who had done precisely the same! :-)

  12. Evan Davies interviews David Cameron on BBC1 in 5 minutes’ time.

    This is the second of the “Leader” interviews. The first one was Clegg, I found it interesting:

    http://www.bbc.co.uk/iplayer/episode/b05rcmts/the-leader-interviews-nick-clegg

  13. Bristolianhoward

    Just joined the discussion when the ITV poll was announced and thought I would share the info. The TV poll seemed to suggest that the Libdem Devon / Cornwall seats are most at risk.

  14. Been touring Tim Farron’s seat which seems to go on forever. Too busy feeding ducks and swans to get any ideas of which way the seat will go- plenty of posters up for both LD and Tory although sometimes difficult to tell where one farm ends and another starts so it could be those “plenty of posters” just covered two farmers! Quite a few Farron window bills up in the towns I guess he will hold the seat.

    Anyway, having been away, has there been an Ipsos-Mori poll yet this month? Also is ICM going to start doing weekly polls now- I think they need to!!!

  15. @Alan

    The other puzzling thing was the error bars didn’t seem to tighten as more polls worked their way into the model. I’d expect those to contract if we are adding more polls to the mix. […]
    If we’re aggregating polls it should become more sensitive over time?

    As I understand it, this is because the error bars are determined not by numbers from the rolling sample, but by long-term estimates of SD. So, the figure to be plugged into the formula for our purposes here would be 0.5 x the MoE figure you derived yesterday – so, not linked at all to the recently recorded VIs.

  16. Anyone else getting a bit annoyed with the Vox Pop’s the TV and newspapers are doing now the election has started?

    I have not yet noticed one person expressing the view of any of the most notable groups of swing voters that polling has identified- ie Con to UKIP and LD to Labour. If anything they seem to concentrate on Anthony’s little old lady in Wales who voted Green last time and is now BNP…

  17. ShevII

    Last time I looked, Farron was 12/1 on.

    I suspect the most likely LibDem to keep his seat

  18. David

    Yes I think the blood letting will be certainly those in Cornwall, plus Mid Dorset&N.Poole, Somerton&Frome, Wells, Taunton Deane, all look goners to me. Where else? Not easy to predict. Torbay could be saved by UKIP voters possibly.

  19. Roger Mexico

    I thought that the polls was specifically LD seats with Con in 2nd place, which would exclude Bristol West? Which gets us back to 14?

  20. Re: South West Lib Dems. I do think Laws will hold onto Yeovil due to the incumbency factor. He is a popular MP locally. So I think the ICM poll, which suggests a total wipeout is not sensitive enough to predict the variations of swing in different seats. I would be surprised if they lost all 14.

  21. Extreme caution advised of those South West seats polls were performed by ComRes over the phone.

  22. Shev11

    We eschew all UK news progs at the moment. I rely totally on UKPR for my election coverage. Excellent it is too.

  23. @ Assiduosity

    I have to agree with your view on the LD’s vote distribution making the national polls virtually useless as a guide to what’s going on in their strongholds. So what if the national vote share goes up or down by 1 or 2%, this won’t flow through evenly in the seats where they are still strong.

    Events could give modest LD improvement at national level by picking up some of their lost VI from the Tories (or Labour), but cost them dearly in seats where their sitting MP’s personal vote had been holding on to votes that might otherwise have slipped away to Labour (or Tories).

  24. Alan
    I already made that point, see above. Thanks.

  25. @ukelect

    have updated the projections, I think my browser needed to refresh before I picked up your changes.

    here are the updated projections, including yours, thanks for the information:

    https://onedrive.live.com/redir?resid=698F47EA25B48A7A!464532&authkey=!AHVtd26ZOIu9tvc&ithint=file%2cpdf

  26. @Candy
    The article made me think that Miliband’s problems stem from vanity”
    I think you misunderstand Miliband. I understand I think where he is coming from and it is definitely not vanity. It makes me wonder why people attach so much importance to all the superficial stuff. What people do is so much more important than what people say or their appearance and if that is how you think then why would you pay attention to all the superficial stuff. I wish the world was full of more people like Ed Miliband.

  27. Unicorn

    I agree the initial boundaries after a reset should be there, but after 10 polls with no reset, we have a larger effective sample size to work with? It’s not 10x the size due to downweighting but it’s significantly larger.

    I thought that was part of the point of these plots to make use of aggregation without weighing too much on old data?

  28. ROGER MEXICO

    ComRes sounds more reasonable to me because the LibDems are heading for a loss of two thirds of their former vote share. I really doubt that they are able to increase their majority in any constituency in the country. Ashcrofts polling numbers for Thornbory& Yate are looking pretty strange. Yeovil and Bath will stay liberal, all other seats are really in play and most of them will flip.

  29. Bristolian Howard

    So you did, I was going to reply sooner but grabbed a bite to eat and replied without refreshing.

    If I could edit my posts I’d give a citation.

  30. @ Candy

    “Most politicians have a strong belief in their political views.”

    I don’t think our current PM and Chancellor believe very strongly in anything. I think it shows more vanity to believe you have such brilliant PR skills that you are able to sell the voters any policy irrespective of whether you actually believe in it.

  31. I’m going to keep banging on about my theory that LibDems will be lucky to get 12 seats. Incumbency will not save many of them. In 1970 they lost 6 of 12 seats and the losses included several with very strong local followings such as Wally Lawler and Eric Lubbock.

  32. Oops! B-Howard is right (as he ought to be) and the omitted seat is Bristol West – I missed that bit in ComRes (though they do seem to have a thing about Glos).

    The Ashcroft figures are:

    ** Cheltenham (Nov) LD +8 Swing 1%

    which reduces the average swing even more and highlights the discrepancy with ComRes

  33. ASSIDUOSITY
    ‘From all the anecdotal evidence on these threads, it seems as though these will be among some of the most contested seats on the ground this election.’

    I worked out the number of seats likely to change hands by region. The South West does come second.

    Region Seats Changing Seats %age
    South West 14 53 26.4%
    South East 7 86 8.1%
    London…. 7 73 9.6%
    East of Eng 7 62 11.3%
    East Midlands 7 44 15.9%
    West Midlands 9 56 16.1%
    Wales…… 3 40 7.5%
    North West 12 74 16.2%
    Yorkshire 5 57 8.8%
    North East 3 29 10.3%
    Scotland… 46 58 79.3%

  34. Anecdote Alert!! Anecdote Alert!!

    It finally happened!! I have heard some people talking politics. I’m settling in for Jazz Night, having a Mezze platter, and some students at the next table were talking about the US economy and the parties’ policies on tuition fees. (I don’t know any more ‘cos I’m a polite Public School boy and it’s rude to listen…)

  35. @ Unicorn and Alan

    Did you check for the power effect?

    I cant’t really engage – extremely hard day today and three more to come, but reading up the comments helped my mood.

  36. @LizH, Candy

    Raphael Behr’s article struck me as a cunning piece of hedging. As someone who was very critical of Miliband from the outset Behr is now having to face the possibility that Ed might be elected. He can’t disown his former opinions but he can fashion them into a narrative, a Bildungsroman of how Ed grew up. This can be read in two ways, emphasising the early failings, or the late improvements, so he can claim he was right whatever the election result. Mr Hodges has just performed a similar trick in The Spectator, though obviously more gracelessly.

    Behr also has a problem of a professional kind. His informants in the Labour party, whom he has cultivated over the last few years on the assumption that they would be the new shadow cabinet once Ed was defenestrated and who provided the anecdotes he used, may not survive long under PM Ed, and he will need to cultivate others if is to write about the new government convincingly.

  37. OldNat

    Unless, of course, [Clegg] has already abandoned the Scottish seats

    Well if he’s putting Salmond in the Commons already, he’s clearly abandoned one of them (Gordon). I would imagine that the SNP will be making some play with this. Though it may just be as Spearmint points out, that he finds the thought of a woman leader (and what’s worse one who isn’t even a long-term denizen of Westminster) incomprehensible.

  38. Well interesting comment on WSM … As a resident, I have not seen on Lib Dem poster up anywhere.
    there are far more Villa fans here than in Kent,

  39. @Pete B

    When I did my seats prediction last month, I had the LDs down yo 17, with 10 losses to Labour, 10 to the Tories (mainly in the South/South West) and 10 to the SNP. Seems to be in the right ballpark.

    However, I struggle to take ComRes phone polling seriously. It really lacks credibility. I find it somewhat surprising that a public sector UK broadcaster would chose such a pollster (who let us remember have the well earned moniker “Comedy Results”) for its GE polling.

  40. regarding the SW polls looking at the Ashcroft national voting intention in SW seats most were showing LD-Con swings in the low teens with an average of about 12% so not dissimilar to today’s ComRes. The big question is whether constituency or national VI is the more accurate

  41. MCCLANE

    Don Foster has built a large personal following in Bath and is now retiring. The seat used to be solid Tory and so it could easily revert.

  42. Lazy shot by Cook. Eng 20/2 (lead WI by 124).

  43. I don’t think we should be taking the Nowcast constituency ‘polls’ as gospel down to the nearest percentage, but they do throw up some interesting hints about what is going on in constituencies which haven’t been polled by Ashcroft (or others). The way in which they tend to confirm the Ashcroft results[1] in these Lib Dem seats is interesting – especially when you consider that there is not much correlation between what Ashcroft/YouGov have found and the simple results you would get by using UNS.

    In the case of Bristol West which should be ‘safe’ Lib Dem the fact that the Nowcast shows it very much isn’t (and that the much-touted Green challenge is missing) is an interesting indicator of what may be the case – and the only evidence we have apart from ‘opinion’. Looking at a similar ‘safe’ unpolled seat, Leeds North West, this is showing as ‘Too close to call’ on the Nowcast, though with Lib Dems ahead of Labour.

    Both constituencies have very high number of students (and presumably also of workers in the HE sector) and a similar electoral history (Tory forever, Lab in 1997, Lib Dem in 2005). This suggests that the Nowcast is certainly picking up something.

    [1] As far as I know there is no link between the two except that both feed onto the forecasts that the EF people make (or rather they use the same raw data as the Nowcast does. As one is phone and the other online (look I’ve linked to the thread theme!) it’s nice to have some degree of comparison.

  44. Pete B

    You have made this point about the Liberals in 1970 previously.

    I broadly agree with you, but I think comparisons with 1970 are redundant, as I think I am correct in saying that tactical voting has greatly increased since then.

    Presumably to do with polling, social media and general news coverage.

    If I am wrong, happy to be corrected.

    I still think the LDs will be lucky to get more than 20 seats.

    But the medium term implications of their slump are very significant: they are going to get completely wiped out across the country with lost deposits everywhere. Their local infrastructure, and local council seats will be decimated. The work of three to four decades has been destroyed. Possible meltdown, and Clegg has to be finished in these circumstances.

    In passing, Cameron most impressive this evening with Evan Davies – the Tory campaign has been extremely poor but is showing signs of getting its act together.

  45. Labour hold a 14 point lead over the Conservatives in London. 46% of likely voters in London say they will vote for Labour, ahead of the Conservatives on 32%. This compares to 37% of Londoners who voted for the Labour Party and 35% who voted for the Conservative Party in 2010.

    The biggest electoral issues among Londoners are the economy (52%), the NHS (49%) and immigration (34%), followed by housing (25%).

    Comres

  46. I thought DC did well with Evan Davis, EM’s turn next week.

    I came across this LD commissioned poll for Jo Swinson’s seat in East Dumbartonshire:

    East Dunbartonshire voting intentions (Lib Dem poll) :

    Liberal Democrats 34.5%
    SNP 32.1%
    Labour 16.2%
    Conservatives 13.1%
    Greens 2.0%
    UKIP 0.7%

  47. There are a couple of things in the news recently, which probably everyone here knows already.

    First the betting odds now make a Labour minority government the most likely outcome.

    Second, Jean-Claude Juncker, the president of the European commission apparently has excluded any renegotiations on the subject of the terms of Britain’s membership of the European Union taking place before 2019.

  48. Millie

    I missed the first 10 mins of the Cameron interview, but the bit I saw was a bit a mixed bag.

    Davies spent quite a bit of time banging on about where are the £12Bn cuts going to hit. To me Cameron looked a bit uncomfortable in this segment as he had to spend a lot of time not answering the question posed and wasn’t allowed to deflect it onto a prepared speech as Davies kept interrupting him.

    The final segment I though was just a bit weird. A long set-up to a clunky jump into a clip from the 1992 party conference all just to ask are the Tories still the nasty party. Well, Cameron wasn’t exactly going to say yes was he?

    Overall on the 2/3rds I saw, my verdict was Cameron was competent, but Davies didn’t do a great job.

  49. Pete B
    57 – 30 = 27.

  50. Sorry Pete – should have been to RAF!

1 6 7 8 9 10 12