ComRes have an interesting post over on their site about differences between online and telephone polling so far this year (as well as making some extremely sensible points about the polls not being all over the place). As they correctly say, telephone polls this year have been showing a tiny Conservative lead, online polls a tiny Labour one. It’s only a small difference, but it’s there and it is not new – at the start of the year I produced a chart showing house differences between the different polling companies over 2014, and even then an online vs telephone tendency was observable: the two most “Toryish” polls were Ipsos MORI and ICM, both done by telephone. The most “Laboury” polls were TNS and Opinium, both done online.

Look a little closer though, and things are not quite that cut and dried. There are many causes of variation between polls, telephone or online fieldwork is just one of them. There is variation between different online companies and between different phone companies. Last year ComRes’s telephone polls actually produced some of the more Laboury figures, the online Populus polls tended be on the Tory side of average. Below is the average for each company so far this year (given the polls have been pretty static in 2015 I haven’t worried too much about timings of different companies polls, it’s just a straight average).

phoneonlineleads

So all three companies who have been showing a Tory lead are done by phone, all the online polls have been showing an average Labour lead. But note the variation – MORI use the telephone, but they are showing a Labour lead on average. Two online polls (YouGov and Opinium) show barely any Labour lead at all, Survation, TNS and Panelbase average around a 2 point Labour lead. This is because there are plenty of other reasons for variation between pollsters too, different approaches to weighting, turnout, don’t knows and so on – I summarised lots of them here. Just looking at one can sometimes be misleading, for example, ICM and Ashcroft also reallocate don’t knows by past vote, which normally bumps up the Tory position by a point or so, so that will also be a major part of the difference between them and companies showing worse results for the Conservatives (one should also bear in mind that the monthly polling companies have only produced 3 or 4 polls this year – so a single odd poll like ICM’s this month has a large impact on the average).

I’ve no doubt that telephone vs online is one of the reasons for differences though, especially when it comes to UKIP. The graph below has even starker differences. With Labour vs Conservatives the difference between phone and online polls is a matter of a few points. With UKIP there is a vast gulf between the figures from different pollsters…

phoneonlineukip

The companies showing lower UKIP scores are all telephone. The companies showing higher UKIP scores are all online. While there is little difference between the phone company showing the highest UKIP support (Ashcroft) and the online company showing the lowest (YouGov), there is a gulf of 9 points between the highest and lowest ends of the scale. Why there should be such a difference between online and telephone polling of UKIP we cannot tell – some of it may be an interviewer effect (people being more willing to tell a computer screen they are voting for a non-mainstream party than a human interviewer), some of it may be sampling (some online samples getting too many of the sort of people who vote UKIP, or some phone samples getting too few, or both). Until the results are in we won’t really know.


566 Responses to “Phone and online differences”

1 7 8 9 10 11 12
  1. @Cloudspotter

    Very useful set of figures. Even if not all those seat changes actually come about it certainly indicates where much of the campaign action will be concentrated.

    They also serve to indicate that whilst we could see potentially significant changes in the parties’ standings votes cast in some parts of the country – London for example – this isn’t always translated into parliamentary representation in our system.

  2. @exileinyorks
    “The final segment I though was just a bit weird. ”

    The Evan Davis interviews seem to follow a pattern. Initial 80% is a grilling, and the final 20% is Evan giving the leader an opportunity to defend themselves or be positive about their vision. The same thing happened with Clegg, in the last few minutes Evans made some frankly ridiculous implications about Clegg’s ability to speak 5 languages and quoted Clegg saying some aspects of other Euro countries are better than Britain. Then he just allowed Clegg to rebutt it all and make some positive statements about the Lib Dem vision etc. That’s what happened with Cameron, Evan gave him the chance to respond to the nasty party image.

  3. @Jim Jam

    Ah…I meant 27.

    There’s clearer a pincer (or should that be mincer) movement on the LD vote as the electorate seems intent on voting on polarised lines. Not much to be gained by being a centrist – particularly not a governing Orange Book centrist.

  4. Carfrew
    As a polite ex scholarship public schoolboy, I know that it’s always polite to listen. Pity so many of our fee-paying public schoolboys bluster on with no humility.
    The trick is to listen, digest and apply the killer punch at the appropriate juncture, but only if it arises.

  5. @ omnishambles

    I agree on there is a clear structure, and giving the interviewee at bit of time at the end to pitch their own message is ok. I just don’t see the point of wasting time on the clunky set up.

  6. @Bantams

    And how reliable would you say the East Dunbartonshire poll is?

    It shows 4% drop in LD support, 2% drop in Conservative support, 21% increase in SNP support, mostly at Labour’s cost.

    Seems to me to be a very interesting place to watch, should the LD poll be credible.

  7. I was in the Devon East constituency again today, and there are large numbers of boards supporting the Independent candidate, Claire Wright. I spoke to six locals during the course of the day, from differing backgrounds and they all said they were voting Independent: it is all to do with the local council apparently.

    It has a Dr. Taylor and Wyre Forest feel.

    Wright is 5/1 with the bookies, and therefore by far the best backed independent candidate in the election, Lady Hermon notwithstanding.

  8. But I applaud your discretion!

  9. @exileinyorks

    Yeah, you’ve got a point. I think he was trying to say something about the tone of the Conservative party and how it’s changed. Evan said he couldn’t imagine a Tory politician making a speech like that nowadays. Maybe they showed the clip for the benefit of younger viewers who don’t know what the Tories were like in 1992, or people who have just forgotten.

    Either that, or the BBC have a nice big TV screen and it would be a shame not to use it.

  10. Anecdotal poster analysis

    Long drive today tho only on main roads

    Brighton Kemptown -big battle ,peacehaven strong for tories,some labour, no sign of the greens .Didnt go towards brighton where most of the labour vote will be.One kirkby poster amusingly amended.

    Lewes -quiet but 95percent for Norman Baker

    Eastbourne -mega battle ,giant posters on main roads for Stephen Hughes .But plenty for the Tories too.

    Rural sussex -no sign of election anywhere.

  11. @Exileinyorks
    @Omni

    Completely agree with the pointlessness of the Peter Lilley component. What was that all about? I was surprised that Cameron wasn’t more disrespectful of that segment.

    Cameron must be pleased with his performance – as Evan said ‘last question’ I thought how comfortable DC must have felt. I thought he answered most questions competently and it was actually quite an informative and useful discussion. As others have commented on this site, Cameron is the Tories’ best asset and they should deploy him more frequently. Having said that, he is not suited to the debate format.

  12. @ John B

    On one side it’s a LD commissioned poll so obviously it would be worded as they wanted. Survation did the work. Nowcast have it as a likely SNP gain and the SNP think it’s a dodgy poll.

    It’s going to be close but the LD’s have been publicising the poll locally, perhaps to persuade a few Lab & Tory voters who voted “no” to lend their vote.

  13. Today’s Guardian is reporting that Clegg is getting Tory help for tactical voting to beat Labour. If Clegg is still an MP on the 8th of May he might have a favour to pay back. ;-)

    “In a letter sent out by the Lib Dems as part of their election literature, John Harthman – who stood against the rival party’s last MP in the constituency, Richard Allan in 2001 – says that Clegg was “incredibly brave” to put “the country’s interests before his own party interests” when he went into coalition with the Conservative party at the last election.”

  14. That East Dunbartonshire poll does seem ineresting and suggests that Jo Swinson has a large personal vote in that constituency.

  15. The Ashcroft poll in Sheffield Hallam also showed a fairly significant (~6%) bunch of Tories helping Clegg

  16. RAF and Millie

    It’s good to know that others think LibDems will get less than 20, even if you don’t go quite as far as me.

  17. @ Spearmint

    In looking at intention to vote Table 2/2 of Comres Southwest LD poll they appear to be saying, in this instance, that Green voters have the highest intention vote.

    Is that unusual, and what are the latest, figures for the various parties in terms of likely switchers?

    I am still finding a lot of genuinely undecided on the door step, who seem glad to get the literature but are not decided as to who they will vote for.

  18. @john b

    I would not be surprised at all to see the Tory vote fall in E Dumbartonshire and swing to the Lib Dems.

    @omnishambles

    It’s curious, much is made of tactical voting when it is anti-Tory but little discussion seems to be made of the habits of Tory voters who tactically vote for other parties, in one situation or another.

  19. ADGE3

    I wonder what effect on the tory vote that statement by Juncker has had as its undermines the conservatives negotiation then referendum pledge for 2017. Though it was reported down the bottom of the main paper pages and ignored by TV.

    “Jean-Claude Juncker, the president of the European commission apparently has excluded any renegotiations on the subject of the terms of Britain’s membership of the European Union taking place before 2019”

  20. Does anyone think there may be a shy Lib Dem factor this time?

  21. @Garry Gatter

    Thanks, but my current ukelect prediction is for Con 269, Lab 279, SNP 50, LD 27, UKIP 2. Perhaps my browser needs to be refreshed as well!

  22. @vgfleet

    I suppose it depends. Some people might take notice of Junker’s statement and in isolation it doesn’t sound good for Cameron, but others might hold the view that Junker is not anywhere near as relevant in any hypothetical renegotiation as the German Chancellor.

  23. Pete B

    We are not alone.

    The bookies have the LDs to take 25 seats or less at 4/6 and the odds are shortening.

  24. @ProfHoward

    In some parts of the country a shy Lib Dem factor certainly wouldn’t surprise me! Reconciling the polls certainly doesn’t seem to be becoming any easier..

  25. I am in Hampstead and Kilburn, now sure about shy LD , but many middle class LDS have changed to Labour (maybe some tories) or greens

  26. CARFREW

    @” I’m a polite Public School boy ”

    Do you mean :-

    a) polite because all Public School Boys are polite

    or

    b) polite despite being a Public School boy, most of whom are not polite ?

    Could you explain why it was necessary to explain your education at all in this context. Why could you not have just said ” I am polite” ?

  27. Did the Lib Dem battle bus really break down today? Great metaphor.

  28. Mike Freer, the Tory MP for Finchley & Golders Green is arguing in the Telegraph that Ashcroft’s recent poll showing Labour 2% on his patchup is slightly skewed.

    He said “many Jewish voters (22% of the total voter numbers) who might have backed the Conservatives in Finchley and Golders Green would have been observing the Passover festival so would not have taken part in the poll by Lord Ashcroft.

    They wouldn’t be answering the phone, they may not even be here; many people will travel or they are very observant. The 9th and 10th especially are the two high holy days.”

    He also said, “The raw data also suggested the poll consulted a disproportionately high number of people who voted Labour in 2010 compared with the result in the seat.”

  29. OMNI

    The bits I enjoyed most were :-

    Cameron saying-yes I’ve been reading that too , as he pulled the 2010 stuff out of his pocket.

    and ED complaining that DC was “bamboozling me with numbers”.
    The biter bit :-)

    I think DC did OK-and became animated where he needed to.

  30. Colin

    :-) :-)

    I thought he was indulging in self deprecating and/or ironic humour rather than inverted snobbery, but I expect he will tell us.

  31. ProfHoward

    That East Dunbartonshire poll does seem interesting and suggests that Jo Swinson has a large personal vote in that constituency.

    It’s possible, though the only reference I would find was in the local paper:

    http://www.kirkintilloch-herald.co.uk/news/local-headlines/libdem-poll-predicts-neck-and-neck-race-1-3744775

    which confirms it is one of those Survation polls which they had a row with the Lib Dems over. Certainly we know from the one poll for which more details were released that there were various prompting questions asked before asking VI aimed to maximise the Lib Dem vote. Given that the poll is still so tight, Swinson would probably be behind in a ‘normal’ poll.

    Certainly there is potential for her to squeeze Conservative votes, we know that some of these are willing to switch to voting Lib Dem, though whether she could also pick up many votes from Labour is another matter – they are not too far behind the Lib Dems normally here, though they are in this poll.

    But squeezing is a difficult game – many Labour and Tory voters may be just as keen to vote tactically for the SNP so as to defeat an old enemy. Unionism isn’t everything, even for No voters. The Yes vote was 39% here and if that goes to the SNP she’s going to need more than the 35% she has here.

    The YouGov Nowcast shows the seat as ‘Likely’ SNP with them in the low 30s ahead of scattered opposition though, with Lib Dems second, just ahead of Labour. So it’s possible if she can herd enough unionist cats into the polling booths and get them to pick her as their representative. But it isn’t easy.

  32. Lea Bridge Labour added,

    Sunny Hundal @sunny_hundal
    Labour has a 14pt lead over Tories in London, with 46% in new poll. Jump from 37% who voted Lab in 2010. #redLondon http://www.comres.co.uk/polls/itv-news-london-political-poll/

  33. I wonder if Labour can use more votes in London – are there many marginals there?

  34. Thought nicola and alex were good on newzoids tonight,the queen was useless. .Ed and Dave were a bit predictable

  35. HOWARD

    Which bit was that ?

  36. @Bantams

    Fair point. I don’t think anyone expects F&GG to go Lab. It certainly struck me as very odd.

    However, we need to bear in mind that Ashcroft’s national polls tend to lean a bit Tory. For a start they have have had very few Lab leads this year. Provided this national effect is also reflected in constituency polls, it is likely that the weighting of this poll and the special factors you describe may be partially offset by the method (Ashcroft phone polling) that slightly benefits the Tories.

  37. Yes, a perfectly competent performance by Cameron. Except for asking people to simply trust him on the evidence of his past record – I just don’t understand how that is supposed to convince people who haven’t been convinced so far.

  38. @ CHRISLANE1945

    Didn’t we see this London poll several weeks ago?

  39. Liz H,
    Well said.
    Crossbat’
    Agree with your comments regarding Behr.What I found rather sad was the
    political journalist who congratulated EM on a brilliant speech and then said that he was sorry he was going to write horrible things about him.What an
    inditement of the great British press.

  40. BANTAMS.
    Hi there, just back from the beach. I have had the poll tweeted.

  41. @ RAF

    Is there a remote possibility that Ashcroft has massively understated the LD’s VI? 15% would do nicely.

  42. Colin

    I wa referring to your very amusing post to Carfrew. :-)

  43. @Prof Howard

    Firstly the 14% Lab lead in London (a swing of 6%) was a ComRes phone poll and simply restates Comres’s previous London phone poll – so no change.

    Secondly, ComRes rather unusually are probably overstating the Lab lead here – no other pollster has Lab 14% ahead.

    Thirdly, and rather surprisingly there aren’t that many Con-Lab marginals in London. Labour can probably pick up a net 8-10 seats in London, but when you set that against the total seats in London it isn’t that much. London is a bit of a doughnut: Labour controls most of the inner part, with the Tories substantially ahead in the suburbs.

    @Richard did some analysis after the last batch of London polls that showed Lab could if everything went perfectly win up to 12 seats.

  44. Looking at the London Comres poll I wondered why, as far as I could see, they didn’t ask whether the respondent was registered to vote or not ?

    I don’t often look at the detail of polls, but I don’t think i’ve ever seen that question asked. If even 5% of respondents aren’t registered, it could make a huge difference to the result of the poll. My guess would be that more potential Labour voters would not be registered if that’s not too partisan.

  45. HOWARD

    Ah-sorry I had moved on.

    Yes -perhaps he will.

  46. COLIN
    Can I just say how refreshing it is to read that you think that it’s not necessary to explain your education within the context of political debate.

    Don’t you just wish that mr milliband agreed with us?

  47. @Bantams

    “@ RAF
    Is there a remote possibility that Ashcroft has massively understated the LD’s VI? 15% would do nicely.”

    I’m convinced the LDs will be squeezed but the Jewish vote is very important in F&GG and to carry out the poll during Passover was a little foolish.

  48. @ RAF

    Comres says the London poll was carried out between March 20th & 23rd so it’s a bit old hat.

  49. Pete B
    Voters are usually asked whether they are likely to vote. I suppose if they reply ‘not likely’ it begs a supplementary asking whether they can and may vote, anyway!

    Incidentally RAF was 10 out on his arithmetic, so he may not be in your camp yet (LDs fewer than 20 seats). His calculation of a loss of 30 seats leaves 27.

  50. Pete B

    TNS (which did ask about registered to vote) had London with the lowest number – 88% – compared with low 90s in the other English regions.

1 7 8 9 10 11 12