ComRes have an interesting post over on their site about differences between online and telephone polling so far this year (as well as making some extremely sensible points about the polls not being all over the place). As they correctly say, telephone polls this year have been showing a tiny Conservative lead, online polls a tiny Labour one. It’s only a small difference, but it’s there and it is not new – at the start of the year I produced a chart showing house differences between the different polling companies over 2014, and even then an online vs telephone tendency was observable: the two most “Toryish” polls were Ipsos MORI and ICM, both done by telephone. The most “Laboury” polls were TNS and Opinium, both done online.

Look a little closer though, and things are not quite that cut and dried. There are many causes of variation between polls, telephone or online fieldwork is just one of them. There is variation between different online companies and between different phone companies. Last year ComRes’s telephone polls actually produced some of the more Laboury figures, the online Populus polls tended be on the Tory side of average. Below is the average for each company so far this year (given the polls have been pretty static in 2015 I haven’t worried too much about timings of different companies polls, it’s just a straight average).

phoneonlineleads

So all three companies who have been showing a Tory lead are done by phone, all the online polls have been showing an average Labour lead. But note the variation – MORI use the telephone, but they are showing a Labour lead on average. Two online polls (YouGov and Opinium) show barely any Labour lead at all, Survation, TNS and Panelbase average around a 2 point Labour lead. This is because there are plenty of other reasons for variation between pollsters too, different approaches to weighting, turnout, don’t knows and so on – I summarised lots of them here. Just looking at one can sometimes be misleading, for example, ICM and Ashcroft also reallocate don’t knows by past vote, which normally bumps up the Tory position by a point or so, so that will also be a major part of the difference between them and companies showing worse results for the Conservatives (one should also bear in mind that the monthly polling companies have only produced 3 or 4 polls this year – so a single odd poll like ICM’s this month has a large impact on the average).

I’ve no doubt that telephone vs online is one of the reasons for differences though, especially when it comes to UKIP. The graph below has even starker differences. With Labour vs Conservatives the difference between phone and online polls is a matter of a few points. With UKIP there is a vast gulf between the figures from different pollsters…

phoneonlineukip

The companies showing lower UKIP scores are all telephone. The companies showing higher UKIP scores are all online. While there is little difference between the phone company showing the highest UKIP support (Ashcroft) and the online company showing the lowest (YouGov), there is a gulf of 9 points between the highest and lowest ends of the scale. Why there should be such a difference between online and telephone polling of UKIP we cannot tell – some of it may be an interviewer effect (people being more willing to tell a computer screen they are voting for a non-mainstream party than a human interviewer), some of it may be sampling (some online samples getting too many of the sort of people who vote UKIP, or some phone samples getting too few, or both). Until the results are in we won’t really know.


566 Responses to “Phone and online differences”

1 8 9 10 11 12
  1. @AnninWales

    “What I found rather sad was the
    political journalist who congratulated EM on a brilliant speech and then said that he was sorry he was going to write horrible things about him.What an
    inditement of the great British press.”

    Yes, I found that piece in the Behr article rather dispiriting too, but I guess reporters, probably fearful for their jobs if the do otherwise, feel compelled to follow the editorial agenda and proprietorial line of the newspapers who employ them.

    I always remember someone I worked with 25 years ago who was a Parliamentary intern for the Lid Dems telling me how chummy and incestuous Westminster was, with most of the political journalists in on the cosy cabal too. What they wrote quite often had no relationship with how they really felt and how they behaved with the politicians in private.

    This applies to MPs too. There was a Marr programme a few months ago that had Anna Soubry as a guest early on and Alex Salmond as an interviewee later. At the end, Soubry and Salmond engaged in a lively spat about the SNP and likely coalitions with Labour. As the credits rolled and Marr bade farewell, the camera hadn’t entirely cut to him and Soubry, unaware, was still in shot. She could be seen giving Salmond a wink and a knowing grin.

    Confection politics, devoid of authenticity and genuine passion and dare I say, honesty too.

  2. @raf @bantams

    It’s surprising that even the Good Lord can forget Passover

  3. DAVID COLBY

    In fairness Carfrew didn’t mention it in the context of politics-which was why I couldn’t see why it was relevant.

    For Labour politicians, the private education of Labour politicians is not relevant to their beliefs-clearly.
    But they clearly think that the private education of Conservative politicians is.

    I actually think that for some nihilist politicians on the left even Oxbridge is unaccaptably “elite” . At least there is a position to be debated with them here ..

    But the use of an opponents education as a tool in personal criticism has always seemed a very unattractive ploy.

  4. @07052015

    Interesting to read your commentary on the campaign in East Sussex. Looks quite lively there.

    I live in West Sussex. Today I went from home in Angmering to work in Upper Beeding, thence to Brighton city centre via Poynings and Devil’s Dyke, then to Littlehampton along the coast, and finally home. Passed through substantial parts of six constituencies, two of which are marginals (Arundel & South Downs, Brighton Pavillion, Hove, Worthing East, Worthing West, and Bognor & Litlehampton).

    Didn’t see a single poster, didn’t see the slightest sign that an election existed!

  5. The Nick Clegg interview with Evan Davis attracted a viewing audience of over 2 million which really ain’t that bad.

  6. Oldnat
    Thanks for that. Do you happen to know how they dealt with the non-registered? At the time they asked, I presume the deadline hadn’t passed, so their vote ‘might’ count.

    Bristolian
    There may be people who assume they are registered (but aren’t), or who don’t know if they are registered.

  7. @ukelect

    whoops, I am having one of those days. here is the updated version. Now all up to date.

    When I started this on the 4th only you (UK Elect), YouGov (nowcast) and Electorial Calculus were showing Labour as the largest party. But now MAY2015 and The Guardian have joined you.

    https://onedrive.live.com/redir?resid=698F47EA25B48A7A!464550&authkey=!ACm9a-xFYs_c5U8&ithint=file%2cpdf

  8. BANYAMS

    That is interesting.

    The BBC has been very low key about them compared with the Shoutathons which they have put up in neon lights.

    I always prefer a one to one interview, provided it is done impartially & forensically.

    Perhaps the viewing public feel that way.

  9. Nice one Gary.

    Now here is something reaaaaaaaly picky – it’s ‘Electoral’ Calculus (not ‘Electorial’)

    :)

  10. Crossbat,
    Indeed.

  11. Prediction tonight con 35, lab 34, lib dem 9, ukip 13

  12. @Smithy

    I go for 34/34

  13. SMITHY

    [YouGov] “Prediction tonight con 35, lab 34, lib dem 9, ukip 13”

    I see what you are thinking: the Tory Manifesto pledges.

    So in that vein I predict con 36, lab 33, lib dem 10, ukip 12

  14. CB11/AIW

    ‘Twas ever this. Lab as bad as well.

  15. Expecting a Tory lead tonight.

  16. I think level pegging tonight and a blue lead in tomorrow’s.

  17. @Geoff

    Thanks Geoff, I have changed that, ready for tomorrow :)

  18. Lab lead still I reckon

  19. Lab lead I reckon

  20. Been debating politics with my son-in-law (never a wise thing to do). Among other things he said that it would be ridiculous to vote Green as they would have us all wearing sandals. As it was only half a joke, I bet him that the word sandal did not appear in their manifesto but he shrugged this off as completely irrelevant, adding for good measure that the conservatives were the only choice as they would have us wearing solid working boots. How many people’s votes are determined by underlying gut certainties such as this?

  21. YouGov/Sun poll tonight – Labour lead by one: CON 34%, LAB 35%, LD 8%, UKIP 13%, GRN 5%

  22. Law of averages given how close things are that a Tory lead is due

  23. One of the reasons that there aren’t many seats for Labour to win in London is that they have been too successful in the past and held on to many they ‘should’ have lost in 2010 (Hammersmith or Harrow West say). So if they win back the obvious ones that Ashcroft confirms (Brentford, Croydon C, Ealing C, Enfield N, Hendon, plus Bent C and Hornsey from the Lib Dems), there are only a handful left (Battersea, Finchley, Enfield Southgate, Ilford North, Harrow E) that look even remotely winnable except in a landslide like 1997. And even in 1997 most of the remaining seats stayed true blue.

    With regard to Finchley, it is the most Jewish constituency at 22%, but it’s possible to overestimate the effect of this. After all Harrow East is 28% Hindu for example, but no one describes it as a ‘Hindu constituency’.

  24. the Tories will be disappointed with that, and Labour pretty relieved. With Cameron not being involved in the debate, Labour will have some reason to hope they could open up a small lead.

  25. Having moved back to live in a village near Weston-super-Mare from Bristol, I find the place has changed over the decades. But it still is a place where it is thought that they could put a blue rosette on a donkey and it would win. However, The Lib Dems have always been the tactical voting choice and took the seat for a couple of terms from 1997. This making it a Lib Dem/Tory marginal on the last 3 occasions. According to the Tory and Labour leaflets received so far, the voting in the local elections since 2010 have Con 36%, Lab 22% and Lib Dem 18% which maybe a reflection how the Lib Dem vote is waning. It is quite possible that UKIP or Labour may become the new tactical choice, but I don’t think they will come close enough to challenge in this particular donkey derby. Just like to say that this seaside resort has got plenty of sand before it becomes mud, but due to the 2nd highest tidal rise and fall in the world, the water is always has mucky look. The big positive plus for Weston is blessed with the best sunsets over the sea you could hope to see (if tide is in) due to it’s geographic position. A photographers dream.

    Bristol West, the place where I previously lived has been very Green since 2010. However, I detect that it may vote Labour enough to perhaps take the seat off the Lib Dems, as the Greens seem to have fallen back. The Tories have no chance here, so I cannot see how this would be one of the 15 Lib Dem seats on their target list. Definitely a marginal to watch though.

  26. @martinw

    YouGov/Sun poll tonight – Labour lead by one: CON 34%, LAB 35%, LD 8%, UKIP 13%, GRN 5%

    Labour lead as expected :D

    What Tory bounce?

  27. Pete B

    TNS excluded them from the stats.

    As to how accurate the figures are – who knows?

    The only figure I’d be confident about is the 94% in Scotland where, due to the referendum, registration has been handled differently.

    Those on the old roll were automatically transferred to the current one, and the process of updating it is happening through the year.

  28. Comment on Tonight’s YouGov poll: labour lead down from 2 to 1.

    The poll was taken after people had time to absorb the Conservative Manifesto Launch and its coverage. Therefore it is not surprising to see a swing towards the Conseratives. Indeed Conservatives may be disappointed not to see more of a bounce. The usual caveat about a single poll applies.

    In sum: continued polldrums.

  29. Thanks Martin, zero bounce for Tories

  30. Pete B

    Oh yes. When I first started this nonsense, I was guilty of wasting time on non-voters’ doorsteps. I soon learned to check my electoral roll before marching up the garden path. That’s not all. By not checking which member of the household you are canvassing the same mistakes can be made. I had interesting chats with Aunties who were in fact visiting from the other side of the country or renters who had only arrived a week ago!

    So glad that’s over long ago now. Sigh.

  31. Yep what I expected. If this had been a Con lead we would have heard well before 10.30pm

  32. Mike Smithson [email protected] · 3m3 minutes ago
    For fourth consecutive night LAB have lead in YouGov Sun poll
    CON 34%, LAB 35%, LD 8%, UKIP 13%, GRN 5%

    ROGER MEXICO/
    What is your view of the Bermondsey Race? Thanks!

  33. @Charles
    “….adding for good measure that the conservatives were the only choice as they would have us wearing solid working boots. How many people’s votes are determined by underlying gut certainties such as this?”

    Personally I want to wear some of those curly-toed slippers as worn by Arab or Turkish potentates in days gone by. Unfortunately I don’t know a party that espouses this policy.

  34. In the words of Allan Christie – things look worrying for…….Cameron

  35. @MartinW

    Fourth Lab YG lead in a row, but the margins of these leads are tiny. Still level folks. 3 weeks to go.

  36. We can at least conclude that the debilitating hoo ha on media about manifestos has made not a jot of difference. Had it done, then we should have seen larger swings, IMO.

  37. LABOUR LEAD SLASHED IN HALF BY STUNNING TORY BOUNCE

  38. The fact this is a fourth YG lead for Labour suggests that crossover has not happened yet.

    Lynton Crosby has argued that it will only be after Easter that crossover happens, so we can expect him to be turning up the heat in the next couple of weeks, to bring it about.

  39. @Pete B

    You can get those slippers in Southall. I have a pair somewhere…

  40. Tonight/ yesterday’s YG.

    It seems that there has been no tangible impact apparent from either of the two main parties manifesto releases (and related dominance of the news cycle).

    If nothing has moved (strongly) by the start of next week (in any direction), then it’s hard to see what can shift the even split between Lab-Con.

    So it will be down to each individual seat – the ‘ground war’- and any local issues/ regional effects/ sitting MP performance and good old ‘events dear boy events’.

    I do wonder though whether we are about to have another ‘shy voter’ election. In the sense of ‘shy labour’ in Scotland and ‘shy ukip’ in England…

  41. DRMIBBLES

    When a bounce is that small, the phrase “even dead cats bounce” comes to mind.

  42. @prof howard.

    one or two point movements in the respective tory/lab polling mean nothing. Its all margin of error stuff thats impossible to ascribe with any certainty to events such as manifesto launches.

  43. @RAF
    lol

  44. Next thing to possibly cause some shifts is Thursdays debate. Have the BBC been advertising it hard? I barely watch it these days.

    The Tory & LD no show could well knock them back a point or two. The mere fact UKIP & Lab get more prime time exposure should help them both at the expense of Tories and LDs.

  45. Just seen a clip on the news of Cameron saying he is 23 seats short of a
    majority.Is that correct?

  46. Scholarship pupils always had the edge over the fee-payers in my memory, in most contexts. What surprises me is that whatever their intellectual calibre, the fee paying public schoolboys are always seemingly able to outperform the state-educated in terms of presentation technique. Not that the technique should trump the content.

  47. Rob Sheffield

    I have a suspicion that there are a fair few shy Lib-Dems in England, Scotland and Wales.

  48. Labour maintain lead after Tory manifesto launch. Some significance in that maybe, although, like the admirable Spearmint, I would have been surprised if there had been anything more than a twitch in the polls. Manifesto launches are for us anoraks, really, and the general population only digest the related news snippets. From what I saw last night, they were fairly negative about the Tory proposals on RBT, particularly the evening BBC bulletin. I didn’t watch much else, to be honest..

    This challengers TV debate tomorrow intrigues me. Risk and opportunity for Miliband, I think. Risk is that he performs poorly and compounds the doubts about him; the opportunity is the mass exposure he will get and the chance it provides to chip away at his negative ratings. Lots of centrist voters to convince, but he’s making progress.

    Miliband has it in his gift to grant centrist voters permission to vote Labour. In that sense, he holds the key to the outcome of this election.

    Big, big night for Miliband tomorrow.

  49. I did expect at least a 2/3% Tory lead on the back of the RTB offer and it may still work though over the next day or so. If it doesn’t, what on earth do they do next?

  50. I have a question for the more committed statisticians and psephologists who understand these things far better than I do.

    Next week the postal votes will be going out. Most postal voters actually vote and fairly quickly too. So my question is, how do polls take account of the fact that a reasonable proportion of voters will have already voted by the time they produce the last week polls?

    Say, for argument’s sake, 20% of the electorate are postal voters (I’m sure someone will know what the actualy %age is) and 80% of those vote, that means that by the last week something like 16% of the electorate will already have voted. Now supposing there is some event in the last week that leads to say a 5% movement in the polls to one particular party, how do polls take account of the fact that this won’t affect the 16% who have already voted? Or don’t they?

1 8 9 10 11 12