ComRes have an interesting post over on their site about differences between online and telephone polling so far this year (as well as making some extremely sensible points about the polls not being all over the place). As they correctly say, telephone polls this year have been showing a tiny Conservative lead, online polls a tiny Labour one. It’s only a small difference, but it’s there and it is not new – at the start of the year I produced a chart showing house differences between the different polling companies over 2014, and even then an online vs telephone tendency was observable: the two most “Toryish” polls were Ipsos MORI and ICM, both done by telephone. The most “Laboury” polls were TNS and Opinium, both done online.

Look a little closer though, and things are not quite that cut and dried. There are many causes of variation between polls, telephone or online fieldwork is just one of them. There is variation between different online companies and between different phone companies. Last year ComRes’s telephone polls actually produced some of the more Laboury figures, the online Populus polls tended be on the Tory side of average. Below is the average for each company so far this year (given the polls have been pretty static in 2015 I haven’t worried too much about timings of different companies polls, it’s just a straight average).

phoneonlineleads

So all three companies who have been showing a Tory lead are done by phone, all the online polls have been showing an average Labour lead. But note the variation – MORI use the telephone, but they are showing a Labour lead on average. Two online polls (YouGov and Opinium) show barely any Labour lead at all, Survation, TNS and Panelbase average around a 2 point Labour lead. This is because there are plenty of other reasons for variation between pollsters too, different approaches to weighting, turnout, don’t knows and so on – I summarised lots of them here. Just looking at one can sometimes be misleading, for example, ICM and Ashcroft also reallocate don’t knows by past vote, which normally bumps up the Tory position by a point or so, so that will also be a major part of the difference between them and companies showing worse results for the Conservatives (one should also bear in mind that the monthly polling companies have only produced 3 or 4 polls this year – so a single odd poll like ICM’s this month has a large impact on the average).

I’ve no doubt that telephone vs online is one of the reasons for differences though, especially when it comes to UKIP. The graph below has even starker differences. With Labour vs Conservatives the difference between phone and online polls is a matter of a few points. With UKIP there is a vast gulf between the figures from different pollsters…

phoneonlineukip

The companies showing lower UKIP scores are all telephone. The companies showing higher UKIP scores are all online. While there is little difference between the phone company showing the highest UKIP support (Ashcroft) and the online company showing the lowest (YouGov), there is a gulf of 9 points between the highest and lowest ends of the scale. Why there should be such a difference between online and telephone polling of UKIP we cannot tell – some of it may be an interviewer effect (people being more willing to tell a computer screen they are voting for a non-mainstream party than a human interviewer), some of it may be sampling (some online samples getting too many of the sort of people who vote UKIP, or some phone samples getting too few, or both). Until the results are in we won’t really know.


566 Responses to “Phone and online differences”

1 9 10 11 12
  1. Ann , I think he was going on the 2010 result

  2. ED

    Tomorrows debate is a chance for UKIP and Labour to extract votes from other parties.

  3. its true the labour lead is small but don’t forget the fanfare and expectancy over the tory manifesto launch. promises on everything.you’ve never had it so good. great praise in most of the press. I honestly thought they’d be in the lead tonight and I’m sure the tories did too.
    swingback will happen after Christmas,after the budget,after easter ,after the manifesto launch. still nothing,very worrying for tories. this wasn’t the plan.

  4. @chrislane1945

    As someone from Bermondsey, now in Tower Hamlets, I think Labour could take it from the LibDems. I remember the Mellish/Tatchell ding dong,

  5. @ann in wales

    Yes.

    And he’s right, technically.

  6. @Norbold
    They ought at least to ask whether people have already voted!

  7. I wonder if people will watch this debate if Cameron and Clegg are away.

    The idea of a “challengers” debate is very strange.

  8. Agree re debates – could be a game changer tomorrow and I wonder if it goes well for Miliband, whether Cameron will have a little rethink about a head to head debate

  9. Ann in Wales

    I didn’t see the news clip, but taken in context within the interview DC was saying he needs 23 more seats than the Tories held at the end of the old parliament. This would be an absolute majority at 326. Arguably a few more than the bare minimum with the usual SF non voting assumptions.

  10. I don’t think RTB is a vote winner like it was in the 80s. Now there is a counter argument that it reduces the available building stock available to rent.
    I don’t see it impacting on poll ratings.

  11. Exactly the same YG figures last Wednesday night.

    @ON

    Glad you disliked the “imaginative” solution to the West Lothian question – it’s always a pleasure to see folk respond exactly as you expected them to.

    In fact it’s not my idea, and is used in some devolved assembly or other (read an article years ago about it, that was the only bit that stuck). It seems a reasonable idea to me, but then I’m not a separatist.

  12. People expecting a Tory bounce due to Right To Buy are ignoring or unaware of the polling around this policy – it isn’t particularly popular.

    Indeed more people oppose it. This isn’t 1979, RTB is hardly a fresh rabbit out of the hat, indeed it was in the 2005 manifesto and didn’t move Tory VI back then either.

    I think it’s time for everyone to face the polling music. And I for one am ready to bow down to our new Labour/SNP overlords.

  13. Paul Bristol
    Good point re the gradual effect of a policy on the polls. It begs two questions in addition to yours : what if the polled public turn against in three days time having thought about it? And what if the polled public appear to remain the same ? What do they do then? It’s an art not a science so totally dependent on subjective judgement.
    I scratch my head asking whysuchandsuch won’t ask suchandsuch question which appears so logical to me. But my logic Is someone else’s badass madness. And who am I to judge?

  14. @ProfHoward
    “I wonder if people will watch this debate if Cameron and Clegg are away.
    The idea of a “challengers” debate is very strange.”

    My thoughts exactly. Perhaps as it’s on the BBC some will tune in.

  15. Paul – I didn’t think the right-to-buy would be a vote winner. Anecdotally I’ve not seen any recent policy attract so much vitriol among people under 40 and private renters, and there’s many of them now due to housing policy failures. The ring wing press were also very sceptical.

    Most know many right-to-buy go the private landlords pretty soon after being sold, and this increases housing benefit costs to taxpayers substantially, which doesn’t play well on the right.

    Those in private rents or who recently bought also see it as deeply unfair, as do those with children struggling to rent or buy. It managed to annoy far more than those who will benefit.

  16. David Laws saying Labour are wrong not to put a date on deficit elimination. Why? How can a potential LD voter from the left of centre relate to that? It seems strange

  17. Is The challengers debate due to Cameron not wanting to take part in more than the one debate? Otherwise it does seem odd.

  18. GARY GATTER – Agreed about Bermondsey. The population is so transient there that incumbency means less. Labour are throwing a hell of lot at it. the LDs seems lacking on the ground.

  19. Good evening all from East Renfrewshire.

    Great result for the Hoops tonight and not forgetting the Reds a few nights ago…YNWA.

    Now…I’m watching the STV interview between Bernard Ponsonby and the Scottish UKIP leader David Coburn. He was asked where he would find the 5 billion he would cut from the Scottish block-grant.

    His answer was almost as tragic as the Lib/Dem fortunes in the West Country., Answer…He would cut the First Ministers salary.

    Tragic!!

  20. Over housing stock in the 80s vs under housing stock today. It was popular because the government owned land that people wanted to own. Today families worry about a lack of government land for their children and family to be housed in.

    Lose more than they win IMO.

  21. Looks like there were no Housing Association tenants with a decent income and deposit in the sample! ;)

  22. Time once more for:

    On This Day in 2010 on UKPR

    Today, Thursday 15th April, 2010, was the big day of the campaign – the first televised leaders debate. The immediate post debate polls were unequivocal – YG had Clegg on 51%, Cameron 29% and Brown 19%, ComRes had it 46/26/20, Populus 61/22/17. Cleggmania commenced.

    It was also the day that Gary Barlow was unveiled as a Tory secret weapon, as @Woodman pointed out –

    “Cameron has just announced a new initiative. On stage at a school in Nantwich with Gary Barlow, he said that a Tory government would introduce a national music talent competition. It will be called School Stars and it will involve local and regional heats, culminating in a national final in June 2011. The winners would get the chance to record a song with Barlow.
    Can somebody please tell me what is going on??”

    Well Woodsman, it’s called an election. Politicians make stupid promises and large numbers of the British public fall for it. Again.
    [For the sake of completeness, the ‘School Stars’ initiative never happened. Again.]

    Anthony warned against paying attention to voodoo polls on the debate, of which there were many, which led to a poster known as @Andy to climb straight in with this observation –

    “Anthony your only saying this because YOUR A TORY.
    ITV Online is NOT a vodoo poll it had over 50,000 votes!”

    Poor grammar, poor manners, poor understanding of polling basics, and poor appreciation of who wields total power over UKPR.

    Anthony responded, outwardly politely and patiently saying –

    “Andy – nope, I’m saying it because I know what a voodoo poll is and understand how polling works”, but sometime later, a body of a former poster was seen floating down the Mersey, apparently.

    Back to the debate. @Robert C focused on the post debate air war –

    “Remember, it’s not just the debate but the debate about the debate. Let the spinning commence.”

    Hmm. There was certainly some debate over this. That would be the debate about the debate about the debate, I guess. Swiftly followed by the debate about the debate about the – well – can you see where this is going?

    @Barnaby Marder gave us this gem –

    “Let’s not try to predict the polls tomorrow, it’s like trying to guess what the special will be for lunch in the pub. It’s mildly interesting but it’s pure guesswork.”

    [Somewhat fascinatingly, with 24 hours YouGov had the shock results – Conservative down three on Roast pork with apple sauce, roast spuds and choice of two veg, Labour up three on Neapolitan pizza and side salad, with Lib Dems taking the lead with Lancashire Hotpot and free hot drink. ]

    Finally, @Eoin Clarke rode to the defence of Anthony, saying –

    “I am a teeny bit annoyed at Anthony’s treatment. For those who are not aware- he does not get paid to do this. It is his own kindnes time and dedication. Anyone with children or a care committment will appreciate that more than anyone.”

    Well said, but blimey – I didn’t realise Anthony was our carer! I just hope he claims the Carers Allowance.

    More later

  23. PI

    Oh, I can see the attraction.

    If folk from Argyll & Bute or O&S had representatives on the councils in Northumberland or Warwickshire, they could teach the poor souls how to run an election. :-)

  24. COLIN
    I agree with you. Just to be clear though, I thought your rant at to Carfrew was uncalled for. He may have said ‘I’m a good catholic boy’ for example or ‘old fashioned’.

  25. SMITHY

    “In the words of Allan Christie – things look worrying for…….Cameron”
    ________

    My words but…………………….a bit premature. ;-)

  26. @AC – very true – good job Cameron’s got plenty of time left….. Oh wait!!!

  27. Cue pressman.

    Polls dont move lets dish the dirt

    Mail goes for thornberry,telegraph goes for axelrod,sun goes on about the miliband kitchens.

  28. JohnTT
    “David Laws saying Labour are wrong not to put a date on deficit elimination. Why? How can a potential LD voter from the left of centre relate to that? It seems strange”

    Odd isn’t it?

    Having been a senior member of a party that campaigned in 2010 on NOT cutting the deficit rapidly, then (briefly, admittedly) being the First Sec of a Govt that went for all out deficit cutting, then, having served his penance, returning as a Minister in a Govt that wasn’t cutting the deficit rapidly, you’d think Laws would be prepared to cut other folk a bit of slack.

  29. http://w ww.theguardian.com/politics/2015/apr/15/david-cameron-tax-havens-cayman-islands-david-maclean-lord-blencathra

    If true, very interesting. Gives an insight into the government’s mindset at the time, suggesting that the very limited measures on tax avoidance were simply negotiating ploys, without serious intent.

  30. Alec
    I admire your ability to look back five years without cringing at your own output. I suspect that’s because you were reasonably sane then! I’ve never been able to look back five years without cringing! I’m hoping my nearly five year old boy’s birthday will change that.
    Good stuff, though. Keep it coming, and please stop after the coalition negotiations!

  31. @ ON, PI

    I think I’ve worked out the Northumberland late declaration issue. The Mackems in Sunderland just over the Tyne like to be first, so just to show they are different the Northumbrians want to be last. ;-)

  32. UKIP

    Based on the polls, I reckon UKIP is about 16% nationwide.

    Having looked at a map in the Telegraph showing where
    UKIP support is likely to be located, it is reasonable to conclude the 16% is so concentrated that it could result in far more seats for UKIP than we are being lead to believe.

    This is backed up by the recent telephone (shy voter!) poll showing UKIP at 24% in Kent (17 seats) where UKIP support is concentrated in 4 seats.

    This would explain the intensifying anti UKIP propaganda campaign, the blatancy of which is now so over the top that it will probably backfire in a big way.

  33. Still looks like neck and neck to me on the trendline: 34% each Conservative and Labour, 8% LD.

    Just been looking for betting odds on seat totals: nowhere can I see what the odds are of Labour and Conservatives getting exactly the same number of seats. Now I know this would not normally be even worth bothering with in most elections. But for this one, maybe. If this were to happen, I would pick 277 seats for each at this point.

    But that’s only if the Conservatives gain from
    “swingback” or whatever it might be labelled. If not, then neck and neck means Labour largest party for generally known reasons.

    Nothing seems to be shifting the polls this week.

  34. Leftylampton
    The way you put it, it seems the speakers are beholden to the set narrative. Original thought is not helpful apparently. I’m surprised that Tim Montgomerie has had such a prominent voice. He seems to be a stronger counter-balance to the stringent Toryism than the Libdems so far. It all seems to be about applying mis-construable numbers for the main parties. Too confusing, therefore poll drums.

  35. Alec,

    I read that piece on tax havens earlier. In principle it should be damaging to the Tories, but in practice I can’t see it getting the airtime to register on many people – it’s not going to change the VI of many Guardian readers either.

  36. Alec
    ” I just hope he claims the Carers Allowance.”

    Poor grammar. Good understanding of who wields the power at UKPR.

    I’ll pass comment on your ability to understand polling basics on 8 May.

  37. Thanks to @Alec for a trip down memory lane

    I had many tussles with the “Dr” Eoin Clarke back in 2010. He didn’t really seem to have much grasp of reality and was totally unable to debate. An interesting character nonetheless.

    Meanwhile 21 days to go and I’m pretty sure each party is trying to lose. Either that or they aren’t that competent.

  38. Simon – I really don’t know how to break this to you but … UKIP are not going to win 46 seats (or whatever ridiculous number you came up with last night).

  39. Clegg refuses to rule out EU referendum deal.

    http://www.theguardian.com/politics/2015/apr/15/nick-clegg-refuses-to-veto-in-out-eu-referendum

    Shaping up to be Labour plus progressive alliance versus tory plus libdems,ukip and dup.

    But can clegg carry his party ?And what will paddy say ?

  40. @Reggieside

    one or two point movements in the respective tory/lab polling mean nothing. Its all margin of error stuff thats impossible to ascribe with any certainty to events such as manifesto launches.

    This is certainly true for the individual polls. However, the margins do come down with any kind of rolling average. I have just been re-running the Exponentially Weighted Moving average (EWMA) analysis of YouGov Lab/Con margins using the MoEs now agreed upon following the recent discussion between @Alan, @Charles, @Laszlo and various others.

    This analysis shows that Labour’s margin over the Tories fell within the MoE on Feb 24th. That is, in a statistical sense they lost their lead at that point because for quite a long period your couldn’t tell whether one party’s VI was ahead of the other’s.

    For a single day (March 10th) the Tories’ moving average was high enough to edge just beyond the MoE barrier. That is, the Tories nudged ahead for just one day.

    After that the weighted average stayed within MoE limits for little more than a month: the much discussed neck-and-neck period.

    But to respond to your post above, for yesterday and today the weighted average has again pushed Labour’s lead beyond the MoE.

    You can’t tell much from the individual polls treated separately. But YouGov haven’t reported a Tory lead for the last six polls and analyses like EWMA picks up that an unbalanced string of results like this is very unlikely to occur by chance and therefore classifies the margin as falling beyond the MoE.

    Bearing in mind the heading of this thread, this doesn’t mean that an equivalent of analysis of all polls would tell the same story. But on YouGov data Labour are now a nose ahead.

  41. INWB

    Eoin does have a PhD.

  42. I plan to enjoy the challengers debate. 5 knives being stuck into Cameron, and nobody to defend!

  43. NY ORKS Great place to be exiled, especially if you,re an ORK!
    I agree, such issues are out of the realm of the decision the voters are making on May 7. What the noise about it does is put pressure on the politicians after May 7, when the narrative might well turn to where the money is and when it might be coming home,to steal a phrase.

  44. Pointless nostalgia anecdote.

    Reading here the reports of UKPR posters’ experiences on the doorsteps, brings back memories of my own endeavours in S Africa, in the dark days of apartheid, My first ever political experience was in the 1970 GE. Fresh out of the army after compulsory military service, was to sign up aged 18 as a member of the Progressive Party – whose only MP was Helen Suzman and began canvassing, putting up posters, leaffeting and general dogsbody in a no hope constituency in which I could not even vote – my own district had no PP candidate – like 146 more of the 166 total Our most optimistic hope was to gain a 2nd MP, to give some support to Helen. I vividly remember my first ever election nght party, listening to the results, and crying bitterly as one after another showed that we had made good progress in many areas, but not enough to get that 2nd seat.

    In many more GE’s and by elections I repeated my efforts, but never ever in a seat with a realistic chance of winning. Initially, that was just because there were not any within (geographic) reach. Later, it was a point of principle – it was more important to me to take the message of liberal values and non- racialism to the unconverted, than to shore up support in what were becoming safe seats. By the 1987 and 1989 elections, I was campaign manager for the DP (successor to Suzman’s Progs) in “Helderkruin”, a large seat on the outskirts of Johannesburg, which we did not even contest just 10 years earlier – and made enough noise against a cabinet minister that the press began to tout it as possible surprise upset.

    We didn’t win – but just a few years later, we won the bigger prize – SA’s first democratic election.

    Along the way, in Helderkruin and earlier, I had to cope with a challenge that won’t have bother too many here (except possibly AS, in Canada) – attempting to conduct a doorstep or telephone canvass in a second language. This was a Nationalist stronghold, with mostly Afrikaans speaking voters.

    Years earlier, I’d done the same thing in a genuinely no-hope seat: a by-election in Johannesburg West, where we had never previously had a candidate, way back in 1972. I vividly recall my very poor attempts to conduct a canvass in Afrikaans, with students on the campus of the “Randse Afrikaanse Universiteit”. The winning Nat candidate, one Carel de Wet, was later SA Ambassador to London. I the circumstances, we thought our vote share of just 4% was a reasonable achievement. On the same day, we also contested for the first time a seat in Vereeniging, where we achieved 2%, against one FW de Klerk – later to be state president, and the man who finally faced up to reality, and prepared for a transition to multi – party democracy.

    My interest in psephology began way back with that first election in 1970. To this day, I can name the 19 constituencies in which the PP had candidates, and recall the approximate share of the vote we scored in each (in some cases, the majority to the exact number – Colin Eglin in Sea Point fell short by just 231). Now, I continue to obsess over the polls, projections, and results for elections here in the UK, the USA, and back home in SA.

    But I’m deeply grateful that I no longer have to take the results quite as seriously as I did in the past – when some years (1974 for instance) had me whooping out loud, and walking on air for days, but others (1970, and 1987) had me literally crying myself to sleep.

    And with the greatest respect to all those here, the issues here just don’t seem quite as important as the fight against apartheid.

    One challenge

  45. [b]Norbold[/b]

    Postal votes have already gone out, my mother received hers todsy

  46. MOG

    ” 5 knives being stuck into X, and nobody to defend!”

    Sounds like a normal BBC Question Time. :-)

  47. RE: RTB.

    There are very few housing association tenants in a position to buy their own homes. Its was never going to be any sort of game changer – its only puporse seems to be making some mood music.

    In the 70s large numbers of working people were living in council housing and many of them found RTB an attractive policy.

    Now that many of same demographic are struggling to pay the rent and have little hope of being able to afford to buy – these people would give their right arm for a council house – rather than the insecurity and higher rents of the private sector.
    Given this, I think boosting public housing and offering it to lower income working people would be a very popular policy.

  48. OLDNAT
    A PhD in being a tosser, as I’m sure his tutors would have to agree, otherwise he wouldn’t have got it!

  49. Norbold

    how do polls take account of the fact that a reasonable proportion of voters will have already voted by the time they produce the last week polls?
    ____________________________________

    Reading this

    http://www.inbrief.co.uk/media-law/publication-of-exit-polls-law.htm

    it seems they would have to take care not to publish any poll or make any predictions based on polling of people who have already made postal votes as that may be illegal.

    “The Times narrowly escaped legal action when they published an opinion poll in June 2004 on how people had voted in areas that had used all-postal ballots. Since the poll was published in the newspaper during the European Parliamentary elections that it referred to, the Electoral Commission (an independent watchdog on elections) alerted the Crown Prosecution Service.

    The Electoral Commission stated that publishing such an opinion poll amounted to publishing an exit poll before voting had closed. Nonetheless, the Crown Prosecution Service decided not to take any action against The Times, after discussions took place on the matter with the publishers”

    I looked at the Yougov polls from 2010 and the only clue I could see was this question

    The general election will be held on May 6. On a scale of 0
    (certain NOT to vote) to 10 (absolutely certain to vote),
    how likely are you to vote in the general election?

    where 10 meant
    Absolutely certain to vote, or have already voted.

  50. @unicorn.

    I understand that – and agree with your analysis. I was more arguing that a one or two point movement in the you gov poll between monday and tuesday doesn’t mean owt on its own.

1 9 10 11 12