ICM put out their weekly EU online poll today. Topline figures were REMAIN 45%, LEAVE 45%, DON’T KNOW 10% and tabs are here. ICM have tended to produce some of the most leave figures and the neck-and-neck result actually follows on from a series of polls showing a small leave lead, but this is due to a change in methodology.

As is often the case, ICM’s poll is actually less interesting than Martin Boon’s commentary that accompanies it – Martin’s response to the polling errors of last year has been one of the most candid and interesting of the pollsters, if occasionally one of the most pessimistic. In his article today he writes about what he considers to be the bleak future for phone polling given how people use the telephone these days, but also writes about the problems ICM have encountered moving most of their EU referendum polling to online.

Specifically Martin writes about how when ICM set online surveys live on Friday nights they get a rush of fast respondents that are skewed towards Leave. These entirely fill some of the demographic quotas set for the poll, meaning there is no room for the slower responding Remain respondents. To tackle this ICM have made two changes – one is to their sampling (they will spread it across the whole weekend, rather than opening it fully on Friday), the other is to weight respondents by how quickly they respond. According to Martin’s commentary the overall effect of this is to improve Remain’s relative position by four points.

I should point out (as Martin does in his article) that this is very much an issue to do with the way ICM carry out their online polls. Other online companies won’t necessarily do things the same way or face the same issues. I can only speak confidently about YouGov’s systems, but I know YouGov’s don’t invite respondents to specific surveys, they have a system that sends invites to respondents automatically, ensuring a constant flow of respondents into YouGov’s system. This means when people click on their invite (be it immediately, after a few hours, or days later) they are sent into whatever YouGov surveys are open at the time and need someone matching their demographics – hence when YouGov surveys go live they get a mixture of both fast respondents and slower respondents, who may have actually been invited before the survey was even written.

There was also a new contribution to the ongoing mystery of the gap between online and telephone polls on the EU referendum, this time from Prof Pat Sturgis (the Chair of the recent BPC/MRS inquiry into the polling error) and Prof Will Jennings (who served on the inquiry). It can be read in full here, but Pat and Will conclude that “While there are of course many caveats required here, this comparison suggests that the true picture may lie somewhere between the two modes, possibly somewhat closer to online. At the very least it suggests a good deal of caution is needed before concluding that one method is right and the other wrong. That will only be known for sure on June 24.”


439 Responses to “ICM – Remain and Leave level pegging, and dealing with fast respondents”

1 2 3 9
  1. Interesting comment that the true picture regarding EU voting intentions may be “somewhat closer to online”.

    Equally as interesting is the immediate caveat!

  2. “when ICM set online surveys live on Friday nights they get a rush of fast respondents that are skewed towards Leave.”

    This is potentially big. I still have the idea that there are groups of Leave supporters actively trying to skew the opinion polls to show Leave to be stronger than it really is, as a way of building a “feel” that victory is there for the taking. This Friday night skew is not proof of that, but it definitely supports the hypothesis.

    It’s the reason that I think Remain will outperform the final polling, even taking into account the status-quo-bonus seen in referenda like this. If the polling is static between now & polling day, I’ll be tempted to put some of my hard-earned money on Remain.

  3. “This Friday night skew is not proof of that, but it definitely supports the hypothesis.”
    Or, it reflects the keenness of leave supporters vs remain supporters, which will most likely be mirrored in their turnout at the actual poll. Opposite conclusion, and surely no less plausible than positing the existence some sort of polling conspiracy.

  4. @ Alun099

    You’d get very little return for your money from the bookies on Remain. Ladbrokes currently offer 1/6.

    Have to say that the idea that Remain led by 31 points per Phone polls in Summer 2015 is odd. There were very few Phone polls, but these did include one obvious outlier from Ipsos Mori poll with a 44 point Remain lead. Clearly this inflated the average. Without this, the BES survey would have been close to the Phone polls average.

  5. Evening All from lovely Bournemouth East.
    Could there be a lot of ‘shy’ Brexit people out there?

    I think REMAINERS ought to be worried

  6. The polls look absolutely solid for a REMAIN win from my reading of them. It’s just the online ones that are showing occasional leave advantages, but I don’t think those that fill in online polls are representative of the UK as a whole.

  7. There have been 15 polls so far in May, 9 online and 6 phone.

    These average out as a 4 point Remain lead, which would probably rise to 5 points when Northern Ireland is added. A significant number of those showing a Leave lead have been ICM Online which appear to have suffered from exactly the problem which Martin Boon had described.

    The idea that the tiny proportion of people who rush to complete online surveys is a reflection of the overall population’s likelihood to vote is bizarre.

  8. “Opposite conclusion, and surely no less plausible”

    Leaving aside the fact that I heard of plans some 6 months ago to do exactly what I described, your reading is equally plausible. However, given the prior information I’ve had, I choose to believe the hypothesis I described earlier :)

  9. Martin Boon makes a strong point about the problems with even pulling together a credible phone poll. It’s a painstaking process, sometimes with enourmous weightings to offset skewed samples. It really shouldn’t work. However, evidence from recent elections shows that it does work.

    It’s like using an incorrect calculation to get to the right answer. You’re not sure how it works. You know it really shouldn’t work. But it has worked.

    Online seems to be the reverse. It should be far easier to obtain a representative sample. This should make online more accurate. However, this really wasn’t so at the last GE.

    So here the calculation looks more accurate but the answer is wrong. You can’t believe it’s wrong. You know it really should work. But it hasn’t worked.

    QED – Phone polling is more accurate than online even though it should be less accurate.

  10. You Gov Poll just out.
    Level Pegging

  11. The Leave campaign does not have much going on these days. They seem to be relying mostly on the turn-out being in their favour and i reckon they aren’t entirely wrong there.

  12. tober542

    “Or, it reflects the keenness of leave supporters vs remain supporters”

    I think that’s probably true – but that might not work out quite as you might expect.

    In the ICM poll “Certain to vote” was Remain 66% : Leave 82%, but when you take 7-10 likely to vote it’s much the same – Remain 95% : Leave 96%.

    It was obvious at the indyref, that Yes voters were “keener” – turning out early in the morning to vote, but eventually outvoted by the less keen who wandered along later to vote No.

  13. “These entirely fill some of the demographic quotas set for the poll, meaning there is no room for the slower responding Remain respondents.”.

    Wow, just Wow! This is absolutely stunning! I’ve been following polls very closely in the USA and the UK for almost 20 years (think religiously following 538.com) and I’ve never heard of such a thing!

  14. I think these conspiracy theories are a little far fetched Alun009 and you seem to be mistaking one person you knew a while back saying they would look to skew the polls with a coordinated national strategy to do so which just does not exist.

    I think this Friday night “rush” for leave is probably driven by devotion and enthusiasm on the part of leavers and the demographics which suggest leavers are more likely to be free early on a Friday night…

    If there is an enthusiasm gap which this suggests then that could play well for leave at the ballot box but not likely enough to win

  15. The polls are all over the place. Over the past few days Remain leads have been substantial. Today’s polls are neck and neck. Frankly the polling companies don’t have a scoobie.

  16. The betting markets suggest that they ‘expect’ a Remain win by a little over 10%. Hence there a shorter odds for a Remain percentage of 55-60% than for 50-55% despite the fact that the polling evidence overall averages around Remain getting 53%-ish.

    It will come as a surprise to some here, but if you don’t think that that the outcome is going to be a 10 point win (or more) for Remain, it is possible to get effective odds of close to EVENS.

    To do this, you’d need to back both a Leave win at 4/1, AND a Remain vote of 50-55% at 9/4. With the correctly weighted amounts on these (e.g. £3.25 on Leave and £5 on the 50-55%) you’d be guaranteed to almost double your money.

  17. “I think these conspiracy theories are a little far fetched”

    I understand why you think that, and I’m really not trying to convince anybody of it when I have no direct evidence I can share. What I would say is that it doesn’t really need a coordinate national strategy. It’s easy enough to achieve this kind of thing with just a few people as long as they are time-rich and dedicated. The point is, the notion isn’t as outlandish as you think.

    That doesn’t make it true, of course, and I’m being careful to call this a hypothesis and one that today’s evidence does *not* prove. Given the fact that I’ve been careful and moderate about this idea, think again about being reflexively scornful of it, and instead ask yourself what would be needed to achieve such a skew? And what would the benefit be?

    I see an opportunity and a motive. I also see a gap between online and phone polls. There are multiple possible explanations for this gap, and we’ve explored some of these in recent weeks. I’ve given you another. Today’s evidence chimes a little with it. Admit it, it does. It also chimes with other explanations.

    One possible test for this would be to examine the responses against the date that panellists signed up. Of course, I’m assuming here that the online panels are ones where the people registered are self-selecting. If not, there’s a another test that could damage “my” hypothesis: if some online panels are by invitation only, then Id’ expect to see some differences in the responses, since these would be very difficult to swamp with proxy profiles.

    Perhaps someone in the know can enlighten us and put this idea to the sword?

  18. Alun009

    “Packing” of online panels was a favourite theme of a former poster here (Amber) for a pollster (Survation?) which produced results she didn’t like. :-)

    The only evidence she ever produced were allegations by her partisan pals about Facebook posts advocating such.

    The pollster tested the allegations (since Labour were still sufficiently strong in Scotland then to cause them concern) and demonstrated that their procedures were robust enough to avoid being conned by a relatively small partisan group.

    Pollsters may be inaccurate, incompetent etc etc – but they aren’t daft!

  19. Re previous discussion of the overcrowded south of England – and its limited survival period, this R4 adaptation of “The Kraken Wakes” seems more scientifically accurate version than Wyndham’s original – and well done Nicola Sturgeon!

    http://www.theguardian.com/media/2016/may/24/nicola-sturgeon-spreads-calm-after-alien-attack-in-bbc-radio-4-adaptation

  20. Scotland is apparently more pro-Remain than is England. It might be worth the Leave campaign promising a future referendum on Scottish independence if they win – on the grounds that a factor in the first one was that the UK was in the EU. As this would change under Leave, a new referendum would be required.

    It might sway a few votes in Scotland.

  21. Normally there is a herd mentality when it comes to voting, as I understand things anyway. People like to vote for a winner, so the reporting of polling is very important.

    In this instance though I wonder whether things might work differently? If in the week leading up to the referendum the polls and their coverage all point to a Remain victory, I wonder whether it might tempt a number of eurosceptic “Remainers” to use their vote as a sort of protest?

  22. Ben
    I agree with the herd theory, but the rest of it might be clutching at straws.

  23. Can anyone explain the purpose of “closing” a demographic cell in online polling?

    I can understand that where the methodology is resource-intensive as with face-to-face or phone polling, the loss of data may be felt less important than the resources saved. But how much are you realistically saving in an online poll by rejecting those potential responses? That such censoring may actually be informative only seems to compound what seems a strange idea to start with!

  24. A while back Roger Mexico demonstrated very large adjustments being made by some pollsters and I wondered then
    When do these adjustments become so large as to in effect invalidate the polls as anything useful?
    This is quite interesting
    https://inews.co.uk/opinion/comment/radical-way-improve-accuracy-british-polling/
    It would mean that polls were no longer snap shots of a point in time but quite large blobs
    They would no longer pick up short term reactions to specific events
    But would they prove more useful and accurate?

  25. @ James E

    You state that “The betting markets suggest that they ‘expect’ a Remain win by a little over 10%.”

    I agree, and expect Remain to win by 10-20%. The Leave campaign is disunited and all over the place in its messaging. The Remain campaign is plugging away at the economy and portraying the outers as lunatics.

    Therefore, your recommended betting strategy is likely to be a Lose-Lose.

  26. Pete B: I wondered though whether some people would be drawn to a protest vote if they thought nothing would change?

    I’ve found both campaigns to be pretty terrible and I also feel I am not remotely qualified to vote in this referendum. I might spoil ballot as I’m not sure we should even be having a say on this sort of thing. A referendum seems to be a snapshot of public opinion on a particular day, with the winner the side that has cut through most with its campaign of fear and propaganda.

  27. OLDNAT:
    Was anything published? I’d be keen to read the details of how they decided this wasn’t an issue.

  28. @James E

    “It will come as a surprise to some here, but if you don’t think that that the outcome is going to be a 10 point win (or more) for Remain, it is possible to get effective odds of close to EVENS.”

    My thoughts from the start were that Remain would win by around 10%. However, that was after the usual Referendum swingback to the status quo.

  29. Richard Dawkins has made me think. He is right-this Referendum shouldn’t be held. The topic is too complex for ordinary voters & we don’t have the information to form a reasoned opinion. This matter should be resolved in Parliament-where there is a political party offering the arguments in favour of Leaving, and numerous parties espousing Remaining.

    So I won’t vote in this Referendum because I am not able to answer the Question -and shouldn’t be asked to do so.

  30. Old Nat (from previous thread)

    Expats can still vote in the referendum and GEs up to living abroad for 15 years or less. The legal action you referred to related to expats who don’t get a vote as they have lived outside the uk for more than 15 years.

    I thought the Tories had it in their manifesto to do away with the 15 year limit and given that most expats, certainly those living in Europe, would vote in, one wonders why they haven’t.

  31. Colin
    I’m afraid I can’t agree with you there. I maybe could if I could believe that our politicians actually understood the issues themselves. They don’t any more than we do, which is why both sides are putting out so much rubbish in their campaigns. How many who voted for Maastricht or any other treaty, had actually read it and understood it? Very few, I suspect.

    There is no chance that Europe will modernise its operation, the democratic deficit will still be there, the corruption will continue and the people in power will continue to be failed politicians like the Kinnocks, jumping on the gravy train and becoming wealthy in the process. Junker himself has still not been brought to task for the illegal tax deals he agreed with multinational companies when premier of Luxembourg. At least Seb Blatter eventually got suspended whilst he was investigated.

    The U.K. Leaving will leave a massive hole in the EU budget and maybe will force some change upon it, including the CAP and the ludicrous situation of decamping to Strasbourg every 4 weeks. It may even lead to the break up of the eu as a political project. The U.K. Could then be in the vanguard of the reforming of a free trading area, which it what it should have remained all along.

    The more I think about it, the more I believe that out is the only option. Time to leave the kibbutz and live on our own estate.

  32. ROBERT

    My instincts lie with yours.

    But I think Dawkins is correct on the Referendum.

    If the EU implodes under the weight of its own inertia & incompetence, we will be “free” anyway :-)

    I see that Juncker’s Brussels instinct led him to advise the Austrians how to vote. These people actually have no idea of what they do & say -even as they throw petrol on the flames of the simmering discontent which they are stoking up.

  33. @Colin

    Depends what the argument is. There was a very interesting program on the BBC, where an apple grower pointed out that she would soon have to pay 9 / hour for labour while her Spanish counterpart paid 3. Not surprisingly she was considering her position. Is that an argument for leaving the EU? Or an argument against unrealistic minimum wages?

  34. Alun009

    I misremembered. The pollster being accused of allowing panel packing was Panelbase.

    There’s a statement on their site from 5/9/2013

    http://www.panelbase.com/media/Polls.aspx

    There was a bit of discussion on UKPR back at that time, and through to the referendum, so there may be further links to be tracked there.

  35. “Or an argument against unrealistic minimum wages?”

    A minimum wage must be a living wage in a country. Otherwise the activity is not suitable for that country.

    The floating rate currency should largely correct the discrepancy overall. If it doesn’t then there is some manipulation going on, and you have to use tariffs.

  36. ROBERT NEWARK

    I thought the Tories had it in their manifesto to do away with the 15 year limit and given that most expats, certainly those living in Europe, would vote in, one wonders why they haven’t.

    They did – they just haven’t got round to implementing it yet – legislation and adjusting systems take time. It’s Cameron who called the referendum so early (it was originally promised to be in 2017), so he’s only himself to blame.

    That said I wonder if there is really that great a desire for it. Only 196,000 applications have been received from expats:

    http://www.ft.com/intl/cms/s/0/e9e16b1a-1d18-11e6-8fa5-44094f6d9c46.html#axzz49fRZkNTZ

    despite extensive campaigns and some of those may not be valid. Although that’s up on the 116,000 who were able to vote at the last General Election, it’s not a great number out of over 46.2 million who could vote then – and this is a poll which you would expect expats to take a greater interest in than normal.

    In truth I suspect that most British citizens who have lived overseas for a long time feel they really aren’t morally entitled to have a vote in a country they no longer live or pay taxes in. They may in particular to feel little connection with the last constituency in which they happened to live before they left, though expats aren’t entitled to vote in local or devolved (eg Holyrood) elections. It’s not like France where there are MPs specifically elected to represent expats.

  37. Lucid Talk / NI Sun poll on EUref

    All intending to vote – Remain 57% : Leave 35% : DK 9%

    Irish Nationalists – Remain 80% : Leave 11% : DK 9%

    British Nationalists – Remain 18% : Leave 69% : DK 13%

  38. ON
    What’s a British Nationalist? SNP+Plaid+UKIP??

  39. @RogerMexico

    I think you are right in saying that some people don’t feel ‘morally entitled’ to vote. I too am surprised at the lack of applications to vote – as an expat living in Europe myself, I feel my and my wife’s whole future depends on this vote. Our ability to continue to work, live, access health care, etc hinges on this vote. It continues to be a time of great tension for us.

  40. Interesting article about how falls in battery prices might well drive electric car adoption, and as a consequence, maintain the oil surplus that has driven down prices, and provide a storage mechanism to smooth out power fluctuations from intermittent renewables.

    http://www.bloomberg.com/features/2016-ev-oil-crisis/

    “Battery prices fell 35 percent last year and are on a trajectory to make unsubsidized electric vehicles as affordable as their gasoline counterparts in the next six years, according to a new analysis of the electric-vehicle market by Bloomberg New Energy Finance (BNEF). That will be the start of a real mass-market liftoff for electric cars.”

    and…

    “Last year EV sales grew by about 60 percent worldwide. That’s an interesting number, because it’s also roughly the annual growth rate that Tesla forecasts for sales through 2020, and it’s the same growth rate that helped the Ford Model T cruise past the horse and buggy in the 1910s. For comparison, solar panels are following a similar curve at around 50 percent growth each year, while LED light-bulb sales are soaring by about 140 percent each year.

    Yesterday, on the first episode of Bloomberg’s new animated series Sooner Than You Think, we calculated the effect of continued 60 percent growth. We found that electric vehicles could displace oil demand of 2 million barrels a day as early as 2023. That would create a glut of oil equivalent to what triggered the 2014 oil crisis.

    Compound annual growth rates as high as 60 percent can’t hold up for long, so it’s a very aggressive forecast. BNEF takes a more methodical approach in its analysis today, breaking down electric vehicles to their component costs to forecast when prices will drop enough to lure the average car buyer. Using BNEF’s model, we’ll cross the oil-crash benchmark of 2 million barrels a few years later—in 2028.”

    And…

    “The good news is electricity is getting cleaner. Since 2013, the world has been adding more electricity-generating capacity from wind and solar than from coal, natural gas, and oil combined. Electric cars will reduce the cost of battery storage and help store intermittent sun and wind power. In the move toward a cleaner grid, electric vehicles and renewable power create a mutually beneficial circle of demand.”

  41. Democracy and sovereignty are very important factors in this referendum campaign. I wonder whether the apparent unconcern about these things in other parts of the EU is because they have much more limited experience of them?

    Most continental countries have been occupied by one or other of their neighbours numerous times, and a number of them have only been democracies for a relatively short term, historically speaking. I’m working from memory here, so please don’t pounce if I’m a year or two out, but for instance Germany wasn’t even united until 1870, and has had democracy only in the 1920s and since the war (and then only for part of the country until about 1990). Spain had Franco till well after the war, Greece had the 1970s junta, and so on. Our democracy is far from perfect but we’ve had it for a very long time and so my thesis is that it’s more important to the UK than elsewhere.

  42. Interesting that only in the case of South Thanet, do lawyers for the Tories seem to have opposed the police application to have the time extended to investigate alleged breach of electoral law.

    Only reason I can think of is that Tory lawyers were having a fun day at the seaside in Folkestone, and popped into the court to get out of the sun/rain (delete as applicable).

    What other reason could there be?

  43. PeteB

    I always find in very ironic that UKIP have so many more Euro MPs than Westminster MPs and still maintain that the EU is less democratic…

    Also I would turn your argument round and say that since Britain has not had a battle on its soil since 1746, we may not feel the need for compromise and neighbourliness as strongly as other EU citizens.. (although 50% of Austrians are perhaps not very neighbourly…)

    One of the main reasons I will vote Remain is actually I fear not so much for Britain as for a domino effect that may break up the EU. Whilst I do not think WW3 will immediately break out I do think the resulting instability will be very bad for all Europe, including Britain even if outside. Another reason is that I could easily have been in the position of Chris and have many friends who have taken advantage of freedom of movement over the years

  44. Wolf

    “Is that an argument for leaving the EU? Or an argument against unrealistic minimum wages?”

    It’s an argument for making sensible long-term decisions about massive productivity differences between certain economic tasks – especially seasonal ones – and not painting over the cracks of the problem with spectacularly stupid short term immigration policy.

  45. Andrew
    The point I was trying to make was that the EU is undemocratic, as demonstrated by the fact that the parliament has no real power.

    “One of the main reasons I will vote Remain is actually I fear not so much for Britain as for a domino effect that may break up the EU”

    Another good reason to vote Leave in my view. It’s interesting that the same fears, hopes or whatever have a different effect on different people.

  46. so it’s an argument for leaving and then hiteching parts of agriculture.

    #

    “I always find in very ironic that UKIP have so many more Euro MPs than Westminster MPs and still maintain that the EU is less democratic”

    The European parliament is a fake parliament intended to distract people from how the EU is actually run and how it was consciously and deliberately designed to be anti-democratic. So the mechanism for voting for a fake parliament being more democratic doesn’t outweigh the fact it is a fake parliament.

    The real reason for the EU is cheap labor; the rest is spin made up to con people.

    And as cheap labor combined with exploding housing costs is the CAUSE of the economic stagnation the EU – as an component of globalism – is partly the cause of the economic problems.

  47. @Somerjohn

    “UKPR is at its best when a genuine interechange of information occurs; at its worst when rancorous point-scoring takes over (all too frequently).”

    ————-

    Well, it’s very easy to avoid rancour when discussing the semantics of free trade vs single market etc.

    Most will leave you to it with such things, and won’t try and misrepresent etc. Renewable energy is another relatively safe bet, if you avoid countryside issues. Enter the fray on summat people care more about and it can be a bit different.

    Also, you ceded a point. That too, tends to help avoid rancour. But if you’d not been wrong and instead stuck to your guns instead, it mighta been different.

  48. @ Mr Jones

    ‘The real reason for the EU is cheap labor; the rest is spin made up to con people.’

    You remind me that Peter Mandelson when EU Commissioner was active in promoting the EU-India FTA which contains a Mode 4 proposal such that, Indian companies operating in Delhi and London could move low-paid workers from India to Britain, undercutting workers domestically.

    Effectively, onshore outsourcing, using cheaper temporary migrant labour and those transnational corporations would also be able to supply labour into other firms allowing them to offload all employer responsibilities under national or EU employment legislation.

    So much for the EU protecting British workers. (As far as I can establish the EU-India FTA is still not signed).

  49. @Wolf

    “Is that an argument for leaving the EU? Or an argument against unrealistic minimum wages?”

    ———

    Potentially an argument for many things. I’ll add a couple more…

    – it’s an argument for full employment policies

    – an argument for not trying to compete in summat where other countries have a competitive advantage

    – it’s an argument for adding value to the product instead

    – an argument for shifting production to Spain

    – some might see it as an argument for tariffs or summat…

  50. @Syzygy

    “You remind me that Peter Mandelson when EU Commissioner was active in promoting the EU-India FTA which contains a Mode 4 proposal such that, Indian companies operating in Delhi and London could move low-paid workers from India to Britain, undercutting workers domestically.”

    ————–

    How come peeps like Mandy never get outsourced or offshored or undercut? Seems a valuable opportunity is being missed…

1 2 3 9