Opinium have a new EU referendum poll in the Observer. The topline figures are REMAIN 43%, LEAVE 41%, Don’t know 14%… if you get the data from Opinium’s own site (the full tabs are here). If you read the reports of the poll on the Observer website however the topline figures have Leave three points ahead. What gives?

I’m not quite sure how the Observer ended up reporting the poll as it did, but the Opinium website is clear. Opinium have introduced a methodology change (incorporating some attitudinal weights) but have included what the figures would have been on their old methodology to allow people to see the change in the last fortnight. So their proper headline figures show a two point lead for Remain. However the methodology change improved Remain’s relative position by five points, so the poll actually reflects a significant move to leave since their poll a fortnight ago showing a four point lead for Remain. If the method had remained unchanged we’d be talking about a move from a four point remain lead to a three point leave lead; on top of the ICM and ORB polls last week that’s starting to look as if something may be afoot.

Looking in more detail at the methodology change, Opinium have added weights by people’s attitudes towards race and whether people identify as English, British or neither. These both correlate with how people will vote in the referendum and clearly do make a difference to the result. The difficulty comes with knowing what to weight them to – while there is reliable data from the British Electoral Study face to face poll, race in particular is an area where there is almost certain to be an interviewer effect (i.e. if there is a difference between answers in an online poll and a poll with an interviewer, you can’t be at all confident how much of the difference is sample and how much is interviewer efffect). That doesn’t mean you cannot or should not weight by it, most potential weights face obstacles of one sort or another, but it will be interesting to see how Opinium have dealt with the issue when they write more about it on Monday.

It also leaves us with an ever more varied picture in terms of polling. In the longer term this will be to the benefit of the industry – hopefully some polls of both modes will end up getting things about right, and other companies can learn from and adapt whatever works. Different companies will pioneer different innovations, the ones that fail will be abandoned and the ones that work copied. That said, in the shorter term it doesn’t really help us work out what the true picture is. That is, alas, the almost inevitable result of getting it wrong last year. The alternative (all the polls showing the same thing) would only be giving us false clarity, the picture would appear to be “clearer”… but that wouldn’t mean it wasn’t wrong.


274 Responses to “Understanding today’s Opinium poll”

1 2 3 6
  1. I agree that sticking to one method might offer false clarity, but at least the week on week changes might mean something. With the current approach of constant change, it is pretty much impossible to compare with previous polls.

    Anyway, there is no actually reliable information until polling day anyway, so the ‘benchmarking’ which occurs before then must be a little suspect?

  2. It does seem a rather odd headline from the figures: presumably that way the media could use the poll to report whichever headline it preferred.

    Will opinion polling organisations start giving actual forecast figures soon: i.e. ones that add up to 100%, as the real result must?

  3. Good morning all from Stevenage.

    “. If the method had remained unchanged we’d be talking about a move from a four point remain lead to a three point leave lead; on top of the ICM and ORB polls last week that’s starting to look as if something may be afoot”
    _______

    Very interesting and there does appear to be a shift towards Brexit even though the methodology change in this poll has disadvantaged Brexit on the top line VI.

    Also I find this interesting from The Guardian website.

    ” However, Opinium said a move to leave had also been reflected in answers to their so-called “nudge” question, which asks those who do not yet know how they will vote in what direction they are leaning. In the last Opinium survey two weeks ago, those split 55% leaning to remain and 32% leaning towards leave”

    “In the latest survey, the gap has narrowed dramatically, with 36% leaning towards remain and 33% towards leave, even when the methodological updates were implemented.”
    ________

    It’s still too close to call, however the direction does appear to be showing the real prospect that the UK is heading for Brexit and I don’t even think the leave campaign has gotten out of first gear yet which will probably depress the remain VI even more.

  4. “Generally the polls show an electorate split by social class, region and party political affiliation. The more affluent favour staying in the EU, while older people are typically more likely to back Brexit”

    _______

    I find this confusing. Most of the private wealth in the UK is held by the old yins, they save more, more likely to own their own home and so on and Tory voters are by in large more affluent than Labour voters (we’re always told the Tories favour the rich) but Labour voters are by 2-1 in favour of remaining in the EU.

  5. It is clear the polling companies don’t a have a clue which side will win and many will have egg on their faces in a few weeks.

  6. As I’ve said before, the most important issue with the polls being so close is going to be turn out, and with remain being the status quo, and leave the more motivated any lack of turn out is going to affect remain to a greater extent than leave.
    No wonder the Welsh windbag Kinnock was desperately attempting to motivate Labour supporters to vote. Not that he’s got a vested interest or anything !

  7. With the movement of people sadly dominating the debate it was a surprise to see, ahem, only eighteen neo n, er, people at an anti immigration do in Bristol.

    There were also hundreds of naked cyclists, which was fun.

  8. Guardian Online is reporting the poll at a 3 point Brexit lead:

    http://www.theguardian.com/politics/2016/jun/04/poll-eu-brexit-lead-opinium

    I dare say someone has pointed out the issues raised in the article above. Also: “Brexit or Remain lead – depends on which is the right polling methodology”, would be the best headline, but would not be snappy.

    As with the whole referendum debate – “Complex issues reduced to misleading soundbite”. Or: “Public not treated as capable of understanding complexity”.

  9. @ Thoughtful

    It’s worth looking at the likelihood to vote figures on pages 19-20 and the subsequent weightings by party on page 23. 92% of UKIP supporters describe themselves as ’10/10′ certain to vote, compared to 79% Con, and 74% Lab. Now all these figures are somewhat higher than the eventual likely turnout, but they are what is being used (per p23) to weight the respondents by party. So this produces the following sample:

    Con 34%
    Lab 30%
    UKIP 18%
    LibDem 6%

    The higher determination of Leave (or UKIP) supporters to vote is already factored into these figures. The question is whether they underestimate or overestimate that greater likelihood to vote. And if you look at the figures by party in detail on page 6, the UKIP vote is crucial, because all the other parties’ supporters (even those who identify as Tories) show a net majority in favour of Remain.

  10. By offering only English, British and other as the options for how people identify themselves, could Opinium be reducing the accuracy of any weighting based on this? After all, this was a poll across Scotland, Wales and Northern Ireland as well as England. Or is this an acknowledgement that the Brexit movement is largely an English nationalist one?

  11. I really hope the Remain campaign focuses on the NHS over the next couple of weeks. For the Outers to say more money will be pumped into the NHS if we leave is disingenuous. Doesn’t Gove want to privatise the NHS? Dont Johnson and IDS want patients to pay for their treatment?

    As things stand we may well be heading towards Brexit with our country moving ever further to the right.

  12. @Mikey: “I really hope the Remain campaign focuses on the NHS over the next couple of weeks. For the Outers to say more money will be pumped into the NHS if we leave is disingenuous. Doesn’t Gove want to privatise the NHS? Dont Johnson and IDS want patients to pay for their treatment?”

    This sort of issue is naturally tricky for Brexit. The whole point of Brexit is to give the UK freedom of choice – which means that a Corbynite government should be free to indulge in state aid without current EU restrictions, and that a UKIP government could impose immigration restrictions anathema to European law.

    Hence, all that Brexit can say on the NHS is that more money could be available – even if many Brexiters think that the NHS is a failing system. Brexiters would be on more comfortable ground if they said that more money would be required to train British doctors and nurses – as surely the whole point of Brexit is to rely less on importing stuff. (And importing foreign workers in preference to training your own, when able volunteers are available, is obvious short termism.)

    However, whatever is said on both sides, complex issues on which reasonable people can disagree will be reduced to overly simple soundbites.

  13. @James E
    The voting intention figures in the Opinium poll before rounding were actually –
    Con – 33.85%
    Lab – 30.38%

    Con lead – 3.47%
    Had the data been rounded by lead rather than % vote share the Tory lead would be shown as 3% rather than 4%!

  14. @Mikey
    “Dont Johnson and IDS want patients to pay for their treatment?”

    Working people already have to pay for prescriptions, spectacles, dentistry etc. It’s not such a massive change to pay for some bits of the NHS.

    The polling is interesting. Brexit seems to be slowly gaining ground.

  15. The Observer do make clear that on the new methodology Remain would be ahead.

    Allan Christie UKIP voters are the most working class and the most pro Leave

    Les Cunningham Opinium has Leave ahead in Wales, Remain with a small lead in England and a bigger lead in Scotland

  16. A real surprise Leave are ahead in Wales given the EU they receive. Under a right wing Tory Government do Welsh voters real!y believe they will receive additional funding to make up the shortfall?

  17. @ Mikey

    I really hope the Remain campaign focuses on the NHS over the next couple of weeks
    As someone who will probably vote to leave I hope so too, but tell me, why is someone purportedly on the Left in favour of the part privatisation of the NHS which the EU is forcing upon us?

    http://www.newstatesman.com/politics/2013/12/how-eu-making-nhs-privatisation-permanent

  18. Les Cunningham – “By offering only English, British and other as the options for how people identify themselves, could Opinium be reducing the accuracy of any weighting based on this? After all, this was a poll across Scotland, Wales and Northern Ireland as well as England”

    They have a “non-English” category which should capture Scotland, Wales and NI.

    Looking at the tabs the choices are
    More British
    Equal
    More English
    Non English
    Don’t know/none

    They also weight for region, to ensure there are enough Scots, Welsh and Norn Iron participants. So it looks like they are trying to get an understanding of the mass that is England (which will also have some people who identify as non-English – descendants of immigrants, or Scots, Welsh etc living and working in England.

  19. The other intriguing thing about yhis poll is the Plaid Cymru supporters.

    They are 14% Remain, 79% Leave, 7% don’t know.

    I googled to find out what Plaid’s official stance on the EuRef was, and they are for remain!

  20. Candy
    I wonder if Plaid voters resent being ruled by England, so they resent even more being ruled by a remoter government?

  21. @Thoughtful
    Yes, the New Statesman helpfully warned us about TTIP 2 and a half years ago. Whether it has the slightest validity now is another matter indeed.
    I did see the first piece of evidence that would point me towards Brexit today, Dan Hannan saying it would put him out of a job.
    Trouble is, he would merely transfer his egregious NHS rubbishing fully onto the UK stage to back up the inevitable fast tracking of TTIP after a Brexit which would be proof that the UK can do trade deals really fast (and continue the ratchet of neo-liberalism – anyone remember the ratchet of socialism?)

  22. @PeteB

    Or perhaps Wales understands that it’s economic interests lie with trading with the rest of the world, rather than the EU (they are in the wrong location for the EU)?

    Another intriguing thing is the certainty to vote:

    Definitely would vote was
    England 72%
    Scotland 81%
    Wales 75%

    The Scottish one looks high. In the Indy ref it hit 85% – but that referendum was about them and they were highly exercised about it. Are they that excited about the EU ref too?

  23. I’ve had a quick scan through the tables, and have noticed that Plaid voters are more concerned about immigration than any party supporters apart from UKIP.

    On a wider point I continue to be surprised that pollsters don’t ask whether a voter is registered or not (unless I’ve missed something).

  24. MIKEY

    A real surprise Leave are ahead in Wales given the EU they receive. Under a right wing Tory Government do Welsh voters real!y believe they will receive additional funding to make up the shortfall?

    You need to be careful with sub-samples. There were only 106 respondents in Wales and that’s before you consider that the samples are not weighted internally to Welsh totals.

    So any figure based on that sort of sample is bound to swing about a lot. It was 46% to 39% for Remain a fortnight ago, 42% to 35% for Leave a fortnight before that.

    The last EU referendum result in a ‘proper’ poll looks likely to have been in the YouGov in early April:

    https://d25d2506sfb94s.cloudfront.net/cumulus_uploads/document/1dtezrp6mr/WelshBarometerPoll_April11th2016_w.pdf#page=7

    (the more recent polls from them concentrated on the Assembly elections)

    when it was Leave 39%, Remain 38%. Though I would expect a new poll this week as Roger Scully is being cryptic on Twitter[1].

    On the subject of Welsh polls it would be interesting to see what happened if YouGov weighted on place of birth as they do for the Scottish polls.

    [1] “Anyone know any example of a political leader, on a 0-10 Like/Dislike scale, scoring as low as average of 2.1? Anywhere? Ever?”, which given that the Welsh Barometer polls ask exhaustive leadership questions about all the UK and Welsh Parties makes you suspect that Neil Hamilton may be involved.

  25. PETE B

    I’ve had a quick scan through the tables, and have noticed that Plaid voters are more concerned about immigration than any party supporters apart from UKIP.

    On a wider point I continue to be surprised that pollsters don’t ask whether a voter is registered or not (unless I’ve missed something).

    Small samples of course (see above), but Plaid supporters may be concerned about immigrants from England. Plaid voters may be terrified if these are no longer allowed to retire to France or Spain, they may all end up in Rhyl. One of the perils of these ‘most important’ questions is that different people read different (even opposite) things into them.

    Actually a lot more pollsters do seem to be asking about registration – ICM did in their recent online poll for example:

    https://www.icmunlimited.com/wp-content/uploads/2016/05/27-29-May-2016-CATI-and-online-polls.pdf#page=29

    they didn’t ask in their telephone poll but it could well have been one of their ‘screening’ questions which they don’t publish and if people said they weren’t they wouldn’t have been included.

    Pollsters who use regular panels such as Opinium or YouGov may already have the information on file and so not need to ask every time.

  26. Andrew Neil reporting an unnamed poll tomorrow showing another Leave lead, anyone know which pollster it might be?

  27. Roger
    Thanks for clearing that up about registration questions, and I loved this line –
    “Plaid voters may be terrified if these are no longer allowed to retire to France or Spain, they may all end up in Rhyl.”

  28. “If the method had remained unchanged we’d be talking about a move from a four point remain lead to a three point leave lead”
    Are not all these figures within the usual error range for such small samples?
    (1000 sample for an electorate of 50 million?)
    If so, why bother, when in a fortnight or so there will be a poll with a sample of 50%+?

    @RM
    “I’ve had a quick scan through the tables, and have noticed that Plaid voters are more concerned about immigration than any party supporters apart from UKIP”
    which may explain why UKIP did so well in in the MEP election in Wales.

  29. GUYMONDE

    Yes 1945-51 comes to mind.

  30. The tightening in the polls is not always reflected in ‘real life’. Last night at the cinema I was talking to an 83 year old daily Daily Mail reader who has become ‘fed up with the constant bile’ poured out in its Letters pages and in its articles. As a result he intends to vote Remain. He also cited the Leave campaign’s failure to give detailed figures (apart from immigration) on what is likely to happen to the economy and jobs. He said that the idea that they will spend an extra £100 million a week on the NHS is ‘a fantasy’. He talked in a quiet voice – almost as though he didn’t want anyone to hear that he was a Remainer!

  31. @ Thoughtful

    I don’t think TTIP will be enacted. At least one EU member will vote against it. My worry is that if we Brexit and the right wing Tories take control in Westminster it could happen. It’s right up Gove’s street.

  32. Dave

    “I’ve had a quick scan through the tables, and have noticed that Plaid voters are more concerned about immigration than any party supporters apart from UKIP”
    which may explain why UKIP did so well in in the MEP election in Wales.

    Actually the Plaid-voting sample is only based on 11 people (weighted to 9) so it’s completely meaningless. They mainly split for Leave (7-1-1) but the bigger Plaid sample (103), in the YouGov poll[1] I linked to, were actually the most pro-EU Party at 62% Remain, 28% Leave. So it’s just a tiny, atypical subsample.

    [1] As in Opinium this is the VI for Westminster (and there was no Lib Dem cross-break). EU support among Assembly voters was a bit lower which suggests while core Plaid voters are pro-EU, those who vote Labour UK but Plaid for Wales may be quite a bit less so.

  33. Good afternoon all from a very hot and sunny rural Hampshire.

    HYUFD
    .
    “Allan Christie UKIP voters are the most working class and the most pro Leave”
    “Les Cunningham Opinium has Leave ahead in Wales, Remain with a small lead in England and a bigger lead in Scotland”
    _________

    Yeah that’s true but when it comes to wealth then older people do tend to have more of it than the rest of the demographic so I think the assumption that richer people are more likely to vote remain is a bit fruitcake.

    I earn more than my grandparents pensions combined but they own their home outright, have far more in the way of savings and both had well paid jobs before retirement…both are voting BREXIT.

    I do however understand that a lot of less well off older yins will also be voting Brexit as will many working class but I don’t think you can simply apply a two but formula where if you’re wealthier then you will vote remain and working class you vote leave.

    Good to see Wales charging towards Brexit.

  34. My own gut feeling is that despite what is said if it is close it will not be regarded as decisive – on either side – indeed if it is a narrow leave decision – I cannot see how it will not be necessary to at least test matters with a Confidence Motion in the government.

    It is far from clear that the government would have a clear mandate to negotiate terms of departure with the EU. It would unprecedented for a government which wanted one thing to negotiate for exactly the opposite without some sort of Parliamentary mandate….

  35. JACKNICKLAUS
    “The tightening in the polls is not always reflected in ‘real life’. Last night at the cinema I was talking to an 83 year old daily Daily Mail reader who has become ‘fed up with the constant bile’ poured out in its Letters pages and in its articles. As a result he intends to vote Remain. He also cited the Leave campaign’s failure to give detailed figures (apart from immigration) on what is likely to happen to the economy and jobs. He said that the idea that they will spend an extra £100 million a week on the NHS is ‘a fantasy’. He talked in a quiet voice – almost as though he didn’t want anyone to hear that he was a Remainer”
    ______

    I don’t know if you’re posting this as a way of getting a point across without labeling yourself biased towards remain but giving you the benefit of the doubt surely we can’t rubbish all the opinion polls over what a 83 year old geriatric Daily Mail reader told you?

    And why did he feel that he didn’t want anyone else to know he was voting remain?
    I mean I was in Stevenage over the weekend and remain people there certainly know a thing or two about megaphone diplomacy

  36. BANTAMS
    “Andrew Neil reporting an unnamed poll tomorrow showing another Leave lead, anyone know which pollster it might be?”
    ______

    He’s not said anything on his twitter account and normally that’s where you can find such info treats.

    What he did tweet a few days back was this…

    Andrew Neil [email protected] Jun 2
    Cameron: at current rate of compliance year 3000 before Turkey joins EU. Few years ago he said he’s Turks biggest supporter for joining
    ____

    Oh dear!!

  37. “English, British and other”

    Erm….what about Welsh????

  38. @ Allan

    Smithson tweeted this earlier:

    “The @afneil effect on Betfair. He reveals that a poll tomorrow will have LEAVE ahead – REMAIN price drops a touch”

    Smithson is very much a remainer so I don’t think he’d make it up.

    C’mon Murray!

  39. BANTAMS

    That’s good enough for me.

  40. Mikey – “I don’t think TTIP will be enacted. At least one EU member will vote against it. My worry is that if we Brexit and the right wing Tories take control in Westminster it could happen. It’s right up Gove’s street.”

    But how would he get it through? Remember that the govt only has a majority of 12 – and already 25 Tory backbenchers forced the govt to back down and accept an amendment to the Queen’s speech regarding protections from TTIP. That was on 19th May – not that long ago.

    I picture you trembling behind your keyboard imaging that the Conservatives will gain superpowers on Brexit! Their majority will triple, nay quadruple! They’ll be able to push through anything they like even stuff not in their manifesto, never mind that just a few months ago the House of Lords (with a swing vote controlled by a massive tranche of LibDem peers) was happily chucking out anything not in the manifesto. The opposition in the House of Lords itself will shrivel away into nothingness, such are the magical powers of the Brexit Tories! :-)

  41. If Opinium are going to weight by attitudes to race and English identity, then they might at least have the decency to publish the crossbreaks by those factors.

    It would also have been interesting to see the proportion off respondents in Scotland, Northern Ireland and Wales who self-identified as English.

    A fairly shoddy set of tables!

  42. The phone polls this coming week will tell us more.

  43. There are rumours that the distribution of EU TINS will shortly commence. I’m sure that this will not be before the referendum however as the realisation that the EU is giving itself the power to centralise taxation would be enough to change the result.
    Or maybe not, because what government signs of policy knowing that within a few short days it might not have to actually enact it !

  44. I’m an outer and up to now thought it would be 55-45 for Remain.

    Now I think Leave are right back in the game.

    Strangely, the areas which haven’t seen lots of immigration are most in favour of leave.

    People, I think we maybe leaving Europe and heading for the open sea…

  45. @Thoughtful

    I don’t know much about EU TINS.

    From what I have gleaned searching the internet for a few minutes is that they sound like they could really help clamping down to tax evasion.

    Is the plan to have all EU States use them?

  46. CANDY
    @ Mikey

    “I picture you trembling behind your keyboard imaging that the Conservatives will gain superpowers on Brexit!”
    ___________

    Well that’s nothing compared to OLDNAT. He’s chucking a tartan tantrum with his kilt on behind his keyboard over Opinium polling tables and being a true Scotsman he doesn’t wear anything under I can assure you it ain’t a pretty sight. ;-)
    ……….
    JASPER22
    “I’m an outer”
    ___

    Fantastic decision, it’s not always easy coming out. ;-)

    I’ll get my coat.

  47. JASPER22
    “I’m an outer”

    Well I just checked my navel (first time in ages) and I’m definitely an ‘inner’.

  48. “Opinium have added weights by people’s attitudes towards race and whether people identify as English, British or neither. These both correlate with how people will vote in the referendum and clearly do make a difference to the result.”

    “Would it not be easier
    In that case for the government
    To dissolve the people
    And elect another? ”

    It would be ironic if the Europhile political class’ attempt to elect a new people was the cause of pushing the public into Brexit.

  49. CMJ
    “Is the plan to have all EU States use them?”

    Rumour has it that that will be announced on June 24th because they don’t want to scare the horses in the referendum. It is also rumoured to be the first step towards EU direct taxation of the populace. That’ll go down well.

    Rumours can affect VI, and hence polling. Perhaps sometimes more than facts.

  50. “The Scottish one looks high. In the Indy ref it hit 85% – but that referendum was about them and they were highly exercised about it. Are they that excited about the EU ref too?”

    With Scotland being more pro-EU Brexit should increase the chance of independence.

    So counter intuitively it might make more sense for SNP voters to vote Brexit.

1 2 3 6