I hope most of my regular readers would assume a Daily Express headline about a “poll” showing 80% of people want to leave the EU was nonsense anyway, but it’s a new year, a new election campaign, and it’s probably worth writing again about why these things are worthless and misleading as measures of public opinion. If nothing else, it will give people an explanation to point rather overexcited people on Twitter towards.

The Express headline is “80% want to quit the EU, Biggest poll in 40 years boosts Daily Express crusade”. This doesn’t actually refer to a sampled and weighted opinion poll, but to a campaign run by two Tory MPs (Peter Bone and Philip Hollobone) and a Tory candidate (Thomas Pursglove) consisting of them delivering their own ballot papers to houses in their constituencies. They apparently got about 14,000 responses, which is impressive as a campaigning exercise, but doesn’t suddenly make it a meaningful measure of public opinion.

Polls are meaningful only to the extent that they are representative of the wider public – if they contain the correct proportions of people of different ages, of men and women, of different social classes and incomes and from different parts of the country as the population as a whole then we hope they should also hold the same views of the population as a whole. Just getting a lot of people to take part does not in any way guarantee that the balance of people who end up taking the poll will be representative.

I expect lots of people who aren’t familiar with how polling works will see a claim like this, see that 14,000 took part, and think it must therefore be meaningful (in the same way, a naive criticism of polls is often that they only interview 1000 people). The best example of why this doesn’t work was the polling for the 1936 Presidential election in the USA, which heralded modern polling and tested big sample sizes to destruction. Back then the most well known poll was that done by a magazine, the Literary Digest. The Literary Digest too sent out ballot papers to as many people as it could – it sent them to its subscribers, to other subscription lists, to everyone in the phone directory, to everyone with a car, etc, etc. In 1936 it sent out 10 million ballot papers and received two point four million responses. Based on these replies, they confidently predicted that the Republican candidate Alf Landon would win the election. Meanwhile the then little known George Gallup interviewed just a few thousand people, but using proper demographic quotas to get a sample that was representative of the American public. Gallup’s data predicted a landslide win for the Democrat candidate Franklin D Roosevelt. Gallup was of course right, the Literary Digest embarrassingly wrong. The reason was that the Literary Digest’s huge sample of 2.4 million was drawn from the sort of people who had telephones, cars and magazine subscriptions and, in depression era America, these people voted Republican.

Coming back to the Express’s “poll”, a campaign about leaving Europe run by three Tory election candidates in the East Midlands is likely to largely be responded to by Conservative sympathisers with strong views about Europe, hence the result. Luckily we have lots of properly conducted polls that are sampled and weighted to be representative of whole British public and they consistently show a different picture. There are some differences between different companies – YouGov ask it a couple of time a month and find support for leaving the EU varying between 37% and 44%, Survation asked a couple of months ago and found support for leaving at 47%, Opinium have shown it as high as 48%. For those still entranced by large sample sizes, Lord Ashcroft did a poll of 20,000 people on the subject of Europe last year (strangely larger than the Express’s “largest poll for 40 years”!) and found people splitting down the middle 41% stay – 41% leave.

And that’s about where we are – there’s some difference between different pollsters, but the broad picture is that the British public are NOT overwhelmingly in favour of leaving the EU, they are pretty evenly divided over whether to stay in the European Union or not.


Time for a post-mortem of the European election polling. I’m not a fan of the sort of horse-race approach to these things – just because they are the final poll of the race, they still have normal margins of error, so if one pollster is a fraction more accurate than another it is often just the luck of the draw. Realistically the best a pollster can ever hope to do is get all the results within the margin of error. Better than that is luck. However away from the public “who won” stuff, comparing poll predictions to actual election results is an absolutely critical tool for pollsters – it’s our chance to compare our figures with reality, to improve and finesse our methods.

The final polls from each company are here. Note that I haven’t included Populus – they did conduct one European poll, but it was a fortnight before the election when just a week is a long time in politics! While I’ve included it in the comparison, one should allow some leeway for ICM for the same reason; their poll’s fieldwork finished a week before the actual election.

CON LAB LD UKIP GRN Average Error
ACTUAL RESULT 23.9 25.4 6.9 27.5 7.9
YouGov 22 26 9 27 10 1.4
(-1.9) (+0.6) (+2.1) (-0.5) (+2.1)
ICM 26 29 7 25 6 2.0
(+2.1) (+3.6) (+0.1) (-2.5) (-1.9)
Opinium 21 25 6 32 6 2.1
(-2.9) (-0.4) (-0.9) (+4.5) (-1.9)
TNS 21 28 7 31 6 2.2
(-2.9) (+2.4) (+0.1) (+3.5) (-1.9)
ComRes 20 27 7 33 6 2.6
(-3.9) (+1.4) (+0.1) (+5.5) (-1.9)
Survation 23 27 9 32 4 2.6
(-0.9) (+1.4) (+2.1) (+4.5) (-3.9)

The most obvious current difference between Westminster polls is the reported levels of UKIP – there is a big gulf between the levels of UKIP support report recorded by companies like ICM, MORI, YouGov and ComRes’s phone polls and polls from newer companies like Opinium, Survation and ComRes’s online polls. We don’t know what the reasons for this are – there are a couple of things like prompting and re-allocating don’t knows that we can account for, but mostly the difference is not easily explained. It may be something to do with interviewer effect, or the representativeness of different companies samples. We can’t tell.

The European elections were obviously an opportunity to check figures against reality. I half expected the polls to all converge together in the run up to the election, as they have a tendency to do before general elections, but in reality we got the same sort of contrast as we do in Westminster polls. Higher figures for UKIP amongst newer online companies, lower figures from YouGov, lowest from ICM… and when the votes were counted the YouGov figure was the closest.

Of course, European elections aren’t general elections. On the issue of prompting, for example, every company prompted for UKIP in their European polling, whereas only Survation do it for general elections. There were no telephone polls for the European election, so it can tell us nothing of them. European elections are low turnout elections, so some of the errors may have been down to too strict turnout filters (ComRes used a very strict turnout filter for Euros and would probably have been better if they’d used the method they use for general election polling. There was the issue of the Independence from Europe spoiler party on the ballot paper and so on. At a purely personal level though, getting UKIP right at the next election is the biggest challenge currently facing pollsters, so I’m relieved that in the first real proper national test we got it right. Phew!


-->

Looking at the search terms people are arriving at the site using there are already fewer people looking for candidate information, and more people searching for exit polls. Straight answer is that there are none, and there won’t be any.

These days the only real exit poll done is the BBC/ITV shared exit poll for general elections. They are extremely expensive and difficult to do, so they simply don’t get done for any other type of election (as Nick Moon of NOP, who along with MORI normally organise the general election exit poll, puts it they are an extremely expensive way of finding out something a couple of hours early).

It’s also illegal to publish any form of exit poll before the polls are all closed. For European elections that doesn’t just mean the polls need to be closed in Britain, they need to be closed across the whole of the European Union. This means it would be illegal to publish an exit poll before 10pm on Sunday (and given that the returning officers are allowed to start counting earlier in the day on Sunday, so long as they don’t announce the results until 10pm, any exit poll would be useful for even less time than usual!).

That means you’ll have to wait for proper results. For local councils, counting starts in about half the councils tonight, with results in the early hours of the morning. The other half will start counting tomorrow morning with results in the afternoon. For the European elections the counting of the votes can start during the day on Sunday, but actual results won’t be released until 10pm.


YouGov’s final European election poll is out tonight, conducted for the Sun and the Times. Topline figures are CON 22%, LAB 26%, LDEM 9%, UKIP 27%, GRN 10%. Like YouGov’s recent polls it shows a very tight race for first place between UKIP and Labour, the Conservatives in third place and a similarly tight race between the Greens and Liberal Democrats for fourth place.

A couple of methodology notes – the poll was weighted by likelihood to vote (so people who said they were 10/10 likely to vote or had already voted by post were given full weight, people 9/10 likely to vote weighted by 0.9, 8/10 weighted by 0.8 and so on) and respondents in each region were shown a list of all the parties standing in their own region.

The only other final call poll still due that I’m aware of is from Opinium – I’ll update when that arrives.

UPDATE: And here it is, Opinium’s final poll for the Daily Mail is CON 21%, LAB 25%, LD 6%, UKIP 32%, GRN 6%. Unless a surprise Populus or MORI poll pop up tomorrow that should be it for European election polling.


Survation have put our their final European election poll, conducted for the Daily Mirror. Topline figures are CON 23%, LAB 27%, LDEM 9%, UKIP 32%, GRN 4%. Survation made an additional tweak to their approach to turnout for their final poll, downweighting people who could not correctly identify which day of the week the European election was on – this marginally increased UKIP support and reduced Labour support. Tabs are here.

Meanwhile the daily YouGov poll for the Sun has topline figures of CON 33%, LAB 35%, LD 11%, UKIP 13%. For the European election Labour and UKIP are neck and neck going into the final day: CON 23%, LAB 27%, LD 10%, UKIP 27%, GRN 8%. YouGov’s final poll is still to come tomorrow.

ComRes marginals poll due in about 20 minutes…