Time for a post-mortem of the European election polling. I’m not a fan of the sort of horse-race approach to these things – just because they are the final poll of the race, they still have normal margins of error, so if one pollster is a fraction more accurate than another it is often just the luck of the draw. Realistically the best a pollster can ever hope to do is get all the results within the margin of error. Better than that is luck. However away from the public “who won” stuff, comparing poll predictions to actual election results is an absolutely critical tool for pollsters – it’s our chance to compare our figures with reality, to improve and finesse our methods.

The final polls from each company are here. Note that I haven’t included Populus – they did conduct one European poll, but it was a fortnight before the election when just a week is a long time in politics! While I’ve included it in the comparison, one should allow some leeway for ICM for the same reason; their poll’s fieldwork finished a week before the actual election.

CON LAB LD UKIP GRN Average Error
ACTUAL RESULT 23.9 25.4 6.9 27.5 7.9
YouGov 22 26 9 27 10 1.4
(-1.9) (+0.6) (+2.1) (-0.5) (+2.1)
ICM 26 29 7 25 6 2.0
(+2.1) (+3.6) (+0.1) (-2.5) (-1.9)
Opinium 21 25 6 32 6 2.1
(-2.9) (-0.4) (-0.9) (+4.5) (-1.9)
TNS 21 28 7 31 6 2.2
(-2.9) (+2.4) (+0.1) (+3.5) (-1.9)
ComRes 20 27 7 33 6 2.6
(-3.9) (+1.4) (+0.1) (+5.5) (-1.9)
Survation 23 27 9 32 4 2.6
(-0.9) (+1.4) (+2.1) (+4.5) (-3.9)

The most obvious current difference between Westminster polls is the reported levels of UKIP – there is a big gulf between the levels of UKIP support report recorded by companies like ICM, MORI, YouGov and ComRes’s phone polls and polls from newer companies like Opinium, Survation and ComRes’s online polls. We don’t know what the reasons for this are – there are a couple of things like prompting and re-allocating don’t knows that we can account for, but mostly the difference is not easily explained. It may be something to do with interviewer effect, or the representativeness of different companies samples. We can’t tell.

The European elections were obviously an opportunity to check figures against reality. I half expected the polls to all converge together in the run up to the election, as they have a tendency to do before general elections, but in reality we got the same sort of contrast as we do in Westminster polls. Higher figures for UKIP amongst newer online companies, lower figures from YouGov, lowest from ICM… and when the votes were counted the YouGov figure was the closest.

Of course, European elections aren’t general elections. On the issue of prompting, for example, every company prompted for UKIP in their European polling, whereas only Survation do it for general elections. There were no telephone polls for the European election, so it can tell us nothing of them. European elections are low turnout elections, so some of the errors may have been down to too strict turnout filters (ComRes used a very strict turnout filter for Euros and would probably have been better if they’d used the method they use for general election polling. There was the issue of the Independence from Europe spoiler party on the ballot paper and so on. At a purely personal level though, getting UKIP right at the next election is the biggest challenge currently facing pollsters, so I’m relieved that in the first real proper national test we got it right. Phew!


440 Responses to “European polling post-mortem”

1 2 3 4 9
  1. Surely it is time for Nick Clegg to go?

    Some of you will know better than me, but I dont think there has ever been a case in history where a party in government has fallen to 5th place in a national popular vote, and joint 6th place in terms of seats won.

  2. Mind you in the short term, 1 or 2% might make a huge difference in the next election, I guess! But I think the long-term trends are more important and interesting.

  3. I get this feeling they’ll all prompt for UKIP from now on!!

  4. @Mr Nameless

    As someone who cares for the LDs, I really believe the grandees have to have a word with him. A party is not merely the preserve of an individual’s beliefs.

  5. I think they need to do so for his own sake, he looks like he’s about to crack.

  6. @ Anthony

    Yes, thanks – that’s fixed it. I didn’t want you to be hiding your light under a ‘blackout’ bushel!

    This EU election result is superb for YG; & it’s great to have some actuals which confirm that it’s YG’s panel that seems to best represent potential UKIP voters.

  7. Looked almost teary. I sort of felt sorry for him.

  8. Well done Anthony. It looks like you are a master of your craft.

    Nick Clegg needs a good holiday for his own health. He looks terrible.

  9. What good will it do for Clegg to go ? Clegg’s not the problem – it’s the party willingness to jump into the Tory bed that has caused the problem.

  10. AMBER.
    Thank you very much for recognising political prescience when it is found.

  11. @Bramley

    He is largely responsible for that. Him, Alexander and Laws.

  12. I do love the Labour spin machine, 2 elections both of which show Labour only 1% ahead of Conservative and Conservative sweeping everywhere, in the Euros Labour lost, or nearly lost everywhere apart from London where they did well, yet the Labour MP talks about how its a great success for Labour

  13. Surely with the Lib Dem leadership change it’s a matter of timing? Not so early that the new leader is out of his (presumably ‘his’) honeymoon period by the time of the GE, but not so late that he doesn’t have a chance to improve their ratings. Keeping Clegg is surely not an option … but then again, they miscalculated badly (and obviously) over coalition, so who knows.

  14. Anthony

    (Thanks for fixing the black lines in IE11 (Chrome was OK) before I even had the chance to complain about them).

    The real key here is about what the turnout filters and weighting that pollsters have used are – and indeed which ones they reported as their headline figures. To some extent how ‘good’ pollsters were had as much to do with what set of figures they decided to highlight as opposed to how well they did the usual functions of sampling and weighting samples.

    Normally turnout adjustment does make some difference but the problem is particularly acute with the Euros because of low turnout and UKIP and Green voters being more likely to be energised by both policy issues and the chance to get representation. Because of the potential low turnout some pollsters, such as ComRes decided to go for a strict ‘certain to vote’ filter (LTV=10).

    On paper this looked fine, with around 50% saying they vote for an expected turnout of 35%, but it probably underestimated two things. One was that some people will always only say they ‘probably’ will vote even though only being struck by lightning would stop them. I suspect this is particularly a problem with younger and/or women voters. If such voters are likely to vote differently from more definite ones, this can distort VI. Secondly online panels contains vastly disproportionate numbers of people who will vote and even 50% may be an underestimate. So ‘certain to vote’ can distort the outcome.

    It’s also worth pointing out that the main Parties will all be slightly over-estimated in polls. This is because the ‘long tail’ of small parties who get no EU representatives will not appear in polls much individually but will still collectively get quite a bit – about 5% this time. Pollsters try to assess them under ‘Other’ (and BNP in this case) but always underestimate, perhaps because a lot of their vote comes from last minute polling booth decisions.

  15. I meant UKIP sweeping nearly everywhere not Conservative

  16. A possible strategy for the LDs would be-

    (i) Struggle through under present arrangements until, say, late August.

    (ii) Clegg stands down.

    (iii) A new leader is elected and debuts at the conference.

    (iv) The new leader establishes themselves by moving out of the Coalition and into a confidence-and-supply arrangement, using some rationale or other.

  17. @maninthemiddle,

    It’s a rather insipid performance from Lab isn’t it, but, the way the cards are falling, I still think they will limp over the line in 2015. Not sure what the legislation outlook would be like if they are in Govt with a majority of say 10.

    rich

  18. @ AW

    I would just like to add my congratulations on the poll prediction. I do suspect that there were indeed a small percentage of Tory voters who having declared for UKIP had a very late change of mind. It would appear that some “shy” Tory voters are more willing to admit to voting for UKIP.

    The Green vote must be difficult to judge because IMO a great deal of tactical voting can affect their vote,

  19. Ed – only a minority blame immigrants solely. But it is a factor in the decline in services and quality of life. The addition of unprecedented population growth over 10 years couple with unprecendented low house building, along with lack of investment in transport, eductaion etc that have caused real problems.

    The middle classes may often benefit from that (eg rising house prices and ability to give children hefty deposits when buying) and this affects poorer members of society hugely. Not that immigration is the sole reason for rising house prices. It is one of many, but an important one.

    As for Clegg – looked like a hasty interview so probably not much time for make up. Seemed like he was babbling on with the usual dis-spiriting words. A man beyond learning it seems.

  20. @DAVE
    Have we got two DAVEs? I don’t remember asking about ICM. At 1.47pm I was eating lunch!

  21. I’ve been digging in the ITN Source archive and have found this interesting report from 1989 – the last time a major national political party was wound up.

    It sounds much the same: Spent the summer at 5% in the opinion polls, “Deifying of a few individuals”, not enough resources to finance a nationwide campaign, tiny membership with the attitude of a persecuted minority, the aim of the party being to “move the British government in a social democratic direction”.

    http://www.itnsource.com/en/shotlist/ITN/1989/09/25/BSP250989014/

    Now all we need is the Loonies to beat the Lib Dems in a by-election and the circle might be complete.

  22. AW How have you calculated your ‘average error’ please?

  23. MIM
    “I do love the Labour spin machine, 2 elections both of which show Labour only 1% ahead of Conservative and Conservative sweeping everywhere, in the Euros Labour lost, or nearly lost everywhere apart from London where they did well, yet the Labour MP talks about how its a great success for Labour”
    Before you indulge in hyperbole you should check your facts.
    Not one of the following councils returned a single Ukip candidate: Birmingham, Coventry, Exeter, Ipswich, Leeds, Liverpool, Manchester, Newcastle-upon-Tyne, Norwich, Preston, Southampton, Stockport, Sunderland. Ukip holds one seat in each of Bradford, Bristol and Wolverhampton, three in Plymouth and Sheffield and, here’s the major exception, six out of 42 in Portsmouth. (source Guardian letters)

  24. RAF
    “He is largely responsible for that. Him, Alexander and Laws.”

    I agree but removing him won’t change what they are enacting & that’s the real problem.

    In all likelihood, Alexander would put himself forward (is this why he’s given up specs in favour of contact lenses ?) & as far as I can see he is even more of an orange booker than Clegg !

    The vocal left-leaning LDs seem to clamour for Cable or Farron but if you were either of them with serious designs on the leadership, you’d wait until after 2015 when the opportunity exists to rebuild the party in line with it’s ethos, surely.

  25. Can I double check with regards lost deposits:

    http://www.independentnetwork.org.uk/research/independents-european-elections

    “A £5000 deposit is required to stand in European elections. This will be returned if a candidate obtains 2.5% of the votes cast in the entire region.”

    Does that translate as £5000 for each candidate from each party in each region (i.e. in Scotland, you put forward six candidates at £30,000), and all retain or lose the entire deposit if the total regional VI is above or below the 2.5%?

  26. Statgeek,

    I think the deposit is for your list of candidates. It’s up to you how many (up to the maximum) you stand, and the number of votes cast for the party determines whether the deposit is retained.

    That’s why it’s all the more astounding that a bus driver can find five grand to chuck away on the Roman Party.

  27. ” Euros Labour lost, or nearly lost everywhere apart from London where they did well”

    North East –
    Lab 36.5%, UKIP 29.2%
    Lead of 7.3%. Labour almost lost!

    Rounded, Lab’s result in the North East (37%) is equal to it’s result in London (37%).

    This also ignores the more nuanced view of the elections, where Labour won by large numbers in Birmingham, Oxford, Cambridge, Liverpool, Manchester and Norwich.

    (Which also means that they did much worse in places outside of these islands of Red in the sea of Blue/Purple).

    But don’t let the facts to spoil your (or Labour’s) narratives about the elections.

  28. Dave, it’s the average of the absolute values of the errors for each party, shown in the table

  29. YouGov top of the class… Well done.

    STV news picture from the count in Edinburgh.
    Spot the UKIP MEP?

    http://www.eveningtimes.co.uk/sites/default/files/imagecache/400xY/2012/10/19296005.JPG

  30. The LibDems will change nothing. Nick Clegg will remain leader & the coalition will continue until parliament is dissolved. Why? Because they firmly believe that there will be another hung parliament & – even with less than 30 seats – that they will still hold the balance of power.

  31. With the Yougov polls being based on a sample of 6000, I’d expect them to have a much lower standard error than a poll consisting of 1000-2000. If we converted the table into number of standard errors away from the result, we’d see that Yougov would be over 3 standard errors out on capturing the lib dem/green vote.

    While I applaud the effort in taking larger samples to produce polls, ultimately methodology determines the bias of a pollster and how good they are. I’m not sure there is statistical significance between the main pollsters here, which in a way is reassuring as it’s better to have multiple good pollsters than just a single one.

    One advantage of taking larger samples is that it differentiates errors due to systematic bias from sampling error, and better corrections to methodology. I’d have more confidence in saying that Yougov overestimated the Greens and Lib Dem vote that I would say that ICM overestimated labour support.

    While both statements may be true, the larger sample size and lower standard error allows one to see small biases better than a sample swamped by standard error. (With elections so few and far between they pretty much are a one off event as far as pollsters are concerned, and they are the only chance to spot any biases introduced through methodological choices).

    I guess it depends on the value of correcting a 1% bias when the standard errors are typically 3% for most polls.

  32. Actually if you look at the average error from the pollsters (last table) all were within the error of margin when combining all the parties together.

  33. I’m not sure why people are calling for Nick Clegg to go. Most people said they faced a wipe out but clearly this did not happen and they held onto one seat.

    A total triumph in the face of adversity.

  34. In 2009 the Tories in opposition polled 27.7% in the Euroelections but a year later they managed 37% in the General Election.. Applying that change to this year’s results would mean Labour getting 34.7% next May compared with 25.4% this year.

  35. An Independence from Europe took 1.5% of the vote. I think it is a fair assumption that if Mike Nattrass a dropped Ukip MEP, had not invented this spoiler party then Ukip would have gained at least another 1% of the national vote, taking them to 28.5%

    Ukip would also have gained another seat in the South West, at the expense of the Greens. I am sure that Mike Nattrass thinks that this was £5000 well spent.

  36. “I am sure that Mike Nattrass thinks that this was £5000 well spent.”
    Hell hath no fury like an ex-party member scorned?

  37. david

    I get this feeling they’ll all prompt for UKIP from now on!!

    Well they all did for the Euros anyway (I think). I’ve discussed this before and it doesn’t seem to make any difference for online polls – Survation who do prompt get similar results to Opinium and ComRes who don’t.

    It may be different for telephone polls. The new Ashcroft polls do seem to prompt for UKIP in their main list. They showed 15% and 14% in the first two polls (though they do use a reallocation that may go against UKIP a little). That again isn’t that different from the level we’ve seen in ICM and MORI, though it’s a small number of polls. There should be a ComRes this week it’ll be interesting to look at that. It may be that when UKIP’s profile is lower not mentioning them does suppress their phone VI though.

  38. And we also ought to mention that YG’s London polls have consistently shown Labour performing very well – which was borne out by both the local & euro elections.

  39. It’s amazing how accurate YouGove can be when they follow the obvious methodology of prompting for the most popular parties. Ukip’s most immediate challenge is targeting the many winnable seats in the GE.

    Nick Clegg’s Sheffield Hallam is Labour’s for the taking. In the council wards that cover the constituency the percentages are as follows;

    LD 33
    Lab 23
    Green 18
    Con 12
    Ukip 10
    TUSC 2
    Ind 2

  40. Here are the scores for UKPR, ranked by average error. I also made a note of who ranked the parties in the right order and who accurately guessed the gap between Labour and the Tories.

    I think I have everyone who gave a numerical prediction for the three main parties, but if I missed someone, let me know and I’ll add you in. Amber didn’t predict the Lib Dems or the Greens, so I assigned her the average errors for those parties.

    With an average error of 0.5, our clear winner is Andysco (or rather, Andysco’s wacky algorithm based on the 2009 election). He didn’t predict the Greens, but he got the other four almost spot on and in the right order: Ukip 26.4, Lab 26.2, Con 24, LD 6.9.

    The runners up:

    Thurrock Mark: 0.76, 2 point gap, had the Lib Dems and Greens tied
    Shev II: 0.86, correct order
    Allan Christie: 0.96, correct order
    T’Other Howard: 1.04, 1 point gap, had the Lib Dems and Greens tied
    Neil Turner: 1.04, had Labour and Tories switched
    Wes: 1.28, had Ukip, Labour and the Tories tied
    Gray: 1.44, correct order
    Raf:1.44, correct order
    Couper 2802: 1.56, had Ukip and Labour switched
    Postage Included: 1.6, had the Greens and Lib Dems switched
    Crossbat 11: 1.6, had Labour and Ukip switched, Lib Dems and Greens tied
    Spearmint: 1.64, had the Greens and Lib Dems switched
    James Baillie: 1.68, had Ukip and Labour tied and the Greens and Lib Dems tied
    Mr. Nameless: 1.8, had the Greens and Lib Dems switched
    AKMD: 1.84, correct order
    R. Huckle: 1.98, had the Greens and Lib Dems switched
    Roger H: 2.04, had Ukip and Labour tied
    Amber: 2.3, had Ukip and Labour switched
    TingedFringe: 2.36, 1 point gap, had the Greens and Lib Dems switched
    Welsh Borderer: 2.6, had Ukip and Labour tied and the Greens and Lib Dems tied
    Richard: 2.84, had Ukip in third place
    Charles Stuart: 3.36, 1 point gap, had the Greens and Lib Dems switched

    Congratulations Andysco, and actually, to all of us. On average we outperformed the pollsters!

  41. TInged Fringe

    I think its you who needs to relook at the facts, they are not similar at all yes in the NE Lab won by a 7% margin over UKIP, in London they won over UKIP by 20%, that’s a huge disparity between the 2 regions. Also London is the only English region where UKIP came 3rd, in every other English region or Wales UKIP won or came a strong 2nd.

  42. @AW: “amongst the long tail of others our final call poll did have AIFE on 1%!”

    Interesting and impressive. ‘Though I guess there was no less potential for confusion/misreading in looking at YouGov’s online list of parties than the actual ballot paper. So it might not be that surprising that AIFE picked up a similar proportion in both.

  43. killary45

    An Independence from Europe took 1.5% of the vote. I think it is a fair assumption that if Mike Nattrass a dropped Ukip MEP, had not invented this spoiler party then Ukip would have gained at least another 1% of the national vote, taking them to 28.5%

    I’m not sure that is true. It may be that people voted for AIfE because they wanted to make an anti-EU protest, but didn’t want to vote for that ‘nasty UKIP’ that had been in the papers so much. The fact that they managed to get 1% in the final YouGov poll suggests that it wasn’t just people confused in the polling booth, but a deliberate choice for most of those that picked them.

    Presumably Natrass and his friends had to find the £5000 deposit in all of the nine regions that AIfE stood in (you only get it back if you get over 2.5% of the vote), so it didn’t come cheap.

  44. SPEARMINT

    “Allan Christie: 0.96, correct order”

    WOW so I came in 4th and not only that I prediction was closer than any of the main pollsters.

    Ooooooooooooooooh!!

  45. #my

    Blame my Samsung s5

  46. @Anthony Wells – As you will know, that “average error” is a rather blunt instrument, and really you should be using a chi-squared test. Also, you missed out the Other category, where YouGov’s performance was worse than other pollsters’.

    A chi-squared test, including a category for Others, suggests that YouGov actually only had the fourth-best final prediction (?²=2.1), behind ICM (?²=1.6), TNS (?²=1.8) and Opinium (?²=2.0). ComRes (?²=2.5) and Survation (?²=4.8!) still performed worst.

  47. Damn, apparently your commenting system does not like the letter chi. :(

  48. @Hal

    I agree. Foreigners are blamed for everything from taking our jobs, to not taking our jobs but taking our dole, to causing our overcrowding.

    This is again unfair. However, it is true that immigration can cause problems for particular groups or in particular areas where schools, and other services can be overloaded.

    Personally I think Labour should say clearly that we can’t run our health services, support our unduly elderly population, meet certain skills gaps etc without immigration. They’re healthier than a lot of us and they pay taxes.

    However, it should also say that it is unfair that some people should have to pay for this through lower wages, overcrowded schools or whatever and that it is addressing this through the minimum wage, targeted support for particular areas etc.

    I am not sure whether this would be popular, but I don’t like the way in which politicians seem unable to say anything unless they have first run it through some focus group.

  49. Does anyone know if there is a handy map (or even just a tabulation) of the Euro results by council area anywhere online? Emily Maitliss had a nice interactive map of this on TV last night, but I can’t find it anywhere on the BBC website. I know this information is out there as I have seen local results in local papers, but it would be nice to see it all in one place!

  50. @Chris Green

    Test of html characters for Chi

    (upper) Χ
    (lower) χ
    (upper) Χ
    (lower) χ

1 2 3 4 9