Most of the time on UK Polling Report I write about what the polls tell us about public opinion. Only when elections come along do we get to do it the other way around, and see what public opinion tells us about the accuracy of the polls.

As regular readers will know I have deep reservations about naive comparisons of eve-of-election polls and results and pollster “league tables”. They are often used in an extremely simplicistic fashion, with people balancing absurd conclusions upon one pollster being one point closer in a poll with a three point margin of error. One can get a slightly better idea in a well polled election race when consistent trends can be identified (as Rob Ford, Will Jennings et al did at the last election, producing a very different league table) but ultimately all a pollster can really hope for is to be within the margin of error of the result. All else is luck.

Five companies produced polls for the London mayoral election (TNS-BMRB also did one poll, but it was early in the campaign so can’t really be compared). Their final results are below.


First round . Second Round
Pollster Johnson Livingstone Jones Paddick Benita Webb Cortiglia Johnson Livingstone
ACTUAL 44 40 4 4 4 2 1 52 48
YouGov (30th Apr-2nd May) 43 38 3 7 4 4 1 53 47
Opinium (24th-30th Apr) 43 37 6 7 3 3 1 52 48
ComRes (23rd-25th Apr) 44 37 5 6 3 3 1 54 46
Populus (27th-29th Apr) 46 34 6 5 5 3 1 56 44
Survation (18th-24th Apr) 42 31 4 10 3 5 4 55 45

YouGov, Opinium and ComRes were all within 3% of all the candidates’ actual support, with YouGov slightly overestimating Brian Paddick’s support, and Opinium and ComRes slightly underestimating Ken’s. On the final round Opinium got the 52-48 split right, with YouGov calling 53-47 and ComRes 54-46. Well done to all three of them.

I’m unsure why Populus – who are normally one of the most accurate and reliable pollsters – ended up so out, showing a 12 point lead in the second round and significantly underestimating Livingstone in the first round. One thing that springs to mind is ethnicity. Since 2008 YouGov have weighted by ethnicity in London polls and it does makes a significant difference to results (Labour support ends up too low without it). There is no mention of ethnicity on Populus’s tables… but then again, neither is there on ComRes’s tabs. Perhaps Populus were just unlucky enough to get a dud sample. They also finished their fieldwork 4 days before the election, so perhaps there was movement towards Ken in those final days (the same applies even more to Survation, whose fieldwork ended 10 days before the election, so there was plenty of time for a swing).

Note that everyone overestimated UKIP’s support… although part of that could be their decision not to put UKIP in their description on the ballot paper (though they still used the UKIP logo, which says UKIP in it.)

Survation and YouGov also produced figures for the London Assembly, figures below.

London Assembly List vote
Pollster LAB CON Green LDEM UKIP BNP Others
ACTUAL 41 32 9 7 5 2 5
YouGov (30th Apr-2nd May) 42 32 7 9 8 1 2
Survation (18th-24th Apr) 33 28 8 10 7 3 11

126 Responses to “London polling post-mortem”

1 2 3
  1. First?

  2. Wren’t survation hopelessly out on Odham & saddleworth too? Or am I misremembering?

  3. Survation- OUCH!

    No posts on here from their employee lately…

  4. Nick P – they did fine in Feltham & Heston and Leicester South though, so have a pretty good by-election record since OE&S.

  5. To complete the panorama of 6 May elections in Europe, here are the results from Germany and Serbia:
    GERMANY, SCHLESWIG-HOLSTEIN STATE ELECTION
    (the number of seats decreased from 85 to 69, so OM is 35 instead of 43)
    CDU 30.8 (-0.7) 22 seats (-12)
    FDP 8,2 (-6.7) 6 seats (-8)
    TOTAL INCUMBENT C-R COALITION 39.0 (-7.4) 28 seats (-20)
    SPD 30.4 (+5.0) 22 seats (-3)
    Greens 13.2 (+0,8) 10 seats(-2)
    SSW (Danish minority, allies of SNP and PC in the European Free Alliance) 4.6 (+0.3) 3 seats (-1)
    TOTAL CENTER LEFT 48.2 (+6.1) 35 seats (-6_
    Pirates 8.2 (+6.4) 6 seats (+6)
    The Left 2.2 (-3.8) 0 seats (-6)
    Others 2.4 (-1.2) ) no seats.
    CDU-FDP coalition falls, but the result was not so catastrophic for the FDP as in other states. Possible coalitions: Center left Red+Green+ Danes, or if the Danes do not agree, Grand coalition CDU-SPD.
    Next electoral appointment next week in Nordrhein-Westfalen, where the actual Red+Green minority gvt. is expected to obtain OM at the expense of both c-right and the Left Party.
    SERBIAN GE
    Democr. Party (Soc. Dem) 22.3 (-16.1) 68 seats (-10)
    Socialists (former Commun.) 14.7 (+7.1) 45 s. (+25)
    United Regions (centrist) 5.4 (in 08 with DS) 16 (-8)
    TOTAL C-LEFT INCUMB. 42.4 (-9.0) 129 (+7)
    Progressive Party (nat.right) 24.0 (new party) 73 (+73)
    Radical Party (far right) 4.6 (-24.9) 0 seats (-78)
    Democrats of Serbia (right) 6.8 (-4.8) 20 seats (-10)
    TOTAL RIGHT OPP. 35.4 (-5.7) 93 seats (-15)
    Liberal Party (allies of LD in ELDR) 6.7 (+1.5) 20s. (+7)
    Minorities 3.6 (+0.5) 8 seats (+1)
    Others 11.9 (+7.3) no seats
    The center-left government, despite its decline in votes, obtained OM (129 out of 250 seats, instead of 122 in 2008), because of a kind of “Greek phenomenon” (many small parties failing to cross the threshold, which in Serbia is 5%), and will be able to govern without the support of the parties of ethnic minorities (which they will have anyway). Note the internal changes in the two alliances: Strengthening of Socialists at the expense of the more moderate DP and UR, and replacement of the Radicals by their more moderate and pro-European splinter, the Progressives.

  6. It look like YG were pretty close. Please please please increase the sample sizes for Scotland / Wales (or all regions), and get even closer on other polls. Pretty please!

  7. As Anthony has said, we were not commissioned to conduct a poll close to the election. The fieldwork begun on the 18th April – 15 days before the election.

    We correctly identified at that time in our consituency polling that senior Tories such as Brian Coleman in Barnet & Camden and Richard Barnes in Ealing & Hillingdon were at risk of losing their seats.

    We also predicted that smaller parties would do better than prior polling had found. Indicating that the Greens, Liberal Democrats and UKIP could gain seven assembly seats between them (UKIP missed by 0.5%). Our survey has also revealed more support for Siobhan Benita than previously suggested.

  8. YouGov’s Assembly poll, with similar fieldwork dates to our own (20th-22nd) (we were 18th -24th) forecasted a softer figure for the greens 6% vs our 8.1% and overestimated the Lib Dems. UKIP predictions were similar.

  9. Some discussion that the Government are going to kick both Lords Reform and reduction in constituency beyond the next election.

    That would be an amazing result for Labour! For that very reason, I can’t see it being true.

  10. http://www.dailymail.co.uk/news/article-2141040/Francois-Hollande-French-president-says-Britain-cares-City.html

    looks Like Hollande has already started his populist attacks on the UK.

    One thing I find interesting is he threatened EU wide taxes, and the financial transaction tax.

    I’m not up to date with EU treaties and what not, but I don’t think the EU has tax setting powers does it??? So is this threat an empty threat or is there an actual way this can happen without our consent?

  11. @Survation –

    Good on you for coming on here to discuss the polling.

    As you say, the figures for some of the minor parties were close to the final result. For the Lib Dems, your polling was similar to YouGov, suggesting their support may have fallen after your polling period.

    Any ideas why your figures for Con and, in particular, Lab were so far out, when YouGov got very close?

  12. @Max

    I wouldn’t believe everything you read in the Daily Mail! Accuracy in stories or headlines is not one of their strong points! However, on your specific question, all taxes at EU level have to be agreed unanimously by the Council of Ministers. Hence why one has never been introduced. Therefore it is an empty threat. However, it wouldn’t just be the UK voting against. I think a number of other EU Member States are against a financial transaction tax. Personally I support it. It is about time the banks paid their fair share for clearing up the mess they created!

    Anyway, on mayoral polling. Looks like all of us who said Populus was way out on Ken’s support were right. We weren’t blinded by partisan thought!

  13. STATGEEK
    It look like YG were pretty close. Please please please increase the sample sizes for Scotland / Wales (or all regions), and get even closer on other polls. Pretty please
    __________

    I’ll second that!! :)

  14. @Survation

    I notice you have only mentioned 3 figures from the YouGov poll with similar fieldwork dates.

    By my reckoning, even using these old figures, YouGov’s average error would have only been 2.1 compared to your 3.4 (and their fieldwork finished before yours!).

  15. Regular polling as from tonight again Anthony ?

  16. VIRGILIO

    Although the “socialists” in Serbia claim to follow democratic socialist principles they are in fact a far,far-right party, those who vote for them understand this and they are typical of a ex-communist party which becomes the party of choice for the nationalists.

    Serbia’s political system is convoluted and cannot be that easily sorted into the left-right political spectrum. Mostly they can be sorted into pro-european and those that hark back to the old days of Yugoslavia.

  17. @A W
    Congratulations. Mike Smithson kept telling us you are the business when it comes to London Mayors. He is clearly right.

  18. The results seems to indicate that their is still a danger in prompting for UKIP.

    Those polls that had them at 7 or 8% which lead to some here calling for them to be included as an option along side the LibDems could well have resulted in them getting poll results close to 10% twice what they actually achieved.

    Anthony,

    Do the results tell us anything about the relative merits of on line v phone poling in London, particularly with regards to ethnicity.

    Peter Cairns (SNP).

  19. Bluebob – “Regular polling as from tonight again Anthony ?”

    Yes indeed

    Peter Cairns – “Do the results tell us anything about the relative merits of on line v phone poling in London, particularly with regards to ethnicity.”

    No, you cannot isolate a single factor like that, or draw conclusions from a single phone poll. An online poll could weight by ethnicity, or not – it’s up to the pollster. Ditto for a phone poll. There’s no link with the mode.

  20. Peter Cairns – in terms of prompting,

    Opinium and YouGov only gave options for Boris, Ken & Brian, and other candidate and then only showed the other candidates if people ticked other.

    ComRes mentioned Boris, Ken & Brian only in the question, but I don’t know whether or not they gave options for all candidates on the first screen.

    Survation used a mock ballot paper, so had options for all the candidates on the first screen.

    I don’t know what prompting Populus used.

  21. VIRGILIO

    Thank you for that interesting detailed post.

  22. Could this explain Survation’s Mayoral outliers? They had massively higher BNP support than anyone else and (just) the top for UKIP, maybe these voters said that when prompted but in the voting booth just wanted to protest against the incumbent an voted for Ken. Though that doesn’t explain the Paddick score.

    Or it could be something else entirely. In any case, I’d argue that getting 4% for the BNP mayoral candidate is a fairly beefy argument against prompting for minor parties. That’s hardly within the MOW.

  23. In order to demonstrate we Tories can, like Boris enjoy a laugh at our own expense, (unlike the sons and daughters of Cromwell on the site), I will tell a joke.

    Why was the Re – Launch of the coalition in a tractor factory ? Because Cameron and Clegg could not organise an event in a brewery. Ha Ha Ha.

  24. @Roly1

    I thought they *were* trying to organise a p***-*p in a brewery…

  25. @Warofdreams

    Since our fieldwork completed – which showed 55-45% Boris (the same level as the TNS-BMRB poll that completed a couple of days before ours that is considered “incomparable as it was so early in the campaign” by Anthony above). You had further negative press revelations for the Conservatives from the Levenson Enquiry – Rupert Murdoch/Jeremy Hunt.

    A YouGov poll before election day dated 20th April put the Conservatives on 29%, their lowest rating for eight years.

    “A devastating YouGov poll today reveals that support for the Tories has dropped to 29%, the lowest level since 2004.” thundered The Times.”

    I would suggest that differential turnout (Tories staying at home) was the reason that Conservative popularity continued to fall during the run-up to the London elections and so naturally the polls closer to the actual election would be closer to the result.

    Anthony it would be helpful if you added the fieldwork dates to your comparison tables to illustrate your point.

  26. ANTHONY WELLS

    You did pretty good on the elections all in all I’d say. Congratulations!

  27. Anthony,

    Thanks for the reply. I knew that Yougov and others hadn’t prompted for UKIP, but was responding to those who had been asking for it because of their indicative support.

    The comment I made then was that I was opposed to prompting until there was god evidence that their actual result was consistently matching their poll scores.

    After these results I still feel that other than in a European election prompting fot UKIP would lead to over estimating their support.

    Oh and if my registration has worked this should come up in a nice SNP colour!

    Peter .

  28. That should of course be good evidence, god evidence is only required for TeaParty candidates!

    Peter.

  29. A YouGov poll before election day dated 30th April (not 20th) -3 days to polling had the Tories on their lowest figure since 2004.

  30. @Anthony Wells

    Congratulations (again – I said so earlier on a non London-specific thread).

    A couple of questions for you arising from the London polling.

    Firstly, YouGov had Boris/Ken tied on the 2nd ballot with the full sample, but that widened to a 6% gap when the decision was taken to filter only on the basis of those saying they were 100% certain to vote. A 6% difference due to turnout weighting alone is quite unusual and caused comment here at the time. Now it turns out that a less extreme filter, bringing in but weighting down the 90%, 80% respondents (etc.) would have got you even closer and possibly spot on with the final result. So my question is: what was the rationale for the 100% filter, and with hindsight might YouGov reconsider this approach for future elections?

    Second, the ethnicity filter is clearly quite critical in London, as ethnicity is by some way the strongest socio-demographic indicator in terms of distinguishing support between different parties. Given YouGov’s success in London, and the fact that non-white ethnicity is significant factor across the UK, albeit generally not quite to the same extent as in London, is there any reason why ethnicity is not also used as a weighting factor in other YouGov polls, and might you change this in future?

  31. @Virgilio

    Many thanks for the info on Germany and Serbia. Because of your efforts, UKPR has become a far better source of detailed information on overseas elections than the BBC or any other media website from the UK.

  32. I think the difference was down to political weighting and how people said they voted in 2010. The actual vote shares in London in 2010 were Conservative 35%, Labour 37%, Lib Dems 22%, Others 7%

    In ComRes it is Conservative 35%, Labour 36%, Lib dems 22%.

    h ttp://www.comres.co.uk/polls/Evening_Standard_London_Mayoral_VI_Apr25th.pdf

    YouGov don’t show the full figures on their tables, but if you assume others on 7% then from their crossbreaks it works out at Conservative 34%, Labour 36%, Lib Dem 23%

    h ttp://d25d2506sfb94s.cloudfront.net/cumulus_uploads/document/dom2smbrfs/YG-Archives-EveningStandard-MayoralElection-030512finalplustabs.pdf

    If you look at the Survation tables the crossbreaks for 2010 vote have got 532 Tory voters, 429 Labour voters and 340 Lib Dem voters… which if you assume 7% other voters works out at Conservative 38%, Labour 31%, Lib Dems on 24%

    h ttp://survation.com/wp-content/uploads/2012/04/London-Elections-Poll-Full2.pdf

  33. “One thing that springs to mind is ethnicity. Since 2008 YouGov have weighted by ethnicity in London polls and it does makes a significant difference to results (Labour support ends up too low without it). There is no mention of ethnicity on Populus’s tables… but then again, neither is there on ComRes’s tabs. Perhaps Populus were just unlucky enough to get a dud sample. They also finished their fieldwork 4 days before the election, so perhaps there was movement towards Ken in those final days (the same applies even more to Survation, whose fieldwork ended 10 days before the election, so there was plenty of time for a swing).”

    This happens in the U.S. and has explained why different national polls show neck and neck races between Romney and Obama and then small but solid leads for Obama. It’s usually due to weighting. This is why the midterm polls were dramatically off (though it didn’t matter all that much).

    “Note that everyone overestimated UKIP’s support… although part of that could be their decision not to put UKIP in their description on the ballot paper (though they still used the UKIP logo, which says UKIP in it.)”

    I would think that a UKIP candidate might gain more support in London by not being listed as a UKIP candidate.

    What’s interesting about the London Mayoral Election is that party identification and party line voting still play a big role in determining the electoral outcome but they’re not neccessarily decisive. There was a big swing to Labour in most of the London Assembly races (even in the races they lost). Yet Livingstone’s performance was only slightly better than his 2008 performance.

    I wonder too if polls in the London Mayoral Race get skewed because of party identification and the way the question is asked. Do pollsters list party affiliation after the name of each candidate or do they just list the candidate? I could imagine a poll asking people “Johnson or Livingstone” and most saying Johnson just out of name recognition, only to have a number show up at the polls and vote for Livingstone simply because they’re used to voting for Labour.

  34. Socali “There was a big swing to Labour in most of the London Assembly races (even in the races they lost)”

    In Bexley and Bromley, the only bit of Kent able to vote in London, and proud home to me, there was a swing of 0.2%, I believe Merton and Wandsworth even swung towards the Conservatives by 3.7%.

  35. @Phil

    The 29th April YouGov gave the 2nd round 52%-48% to Boris,
    and Boris/Ken 44%-41% in the first round, which I’m guessing was without the likely voter filter (and remarkably accurate).

    In 2008 the two final YouGov polls showed a narrowing of the lead, whereas their two final polls in 2012 seemed to show the lead widening.

  36. I suspect that the London Mayor contest may at some time be open to a celebrity Independent from neither of the big two parties.

    Oddly enough on the candidate issue I actually think Lord Sugar would have beat Boris running a “true Londoner” campaign.

    I suspect he would have got more Jewish votes than Ken, but then so would I.

    Anyone any suggestions of an Independent who could break through.

    Peter.

  37. @ Max

    “In Bexley and Bromley, the only bit of Kent able to vote in London”

    Interesting definition of Kent which includes Bexley & Bromley, but excludes Greenwich & Lewisham. I take it you favour the 1889 to 1965 boundaries?

  38. @Peter Cairns.

    “I suspect that the London Mayor contest may at some time be open to a celebrity Independent from neither of the big two parties.”

    Substitute “at some time” with “at all time”. This kind of race is pretty much all about personality politics, with party politics taking a firm back seat. It’s stark contrast to the local and general elections which are more to do with the parties than the PMs.

  39. I have added the fieldwork dates to the original post.

    Phil –

    I was locked away in the BBC, so don’t know why Peter decided to go with the strict filter in the end. It’s another reason I’m always a bit dubious about judging pollsters on the final results only… they often used different methodological tweaks for the all important final poll (for example, for the last couple of elections MORI only reallocate don’t knows in their final polls).

    In terms of ethnicity elsewhere, because it’s a much lower percentage across the country it has less impact (it also seems to fall out at about the right percentage anyway in our GB polls).

  40. Whilst this comment will create howls of anguish and disapproval in certain quarters, it most likely true. Not every Labour vote last week, whether in London or anywhere else, was the vote of a dyed in the wool, left leaning revolutionary. Many votes cast were deliberately aimed kicks in the buttocks of the coalition. I personally could not bring myself to to do it, any more than Mr Lampton or Nick P could vote Tory to shake, say Gordon Brown. However, many a less committed voter has little to lose by bringing a government they perceive to be not listening, up, by the short and curly hairs. These voters must be very hard for any pollster to judge correctly.

  41. @JAY BLANC
    Substitute “at some time” with “at all time”. This kind of race is pretty much all about personality politics, with party politics taking a firm back seat. It’s stark contrast to the local and general elections which are more to do with the parties than the PMs.

    I cannot help wondering if these words of wisdom would have been written by yourself if Kenneth had won ?
    Please don’t trouble to reply, I am sure you scrupulousness in judging British party politics is at its usual incorruptable level.

  42. “@ Max
    “In Bexley and Bromley, the only bit of Kent able to vote in London”
    Interesting definition of Kent which includes Bexley & Bromley, but excludes Greenwich & Lewisham. I take it you favour the 1889 to 1965 boundaries?”

    No, I just base it on Address, I have a Kent address as do many others in Bromley and Bexley, but the line stops somewhere in them, as there are people I know in Bexley, who have a South East London Address. So I doubt Kent address extend as far as Greenwich and Lewisham.

  43. JAYBLANC

    @”This kind of race is pretty much all about personality politics, with party politics taking a firm back seat.”

    Hmmm………or for an alternative view :-

    http://conservativehome.blogs.com/thecolumnists/2012/05/from-stephanshaxper-boris-didnt-win-because-hes-a-true-conservative-or-because-hes-a-clown-its-becau.html

  44. AW,
    Was the fieldwork for tonight’s YouGov poll yesterday and today?
    If so, whatever the result, could there be distortion caused by the bank holiday?

  45. Boundary changes likely to be postponed…Must be worth it just to keep Nadine Dorries stop talking about the posh boys

  46. SMUKESH

    @”Must be worth it just to keep Nadine Dorries stop talking about the posh boys”

    It would be stupid because :-

    * Reducing the cost of Parliament was a GE committment.
    * The new Boundaries were intended to remove some of the current pro Labour bias.
    *”Posh Boys” resonates-it is a problem which needs fronting up to & addressing-not ignoring..

  47. ………..on the other hand-Election promises are just a means to an end :-

    http://uk.reuters.com/article/2012/05/08/uk-france-spending-idUKBRE8470QD20120508

  48. Smukesh,
    I can’t see them postponing the boundary changes over Lords reform.
    One gives a massive benefit (by reducing pro-Labour bias) to the Tories, the only massive benefit to the LibDems.. and the only ones upset over both are a few MPs on both sides (on the Tory side, there’s overlap between the two).

    It seems silly to not go ahead with the reforms – especially if the Cons and Libs are looking at coalition post-2015 in the event of a hung parliament.

    I can’t find the exact figures, but if the boundary changes don’t go through, Labour only need a small lead to win a majority while the Tories need a massive lead. Hardly worth the risk.

  49. @COLIN
    `It would be stupid because :-
    * Reducing the cost of Parliament was a GE committment.
    * The new Boundaries were intended to remove some of the current pro Labour bias.
    *”Posh Boys” resonates-it is a problem which needs fronting up to & addressing-not ignoring`

    You have given several rational reasons for going ahead with boundary reform.
    1)But it will also disadvantage the Lib Dems given their VI as they showed better performance in local elections in seats with sitting MP`s…The advantage of incumbency would be neutralised.
    2)The Tories seem reluctant to consider Lord`s reform which is seen as quid pro-quo for boundary changes by Lib Dems
    3)If Tory unity is affected by boundary changes,then the party leadership is damaged so as to make an election victory even more difficult anyway.So far,they haven`t managed to effectively combat Nadine Dorries.The feeling is she speaks for many backbenchers.

1 2 3