Fun over, here’s the sensible grown-up look at how the final polls in the London race compared to the result. The final result and the final polls from each of the three pollsters who carried out regular polling in the race (ICM’s single poll was very early in the campaign) are below:
| CON | LAB | LD | |||
| Final Result | 1st May | 43 | 37 | 10 | |
| YouGov | 30th Apr | 43 | 36 | 13 | |
| Ipsos MORI | 24th Apr | 38 | 41 | 12 | |
| mruk | c. 24th Apr | 43 | 44 | 9 |
The accuracy of the polls was thrown into unusual prominence for a couple of reasons - firstly because, rather than the usual politican’s response to unfortunate polls (”I never look at polls” or “the only poll that counts is on polling day”), Ken Livingstone spent much of the campaign viciously attacking YouGov’s methodology, criticism that was picked up in some of the papers and which has resulted in a narrative in some quarters of the race not just being Johnson vs Livingstone, but also being YouGov vs MORI.
A second reason was the closeness of the race. In 1997 and 2001 some pollsters actually performed very poorly, but it didn’t receive much coverage because they all got the right winner. If the polls predict a comfy Labour win, and Labour do indeed win comfortably, no one is too fussed that the polls showed their lead to be rather comfyier than it actually was. Here we had one pollster showing Johnson comfortably ahead, while rival pollsters showed Livingstone very marginally ahead. When pollsters show different winners the difference suddenly becomes very apparent.
The result, of course, is that YouGov’s final poll was almost spot on with Johnson and Livingstone’s shares of the vote, though they overestimated the level of support for Brian Paddick and slightly underestimated the level of support for “others”. The reaction will be that YouGov have been vindicated.
Rather less positive will probably be the reaction to Ipsos MORI, who have been painted into the role of YouGov’s opponent. Their final poll was outside the margin of error for Livingstone and Johnson, but in fairness was conducted over a week before polling day. It is possible that there was a swing towards Boris Johnson in that final week and that MORI were not wrong at all - a poll is but a snapshot in time and there can be no guarantee that people won’t change their mind before they actually cast a vote. That is one possible explanation, but of course, it’s also possible that there are other reasons, dealing with turnout, a sample that is too Labour, a shy-Boris supporter effect or something else entirely.
I expect MORI will also point out that even they didn’t expect turnout as high as it was in their poll and that they published a paper a day or two before the election making predictions about how, with a lower level of turnout they would expect Boris to be marginally ahead. The question will be whether these perfectly good points will be enough to defend their reputation in the court of public (or in this case, commentariat) opinion.
UPDATE: Ben Page of Ipsos MORI has left the comment below: “At Ipsos MORI we will be reviewing our political polling methodology to look at lessons learned and publishing our conclusions and any changes we make in a few months’ time” (more below). In my opinion that’s the best thing to do. There is nothing to be gained from just blaming a result that looks wrong on a late swing, the right thing to do is to look in detail at the figures, try and work out what if anything did go wrong, and see how it can be put right.
















39 Responses
I TOLD YOU SO
would be a fair thing for YouGov to say right now
or in other words
SMUG TIME!
May 3rd, 2008 at 12:58 amCongratulations to Boris & YouGov.
May 3rd, 2008 at 6:59 amWell done guys, you were spot on. well done Boris - prove me wrong! Oh and I liked his speech. Good start.
What about that poll which put Boris 12 points ahead, was that the ICM one from two weeks ago?
May 3rd, 2008 at 7:04 amAntony
on a different subject BBC have said Labour finsihed behind the lib dem on 1st May. I wonder how much credibility this has
Labour won more council seats, more councils than the lib dems and was well ahead of them in London. Will yougov be producing an analysis of what the party breakdown was?
May 3rd, 2008 at 7:19 amCongratulations to everyone at Yougov - a great result you can be proud of.
At Ipsos MORI we will be reviewing our political polling methodology to look at lessons learned and publishing our conclusions and any changes we make in a few months’ time. As you say, our last poll was over a week before the election, and therefore not directly comparable, and subject to large margins of error, along with considerations of turnout effects etc. But it seems most probable to me personally that the telephone polls in the London mayoral campaign were over-estimating Ken Livingston’s share of the vote, rather than there simply being a late swing to Boris Johnson. Adjustments for turnout (and calculation of it) all need to be looked at, along with sample profile etc, and we will.
May 3rd, 2008 at 7:29 amhow did the evening standard come to the conclusion that johnson would win on 1st pref or was that bbc misreporting?
May 3rd, 2008 at 7:52 amBen: Yes but YouGov did a poll the day after your last which was 46:35:12. It was also clear from your published question that there was a framing effect. And Ipos MORI gives the impression of being the Left’s Favourite Pollster that gives the results the Left want to hear. Compared to the Retrospectives the average error of your 4 polls was an anti-C bias of 4, and a Std of 4 as well.
May 3rd, 2008 at 7:58 amIn response to that one, no pollster goes round doctoring their methodology to produce results to favour one side or the other as far as i know. I don’t think the Conservative leads we are producing in national polls are ones the government wants. I cut my teeth in research helping Michael Howard sell off the water industry in the 1980s and resent the idea that we would doctor research to produce what sponsors wanted. We are always resisting attempts to do that and lose clients as a result. It is a fact that we have always done a great deal of research for HM Government - whoever is in power. Under the previous Conservative government, HM Government was one of our biggest clients, as it has been over the last decade.
May 3rd, 2008 at 8:28 amIn response to NBeale, as someone who is on the Left I can’t see for the life of me why I or anyone of my persuasion would want to hear that my party is ahead, when in fact it’s not. That really would not be sensible politics at all.Only polls which tell us what’s actually happening rather than what the pollster thinks we’d like to hear are of use. When I said a few days ago that the truth was somewhere in between (that is, between the poll putting Johnson 11% ahead and the ones putting Livingstone slightly ahead) I seem to have been correct.
May 3rd, 2008 at 8:54 amNBeale - I don’t think that can explain it (if, that is, you are thinking of the preface to MORI’s question that mentioned that Ken Livingstone was the ncumbent).
It’s probably reasonable to think that something like that could skew a poll slightly, but in this case YouGov also prefaced their question with a statement saying that Ken Livingstone was the current mayor. The two companies worded it in different ways, but I think the effect of the wording they both used was probably much the same.
Matthew - BBC misreporting I think. I didn’t see the Standard, but Sky’s report said the Evening Standard has declared Boris the winner because he was so far ahead on 1st preferences there was no realistic chance of the second preferences being enough to overturn it.
Paul - YouGov won’t be, they are pollsters, not psephologists. Rallings and Thrasher however will do in time. They use a much larger sample of seats to make their calculations than the BBC do on the night, though they are usually quite close to each other. It’s important to remember that the shares of the vote published at local elections aren’t actually the total vote for each party - they are a “national equivalent vote”. This means it’s a notional figure based on what the BBC (or Rallings and Thrasher) think the result would be if ALL councils had elections that year. The actual vote totals (which Rallings and Thrasher will also calculate and publish) would show Labour doing much better because the elections were disproportionately in Metropolitan boroughs where Labour are strongest.
Ben - glad to hear that and will look forward to your findings.
May 3rd, 2008 at 9:02 amAnthony,
YouGov have again proved the quality of their polling methodology.
I bet we’ll still here the ‘…oh it’s YouGov’ line from Labour MPs though.
May 3rd, 2008 at 9:10 amWell done YouGov!
May 3rd, 2008 at 9:14 amThis probably means YouGov are the only poll worth having any faith in on a national scale too.
However, if even they have overstated LibDem support by 3% despite being the least generous of the polling organisations towards the LibDems, then Nick Clegg should be extremely worried about his prospects at the next general election.
May 3rd, 2008 at 9:23 amI do not think for one moment that polling companies (except the odd fringe one) now try to skew results, although it was not unknown in the past. There is no point to it and it destroys their reputation (That’s why MORI are being so upfront - very sensible). What some of them do (not by design) is give an impression of being alligned with a particular political leaning, and perhaps they all could look at that.
May 3rd, 2008 at 9:32 amAnthony
My congratulations as well on an excellent outcome for YouGov; extremely impressive given all the criticism that’s been floating around!
1. Do you think we can expect some local polls from anywhere in the run up to the Crewe and Nantwich bye-elections?
2. On a more specific polling point, I think it would be interesting if there was some ‘expert polling’ of economists regarding the right way to deal with the current downturn. I have seen some quite favourable reaction to Bush’s policy in the USA where a one off tax rebate is now being distributed and is expected to give consumers a boost and may reduce the severity of any recession over there. In my ramblings around the web, I’ve come across some speculation that Labour could do itself some good by temporarily loosening ‘the Golden Rule’ and reducing VAT by 2.5%, say, which would put a bit of extra money in people’s pockets as well as lowering inflation for the year following the reduction. I’ve also come across a revival of mutterings around 50% income tax on earnings above £150k-£200k as a way of paying for such a VAT cut; this would be seen, amongst other things, as putting some heart into wavering labour core voters. I suspect professional economists might find all sorts of fault with these fiscal changes but I could also see that they could act as a catalyst for a Labour revival?
May 3rd, 2008 at 9:43 amBarnaby, you don’t want to see biassed polls because you’re a healthy human being. Alas, the same doesn’t apply to many in various Westminster and Town Hall bunkers. Shooting the messenger is not a new thing.
Ken’s attack on YouGov and his formal complaint, questioning YouGov’s professionalism, was disgraceful. It was, unfortunately, yet another example of his true character and Ken now has the repost he deserves from the voters.
May 3rd, 2008 at 12:44 pmGreat performance from YouGov. It was good to see a relatively high turnout in London this time.
May 3rd, 2008 at 12:45 pmAnthony: I think the MORI question* has framing bias. But I can’t find the text of the YouGov question. Can you post it?
Ben: Did you always ask the question in this way, or did you rotate the order at random?
* In the next election for Mayor of London, the present Mayor, Ken Livingstone, is standing for re-election as the Labour Party Candidate. Boris Johnson is standing for the Conservatives, and Brian Paddick is standing for the Liberal Democrats, and there will be other candidates too.
In the election for Mayor, voters will have two votes, one for their first choice as Mayor and one for their second choice. If the election were held tomorrow, which candidate would be your first choice?
May 3rd, 2008 at 12:51 pmWell done at Yougov. I had great doubts about the Mori and mruk polls as they did not seem to agree with the groundswell of opinion which was heard in the whole London area.
I feel that the other pollsters have a great deal of work to do in order to restore their credibilty, perhaps one of them may sink back into the mruk.
May 3rd, 2008 at 1:53 pmLaughing loudly at Victor’s last line there.
As a daily reader from across the water (the big one, not Bonnie Charlie’s) I would just like to say that I find this site (along with two or three others) indispensable for UK politics. Interesting point about the “national equiv vote.” One learns something new every day….
And you know, I somehow can’t believe in a real shy-Boris factor. I deep down actually thought that Ken’s 8-year patronage machine was almost going to come through in the end. If anything I thought that machine would contribute to a shy-Ken factor. Instead it apparently produced a shy/lazy/not home Ken voter!
May 3rd, 2008 at 3:48 pmPolitics is going to be more fascinating and compulsive than its been for a long time. We now have Conservatives holding the reins of power for the first time in a generation. It has been so one-sided that it was difficult to compare and contrast. Political anoraks are in for an exciting two years. The polls will, quite rightly, come under even greater scrutiny. Watch this space!
May 3rd, 2008 at 3:49 pmAnthony,
May 3rd, 2008 at 4:07 pmI return to the discussion after a sabbatical hopeful that the more sober tone of the last few months will continue. Yes I know that the ‘Oracle’ refuses to moderate his partisan remarks but none of them are personal and perhaps if people ignore him for awhile he will get the message.
My congratulations must firstly go to YouGov for getting the result right in London and in the process establishing themselves as the number one opinion pollster in the UK-a position which pre-Ipsos MORI used to occupy in the 1970’s. I wish Ben Page well in his post mortem. Clearly there are things you need to put right Ben but not for one moment do I think that you produce polls your clients want to hear.Actually some Conservatives like some of your recent less favourable polls since it apparantly stops activists getting too complacant…
I would also like to congratulate my old sparring partner Mark Senior on correctly forecasting that the Lib Dems would by May 2008 be back where they were last May. He was right and I was wrong.
Well done to Councillor Peter Cairns for the way the SNP have-to my surprise and relief- effectively governed Scotland in the last year. I think that a lot of folk up here feel that they have shown up the awfulness of the previous administration.
At the time I left this site a number of Labour supporters were after my hide for daring to suggest that the Conservatives had a better than-I can’t remember the exact figure-15% chance of getting a majority at the next GE. I have no intention of gloating over the results of the council elections-that’s un British- but I repeat what I said before ‘ IT DOES’NT WEAKEN YOUR POLITICAL ARGUMENTS TO ADMIT THAT THE OTHER SIDE MIGHT WIN’ By definition a horse race assume more than one horse…..
And so to Crewe and Nantwich. Labour seem to have opted for a 5 furlong sprint here-but am I the only one who thinks that this may be bad tactics?
What a relief eh Anthony. The attacks on the company were disgraceful. I don’t remember Labour Mps and supporters giving you a hard time last September when you put them 11 points ahead.
I think its unfair to suggest MORI have backed Labour.
May 3rd, 2008 at 4:13 pmTheir methods do seem to overrepesent Labour’s share, but in this instance, they were pushed into a corner by others.
NBeale - I haven’t got it to hand, but it was very similar in it’s essentials. People were told that there was an election, that Ken Livingstone was the incumbent, and were then prompted with the three main candidates.
David - it’s unlikely, by-election polls are actually pretty rare things these days (and never had a good record of accuracy when they did happen)
May 3rd, 2008 at 4:28 pmIn answer to a question from Adam Boulton about “which Boris” we would see during the new Mayoralty, the New Mayor of London said :-
” I was elected as New Boris; I will govern as New Boris”
Classic & wonderful.
May 3rd, 2008 at 7:04 pmI bet Tony Blair smiled at that one-and Gordon didn’t!
Well for what it’s worth, i went for this.
“I think the actual result will end up with Boris between 54-56% to Ken’s 44-46%.”;
The final result seems to be 53% to 47% so it was just below the bottom of my prediction.
Peter.
May 3rd, 2008 at 11:18 pmHave recovered from Friday night…,
Anthony, I notice you say that Mr Paddick did less-well then YouGov suggested. Yet, in GLA elections, the LibDems and Greens held up.
A peripheral analysis would suggest that their difference between the polls voters seem to have split the mayoral vote between Boris and Ken. Then again, only Prof. Thrasher will be able to tell us that!
Kudos to Ben Page. Your “apology” was well-written. One question remains. Why did Ipos-Mori/UNISON polls differ so much from the Ipos-Mori/Observer poll which bisected them?
Peter Cairns, thanks for the SNP support…! Mike “The Oracle” Richardson, maybe it is time to hide the Conservatives’ strategy come the next election?
Big losers where UKIP. Maybe the E.D.P. should consume the English branches. Then get Andrew Constantine (F.E.P.) back to lead us in London. And then, 2012…?
May 4th, 2008 at 11:31 amFluffyThoughts, you’re right about the poor UKIP performance.
May 4th, 2008 at 8:31 pmI think most of their supporters must have tactically voted for Boris in order to turf out the outgoing Mayor who they considered to be a hardline communist.
No the UKIP vote went down relative to 2004 because UKIP always do well in European polls and any other votes (ie council or London) that coincide with EU elections. They are viewed as a one trick party which do not add much at local or London level.
May 5th, 2008 at 2:25 pmNick Keene.
I also have not returned to this site after the debate about what chance the Conservatives have of an overall majority. If I remember correctly, it was I who started this exchange by having the temerity to give my own personal opinion that this was around 15%.
I am in no sense a Labour supporter.
Then you took exception to this and went after MY ‘hide’, claiming that you couldn’t believe how I could say such a thing, and even suggesting my reputation was tarnished when I (or another commentator of the same name!) had predicted Labour would win the 1970 election (when I was 14!!).
I am not cognisant of whether any partisans subsequently joined in to criticise your right to have another opinion.
I would point out that Professor John Curtice (who is probably the most respected psephologist academically) said on Saturday that he had thought before last week’s results that he had believed the Conservatives had “no chance” of winning (I presume an overall majority), but now he believed it was possible. John Curtice is also to my knowledge not a Labour supporter.
In fact, these results, which were excellent for Cameron and the Conservatives and dire for Brown and Labour, do not in themselves say anything very much about the putative result of a general election in, say, 2010. It is still a mountain for the Tories to climb given the swing required and the bias in the system to Labour. They do clearly have a chance of an overall majority and a pretty good cance of being the largest party, and of forming a government. But I’m sure they will no be taking anthing for granted yet, and the May 2008 results do not clinch any argument that they will win an overall majority.
May 5th, 2008 at 4:24 pmI don’t think predictions have anything really do do with how strong or weak one’s own political argument is. I’m not interested in having a political argument!
I think the true underlying position of the parties is that the Tories are in better shape than Labour was in 1990 - but not where Labour was in 1995.
May 5th, 2008 at 5:59 pmI am encouraged by the gains in the north. I think this argument that we can’t take nearly enough of those seats may - may - be laid to rest.
May 5th, 2008 at 6:03 pmSome of us have added up the votes in marginals and there have been some pretty strong swings even since 2007.
Does anyone know what was the projected share of the vote for the three main parties in the 1990 and 1995 local elections?
May 5th, 2008 at 7:32 pmIn 1990 it was CON 33%, LAB 44%, LD 17%
May 5th, 2008 at 9:27 pmIn 1995 it was CON 25%, LAB 47%, LD 23%
Actually, I thought 1990 was only 40 for Labour - although reported at 42 on the overnight BBC program.
In 1995, they were saying 48, but it got reigned back a bit I think - 47.
I do think, though, it seems to be possible for a government to take an even worse hammering than in the 1990s and not in truth be quite as unpopular as government’s then.
Labour only got 29% in May 2000, below the supposedly fatal 31% for the Tories in the 1993 County Council elections - but perhaps those were worse because of the way the votes stacked up against them.
Anyway, whatever slight differences either way, I think the true position is the Tories are better placed than Labour was in 1990 - but we’re not in 1995 territory - it’s still more open than that.
May 5th, 2008 at 10:35 pmJoe James,
Are we treating electoral-cycles like-with-like. The current polls are heavily focused upon the metropolitan areas. I would hazard a guess that 1990 and 1995 were not such.
Point being Labour should poll highly in these elections (as Anthony mentioned). One fears for New Labour in the Euro-elections next year when the anti-EU vote turns out. Judicial review pending of-course!
May 6th, 2008 at 8:53 amJJB - Those are the Rallings and Thrasher national equivalent vote figures, not the BBC ones. They normally differ slightly.
Fluffy - the point of the national equivalent vote is to iron out the differences between the mainly metropolitan rounds and the mainly district rounds of local elections, so if they are doing their job they should be comparable.
May 6th, 2008 at 9:06 amRobert Waller
May 6th, 2008 at 9:56 amGreetings from Auld Reekie.Dinna craik.Bide Awee.Let the tow gang wi’ the bucket. In other words lets see what happens in Crewe and Nantwich…..
If the Conservatives win Crewe and Nantwich, especially against Tamsin Dunwoody, who will get a symapthy vote - “win it for the Gwypper”! - it will be a triumph and their best byelection result for about 30 years; which isn’t saying much, though.
May 6th, 2008 at 8:23 pmRemember all their hammerings before the 1983, 1987 and 1992 victories! Byelections are very odd animals, and say even less about subsequent general elections than May locals do, with their millions of votes all over the country.
More relvantly, my economic sources say it now looks like 2009 will be bad too, with some likely recovery in 2010. This may reduce Labour’s losses then - but it may also be too late for Brown personally.
A new Labour leader is likely to produce some honeymoon effect, at least in the short term (autumn 2007!, Major), but it will not necessarily save them.
I would finally point out that I what I do give a very reasonable chance to is a Cameron minority government. This could happen even if Labour remained narrowly the largest party, as they would then have ‘lost’, and the C would be on a charge. It’s always been the overall majority I’ve been so unsure about.