This morning’s YouGov poll for the Sun had topline figures of CON 33%, LAB 38%, LDEM 11%, UKIP 11%. The five point lead follows two six point leads on Wednesday and Thursday, so we’re certainly looking at the lower end of YouGov’s recent spread of results. Full tabs are here.

There is also a new Populus poll following on from their return to regular voting intention polls at the start of the week – going forward they are expecting to publish at least one poll a week, sometimes more. Their latest figures are CON 31%(nc), LAB 39%(+1), LDEM 12%(-1), UKIP 10%(nc). Changes are from the start of the week and obviously don’t show any movement beyond normal margin of error. Full tabs are here


Ipsos MORI’s monthly poll for the Evening Standard is now out, and has topline figures of CON 29% (-2), LAB 40% (+5), LDEM 10% (nc), UKIP 12% (n/c). Whereas ICM showed a big swing back to the Conservatives on Monday, MORI show a big swing back to Labour today… I think it only serves to underline how polls are always at the mercy of normal random error, and how we really shouldn’t get too het up about individual polls. Watch the trends, watch the averages.

In the other questions MORI’s monthly tracker of consumer confidence found optimism continuing to grow – 35% now expect things to get better over the next year, compared to 30% who expect things to get worse. Full tabs are here.

Meanwhile the daily YouGov poll for the Sun had topline figures of CON 32%, LAB 38%, LDEM 10%, UKIP 12%. Full tabs are here


-->

Whenever two high profile polls come out at the same time I see lots of confused comments asking about who is right, or how come the polls are showing such different things. Today we have the well publicised ICM poll showing Labour and the Conservatives neck-and-neck and the regular YouGov/Sun poll giving Labour a nine point lead. On the face of it these are very different results. How come?

Well, there are two main causes of variation in polls. The first is the differences in methodology between polls from different companies – are they online or telephone, how do they sample, how they prompt, what things they weight by, what they weight to, how they deal with turnout, what they do with people who say don’t know? All these things add up and have a party partisan effect, so one pollster may consistently produce figures that are better for Labour, or worse for the Lib Dems or whatever.

The second cause is normal random sample error. It comes from the fact that samples for polls are, to some degree, random. The margin of error is normally quoted as plus or minus three points, meaning that 95% of the time, the figures in the poll will be within 3 percentage points of the “true” figure. Now, in many ways this is a polite fiction – the formula assumes a genuine random sample, which no polls are, and ignore lots of other factors – but I would regard the plus or minus three points as just a good rule of thumb.

If you look at leads rather than shares in polls it is easy to forget just how large that 3 point margin of error is – imagine the real situation in the country was that the Labour party was on 38% and the Conservatives on 30%, and that the pollster conducting the poll had a method that was perfectly accurate. You would still get polls that has Labour varying from 41% to 35% and the Conservatives varying between 27% and 33% – that is, leads of between 2 or 14 points!

Anyway, let’s come back to today’s polls. ICM have a ZERO point Labour lead, YouGov have a NINE point lead. Let’s dissect that and see where the difference comes from.

Looking at YouGov first, because they conduct daily polls we actually have a pretty good handle of YouGov’s random error. If levels of support are actually pretty steady, the day-to-day ups-and-downs of YouGov’s tracker will mostly be down to normal variation within the margin of error. For example, the average Conservative scores in YouGov polls so far this month is a little over 31%. Below is how each day’s Conservative figure has deviated from that score.

So in four polls it was bang on average, in five further days it was one point above or below, on two days it was 2 points above or below. This is all what we would expect to see. We’d get a similar chart for Labour.

On average YouGov’s polls this month have been showing a Labour lead of SEVEN points, so I expect the sample variation in today’s YouGov poll is a bit on the Labour side, and two points of the difference in leads between ICM and YouGov is sample error in the YouGov poll.

Now let’s move onto methodology effects. Earlier this month I produced the chart below showing the different house effects of the main polling companies – the partisan effects each company’s methods have on their topline results.

As you can see, on average ICM show Labour leads that are about 3 points lower than those shown by YouGov. That’s an average – it too will vary from month to month. The cause is largely down to two differences between the companies polls – ICM weight their data according to how likely people say they are to actually vote. They also make some estimates about how people who say don’t know will actually end up voting, assuming that 50% will end up backing the party they did back in 2010. YouGov don’t do either of these things.

If we look at ICM’s detailled tables here we can see the effect these two approaches had. ICM started out with CON 33%, LAB 38%. After weighting it by how likely people said they were to vote this moved to CON 35%, LAB 37%. After reallocating don’t knows it became CON 36%, LAB 36%. There will be other, less tangible differences in methodology, but crudely speaking another five points of the nine point difference came from the different methodologies of ICM and YouGov.

That leaves 2 points difference to account for, which is all likelihood is down to random error in the ICM poll. Whereas we get 20 odd YouGov polls a month, we only get one from ICM so we don’t have the data to average out all ICM’s polling data this month and see if this particular poll is a bit too Labour or bit too Conservative. But because we can’t measure it, doesn’t mean sample error isn’t still there.

In summary, about half the difference between ICM and YouGov’s polls towards is down to methodological differences, about half is down to normal random sample error (YouGov a bit more Lab than usual, ICM probably a bit more Tory than usual). Case closed.

Except, it probably isn’t. You’re probably still asking “Which poll is right?”, is the Labour lead zero or nine? If polls are all over the shop, what possible use are they? The answer is that one single poll taken in isolation isn’t that much use, the solution is not to take them in isolation, look at the big broad picture, the underlying trends, the averages over many polls.

And, for the record, Labour have probably got a lead of around about 7 points, like it says up at the top right of the page.


ICM’s monthly poll for the Guardian has topline figures of CON 36%, LAB 36%, LD 13%, UKIP 7%. Since last month it shows a slump in UKIP support and a recovery for the Conservatives. As ever, the more unusual a polls findings the more careful you should be – it could be a sign of UKIP support collapsing back towards the Conservatives, or could just be sample error. In YouGov’s polls over the last month we have seen some hints of a narrowing of the Lab lead and signs of UKIP support fading, and ICM do tend to show the lowest levels of UKIP support and relatively low Lab leads – all the same it looks quite odd. I would wait and see if it is repeated in other polls before getting too over-excited.


We haven’t had a Populus poll so far this year since their contract with the Times ended. Today they re-enter the field, but using an online methodology. Topline figures are CON 31%, LAB 38%, LDEM 13%, UKIP 10%.

Obviously there are no changes since their previous poll given the shift in method – which Populus themselves explain here. It’s worth reading in full if you’re interested in polling methods, but to give a quick summary of the changes, parallel testing by Populus found that they couldn’t produce comparable online results using past vote weighting, so they have moved from past vote weighting to party ID weighting (the approach currently used by YouGov). Populus have also dropped their reallocation of don’t knows, the old spiral of silence adjustment, on the basis that interviewer effect shouldn’t be a factor in online polls.

Full tabs are here.

UPDATE: We also have the monthly ICM poll for the Guardian due tonight (and MORI’s monthly poll for the Evening Standard is also due sometime soon, possibly tomorrow). I’m out and about tonight and will update with details of ICM when I get home.