Two new polls out today, both good for Labour. Populus this morning had toplines of CON 31%, LAB 38%, LDEM 9%, UKIP 14% (tabs here). Lord Ashcroft’s weekly poll has topline figures of CON 27%, LAB 34%, LDEM 11%, UKIP 15% (tabs here).

The Ashcroft poll comes after a poll last week that showed the Conservatives 2 points ahead, and has naturally provoked some comment about volatility. In one sense it’s fair comment – Ashcroft polling has been volatile. In another sense it’s not – Ashcroft’s polling hasn’t necessarily been any more volatile than you should expect, it’s just that we sometimes have slightly unrealistic expectations of how accurate a poll of 1000 people should be!

The standard margin of error on a poll of 1000 people is plus or minus 3 points. However, voting intention figures aren’t based on the whole sample, only on those who give a voting intention – in a phone sample of 1000 that’s typically 500 or so people, giving a margin of error of plus or minus 4 points. I should add that the margin of error is based upon what the margin would be in a pure random sample. This is very much a polite fiction – no voting intention polls are actual pure random samples. Many are from internet panels, even quasi-random phone polls aren’t actually random because of low response rates. Weighting effects would also change the actual margin of error.

Looking at Ashcroft’s nine regular polls to date, the average level of Labour support has been 33%, and all nine polls have been within 2 points of this. The average Lib Dem support has been 8.5%, and all nine polls have been within 2.5% of this. What’s made them look erratic is the level of Tory support, which has averaged 29%, but has varied between 25% and 34% – two of Ashcroft’s Tory scores have differed from the average by 4 points, one by 5 points. This assumes that there hasn’t been any genuine movement in Tory support, when it’s possible there has. Ashcroft’s highest Tory score came in his first poll in mid-May, at a time when ICM also showed a Tory lead and YouGov a neck-and-neck. Ashcroft’s lowest Tory score came just after the European results when UKIP had a post-European election boost.

Bottom line is that while Ashcroft’s polls look erratic, they probably aren’t much more erratic than we should expect from topline figures based on 500 people. There isn’t anything strange about their methodology, nothing odd going on, it’s just the normal limits of how precise polling with a given sample size can be. And it’s a useful reminder of why we shouldn’t read too much into individual polls, and it’s the underlying trend and average that count.

Tonight’s YouGov poll for the Sun has topline figures of CON 35%, LAB 37%, LD 8%, UKIP 12%. Another two point lead, but it comes after a five point lead yesterday so it could still easily be margin of error.

There’s been a couple of other YouGov polls in the last day or two that I meant to post on but haven’t had the time. First there are the latest Welsh voting intentions for ITV and Roger Scully at Cardiff University. Topline figures there are:

Wales Westminster: CON 25%, LAB 41%, LDEM 5%, PLAID 11%, UKIP 14%
Welsh Assembly constituency: CON 21%, LAB 37%, LDEM 5%, PLAID 20%, UKIP 13%
Welsh Assembly regional: CON 21%, LAB 34%, LDEM 5%, PLAID 18%, UKIP 16%

If these figures were repeated at a Welsh Assembly election then Labour would remain just short of a majority on 29 seats, but UKIP would break through with 8 seats and the Lib Dems would be reduced to just 1. Note that the YouGov Welsh weightings have been updated for this poll (detailled here) so I haven’t done changes since last month.

Secondly there was a new Scottish referendum poll for the Times. Topline figures there were YES 35%(-1), NO 54%(+1). Without don’t knows, it becomes YES 39%(-1), NO 61%(+1). That means the last two YouGov Scottish polls have shown a slight movement to NO, but as it has been for the whole campaign, the movements are tiny and barely distinguishable from normal sample error. Other recent Scottish polls have shown movement in the other direction, so I’m still not convincted there is any real movement either way.

There has been some minor movement on the economic questions – by 49%(+4) to 27%(-3) people think Scotland would be worse off economically if it became independent, by 43%(+4) to 17%(-2) they think they personally would be worse off. The changes are since March, and suggests the economic argument may be moving away from the Yes campaign.

While we’re on the subject of Scottish polling, Peter Kellner had a lengthy article looking at a potential cause of the differences between polls here – specifically looking at the recalled Scottish European vote in polls following the European election and the different approaches to weighting by Holyrood past vote. It’s something I may return to in another post if I get time, but worth reading Peter’s take now (UPDATE: And Survation’s take here)

Time for a post-mortem of the European election polling. I’m not a fan of the sort of horse-race approach to these things – just because they are the final poll of the race, they still have normal margins of error, so if one pollster is a fraction more accurate than another it is often just the luck of the draw. Realistically the best a pollster can ever hope to do is get all the results within the margin of error. Better than that is luck. However away from the public “who won” stuff, comparing poll predictions to actual election results is an absolutely critical tool for pollsters – it’s our chance to compare our figures with reality, to improve and finesse our methods.

The final polls from each company are here. Note that I haven’t included Populus – they did conduct one European poll, but it was a fortnight before the election when just a week is a long time in politics! While I’ve included it in the comparison, one should allow some leeway for ICM for the same reason; their poll’s fieldwork finished a week before the actual election.

ACTUAL RESULT 23.9 25.4 6.9 27.5 7.9
YouGov 22 26 9 27 10 1.4
(-1.9) (+0.6) (+2.1) (-0.5) (+2.1)
ICM 26 29 7 25 6 2.0
(+2.1) (+3.6) (+0.1) (-2.5) (-1.9)
Opinium 21 25 6 32 6 2.1
(-2.9) (-0.4) (-0.9) (+4.5) (-1.9)
TNS 21 28 7 31 6 2.2
(-2.9) (+2.4) (+0.1) (+3.5) (-1.9)
ComRes 20 27 7 33 6 2.6
(-3.9) (+1.4) (+0.1) (+5.5) (-1.9)
Survation 23 27 9 32 4 2.6
(-0.9) (+1.4) (+2.1) (+4.5) (-3.9)

The most obvious current difference between Westminster polls is the reported levels of UKIP – there is a big gulf between the levels of UKIP support report recorded by companies like ICM, MORI, YouGov and ComRes’s phone polls and polls from newer companies like Opinium, Survation and ComRes’s online polls. We don’t know what the reasons for this are – there are a couple of things like prompting and re-allocating don’t knows that we can account for, but mostly the difference is not easily explained. It may be something to do with interviewer effect, or the representativeness of different companies samples. We can’t tell.

The European elections were obviously an opportunity to check figures against reality. I half expected the polls to all converge together in the run up to the election, as they have a tendency to do before general elections, but in reality we got the same sort of contrast as we do in Westminster polls. Higher figures for UKIP amongst newer online companies, lower figures from YouGov, lowest from ICM… and when the votes were counted the YouGov figure was the closest.

Of course, European elections aren’t general elections. On the issue of prompting, for example, every company prompted for UKIP in their European polling, whereas only Survation do it for general elections. There were no telephone polls for the European election, so it can tell us nothing of them. European elections are low turnout elections, so some of the errors may have been down to too strict turnout filters (ComRes used a very strict turnout filter for Euros and would probably have been better if they’d used the method they use for general election polling. There was the issue of the Independence from Europe spoiler party on the ballot paper and so on. At a purely personal level though, getting UKIP right at the next election is the biggest challenge currently facing pollsters, so I’m relieved that in the first real proper national test we got it right. Phew!

ComRes have released what they say is their final poll before the European elections on Thursday. Topline figures are CON 20%, LAB 27%, LDEM 7%, UKIP 33%, GRN 6%. The lead for UKIP is far more modest than the eleven points in their weekend poll, but is still pretty robust. The Conservatives remain in third place, the Lib Dems and Greens continue to battle for fourth place. Tabs here.

A lot of the apparent difference between different European polls is down to different treatment of turnout. ComRes have a very stark turnout filter, taking only those people who say they are 10/10 certain to vote. YouGov have a very liberal approach, including everyone who gives a voting intention (though they often tighten it up for final call polls); ICM weight by turnout, the effect of which is sort of a mid-way between the two extremes.

If you look at the tables, we can work out what the polls would have shown using different methods, letting us compare like-to-like. So, if all three pollsters who’ve reported in the last couple of days only took those respondents who said they were 10/10 certain to vote the figures would be:

ComRes – CON 20%, LAB 27%, LDEM 7%, UKIP 33%, GRN 6%
ICM – CON 21%, LAB 30%, LDEM 7%, UKIP 29%, OTH 14%
YouGov – CON 21%, LAB 26%, LDEM 8%, UKIP 29%, GRN 10%

But if all three respondents included the answers of all respondents giving an intention to vote the figures would be:

ComRes – CON 21%, LAB 30%, LDEM 8%, UKIP 27%, GRN 7%
ICM – CON 26%, LAB 29%, LDEM 7%, UKIP 24%, GRN 6%
YouGov – CON 23%, LAB 27%, LDEM 9%, UKIP 26%, GRN 9%

As you can see, it doesn’t explain all the difference (ICM. for example, would have Labour leading even with a strict turnout filter), but it does reduce the contrasts a bit. A softer turnout filter tends to help Labour, a strict one tends to help UKIP.

UPDATE: Two more Westminster voting intention polls tonight – the monthly telephone ComRes poll for the Indy has topline figures of CON 30%(nc), LAB 35%(-1), LDEM 8%(-1), UKIP 14%(+2). The daily YouGov poll for the Sun meanwhile has topline figures of CON 33%, LAB 37%, LDEM 9%, UKIP 11%. YouGov’s Sun poll also asked European voting intention, with topline European figures of CON 21%, LAB 28%, LDEM 10%, UKIP 24%, GRN 12%.

Lord Ashcroft this morning announced he was going to start publishing weekly voting intention polls in the run up the election. The polls are conducted by telephone so will be by far the most frequent phone polls (the ICM/Guardian, Ipsos MORI and ComRes/Indy series are all monthly efforts).

Topline voting intentions for the first on, conducted over the weekend, are CON 34%, LAB 32%, LDEM 9%, UKIP 15%.

I misread that to start with, and began to write a post about it being a useful addition to the polling world, but showing figures similar to everyone else… but then realised it wasn’t showing a 2 point Labour lead, it was showing a two point Tory lead. The first for a couple of years. As ever, unusual results demand closer scrutiny and currently we don’t know anything about it other than it was done by telephone – we don’t have a track record to compare, or any clue about the methods used. I’ll update later one I’ve some details to rake over…

UPDATE: Methodological details of the poll are as follows – the poll is past vote weighted (accounting for false recall, so the Tories are actually weighted slightly lower than than in 2010, Labour slightly higher), the voting intention question is prompted for the Conservatives, Labour and Lib Dems like most other companies. Results are weighted by likelihood to vote and a proportion of people who say don’t know are re-allocated to the party they voted for in 2010. Fieldwork was 50% landline, 50% mobile (weighted to be 15% mobile only households).

There is nothing here that should produce unduly pro-Conservative figures, in fact it’s broadly the same methods as Populus used to use in their telephone polls before they switched to online polling, and their polls were normally inline with other companies. What to make of the Tory lead then? Well, the poll seems methodologically sound, but it’s subject to the same margin or error as any poll, so treat it with the same caution you would if ICM or YouGov or MORI had popped up with a slim Tory lead. It might be a sign that the Tories have overtaken Labour, or might just be a an outlier, wait and see if it’s repeated it in other polls.