This morning’s Mail on Sunday had a new poll of South Thanet which they built up into a UKIP “covering up” an unfavourable poll showing them headed for defeat. ComRes have subsequently released the tables for the poll here, revealing it was actually commissioned by ChartwellPolitical, an agency founded by two former UKIP staffers.
First let’s cut away the Mail’s hyperbole – it’s really not a “LOSER POLL!” and doesn’t show Farage heading for a humiliating defeat. UKIP are one point behind the Conservatives, with Labour one point behind them – CON 31%, UKIP 30%, LAB 29%, LDEM 5%. Given the margin of error, one couldn’t confidently say which of the three parties are ahead. What it actually shows is an extremely tight race. However it’s significantly less positive for UKIP than the previous polling in Thanet South – a Survation poll back in February that showed UKIP ten points ahead, and it resulted in UKIP attempts to rubbish the poll and its methodology last night.
In terms of methodology, the poll is mostly done using the same methods ComRes use in their national telephone polls – same turnout weighting and filtering, same squeeze question, same treatment of don’t knows. There are two important differences between the way ComRes do their national and constituency polls though. First respondents were prompted with the individual candidate names, secondly the poll was NOT politically weighted (if it had been, it would probably have been better for the Tories to some degree, depending on how much false recall ComRes allowed for in setting targets).
Most of the criticism of the poll last night (including some from UKIP themselves) was frankly complete nonsense. I can only assume a lot of it was sourced from “what some bloke on Twitter reckoned”. To sum up, the difference isn’t because candidates weren’t named – they were. It isn’t because 2010 political or turnout weights were used – they weren’t. It wasn’t because people who are unlikely to vote were included – they weren’t. It wasn’t because ComRes reallocated people by 2010 vote – they don’t. The idea that the question wording mentioning “your local MP” favoured Laura Sandys seems somewhat stretched, given the question included candidate names and Laura Sandys wasn’t one of those candidates.
To look at the more substantive things people have asked though, the initial voting intention question in the poll found Nigel Farage ahead. What put him behind in the final figures was weighting by turnout and squeezing the don’t knows. Neither of these are strange and unusual, they are ComRes’s normal method and are perfectly justifiable.
Looking at turnout first, ComRes found that Labour and Conservative voters said they were more likely to vote than UKIP voters. In Survation’s poll in February they actually found just the same thing, and their approach to weighting by likelihood to vote is very similar to ComRes’s (there is a difference in how they treat people who are very unlikely to vote – Survation weight them down very heavily, ComRes exclude them. In practice this difference has minimal effect). The difference between the Survation poll showing a ten point UKIP lead and the ComRes one showing a one point Tory lead is NOT turnout weighting.
The other difference is don’t knows. In the Survation poll people who said they didn’t know how they would vote were ignored. In ComRes, they were asked a follow up “squeeze” question – how would they vote if they legally HAD to. For people who still didn’t give an answer, ComRes asked if they identified with any party, and took that as their most likely vote. In practice these squeeze questions helped Labour and the Conservatives, but didn’t squeeze out much in the way of extra UKIP support.
There is nothing at all methodologically “wrong” with this poll… but then, there wasn’t anything “wrong” with the Survation poll in February either. There are different methodological approaches, and there are good arguments to be made for and against them, but we don’t have the evidence to say which is right. More importantly, a lot of the difference here isn’t because of methodology… it’s just because ComRes found fewer Ukippers and more Labour and Conservative voters than Survation did. Perhaps that’s because UKIP have lost support since February. Perhaps that’s just normal sample variation. We can’t tell, we can only say that South Thanet may still be a tight race after all.
