Two types of outlier

This morning TNS released a new poll showing figures of CON 28%, LAB 35%, LDEM 6%, UKIP 18%, GRN 7% (tabs). The seven point Labour lead is striking, and out of line from the broader trend. Usual caveats apply, but for once I haven’t seen many people over-react to it. Most have sensibly enough seen it as the just the other side of the coin to the ICM poll earlier this week showing a solid Tory lead – two outliers in opposite directions. However, it is worth looking at the different reasons why these two polls went against the trend.

On average ICM produce figures that are slightly more Con/less Lab than average, but only by a tiny bit, generally they show Labour -v- Tory leads that are much the same as those from other companies. The reason that their poll on Monday showed a four point Tory lead when other companies are showing the race almost neck-and-neck is almost certainly just random sample error. Samples vary from day-to-day, month-to-month and sometimes you get one that’s a bit Laboury or a bit Conservativety in ways that weighting does not correct. I expect ICM’s next poll will be in line with those from other companies.

TNS is a different sort of outlier. The seven point Labour lead may be different from that shown by other pollsters, but its actually in line with TNS’s previous polls. In their previous four polls they showed Labour leads of 6 points, 0 points, 7 points and 7 points. Over on my chart of house effects TNS’s polls on average show a Labour lead three points larger than other companies (and if anything the gap is growing!) This isn’t random sample error, this is a consistent methodological difference between TNS and other companies, and unless they switch methods I expect their future polls will continue to show bigger Labour leads than other polls.

Two outliers, but one is probably just a random blip from a pollster who normally shows the same as other companies, one looks like a typical poll from a pollster who regularly produces polls that show a bigger Labour lead than the pack.

TNS Scottish poll

This morning TNS released a new Scottish poll. Topline Westminster voting intention figures are CON 16%, LAB 31%, LDEM 4%, SNP 41%, GRN 6%, UKIP 2% (tabs here). Under normal circumstances these would obviously be good figures for the SNP, but these are not normal circumstances and it’s a much smaller SNP lead than that suggested in recent polls by YouGov, Survation and Ipsos MORI.

Unlike their GB polls which are now done online, TNS’s Scottish polls are still done using face-to-face interviews. This means the fieldwork tends to take significantly longer, and the polls are then often not reported until a week or so later. The fieldwork for this poll was conducted between the 14th January and the 2nd February. This means the Survation and MORI polls from last month which showed 20 point and 28 point SNP leads for the SNP had fieldwork done at the same time as the start of this poll. The YouGov poll last week which had a 21 point SNP lead had fieldwork done at the same time the fieldwork for this poll was finishing (so is mostly significantly newer than this one!). What this means is that much of the reporting and headlines on this poll are just rubbish – the poll does NOT show the SNP lead falling. It shows a smaller SNP lead – this may well be for methodological reasons, or perhaps a bit of random sample variation, but given the respective timing of the fieldwork it cannot be that public opinion has changed since the previous poll showing a 21 point lead, as this poll was mostly conducted before that one. It’s a thoroughly bad idea to try and draw trends between polls conducted using very different methods anyway, but certainly check when the fieldwork was done and get them in the right chronological order.


Today’s daily YouGov poll for the Sun has topline figures of CON 34%, LAB 33%, LDEM 6%, UKIP 13%, GRN 7%. This is the first Tory lead from YouGov this week – their underlying average has the two main parties very close to each other, so we should expect normal random sample error to regularly produce both Labour and Conservative leads until and unless one party opens up a proper lead.

Also out in the early hours of this morning was a new TNS poll which has topline figures of CON 27%, LAB 33%, LDEM 6%, UKIP 18%, GRN 8%. It looks as if I may have rather jumped the gun when I said the methodology change in TNS’s last poll had brought their figures more in line with other companies, as this one seems to be back to their traditional pattern of showing a significantly larger Labour lead than other companies.

A quick update on the latest voting intention polls. This morning’s YouGov poll for the Sun has topline figures of CON 33%, LAB 34%, LDEM 6%, UKIP 14%, GRN 8% (tabs) – more typical figures than YouGov’s Tuesday poll.

There is also a new TNS poll with topline figures of CON 31%(+3), LAB 31%(-4), LDEM 8%(+2), UKIP 16%(-2), GRN 7%(+2) (tabs). Up until now TNS have tended to release their polls incredibly slowly, often a week or more after their fieldwork was finished, rendering them out of date by the time we see them. Fieldwork for this one however finished on Monday so it’s more timely than usual. The last couple of TNS polls showed Labour leads of 7 points, significantly and consistently larger than any other company. Today’s looks more like other polls. (UPDATE – thanks to those in the comments section who missed something obvious in the TNS tabs that I overlooked! TNS have always weighted by 2010 recalled vote, but they are now weighting by 2010 recalled vote AND 2014 European vote – whether or not this is responsible for their figures coming into line with other companies’s is unclear.)

Finally Ipsos MORI put out their latest Scottish poll yesterday. Westminster voting intentions in Scotland were CON 12%(+2), LAB 24%(+1), LDEM 4%(-2), SNP 52%(nc), GRN 4%(-2). We’ve now had three Scottish polls conducted in 2015. The first one from Panelbase showed the SNP dropping four points and raised some speculation about whether their huge post-referendum surge was fading away again. The second from Survation also had the SNP down, but only by two points and this one from MORI has the level of SNP support holding steady. (On top of that, when tables for the Panelbase poll appeared it turned out that the voting intention question wasn’t asked first, it was asked after a question about whether or not falling oil prices damaged the economic case for Scottish independence, so the SNP fall in that first poll may be a question ordering effect rather than a genuine change)

TNS have a new Scottish poll out tonight which echoes the big swing towards YES than YouGov have been showing in their recent polls. Topline figures are YES 38%(+6), NO 39%(-6), Don’t know 23%(nc) with don’t knows, YES 49%(+7), NO 51%(-7) without. Taking just those certain to vote YES and NO would be equal. Full tabs are here.

As usual with TNS’s face-to-face polling the fieldwork is a little older than the telephone and online polls we see – fieldwork took place between the 27th August and 4th September, last Thursday. This means that much of the fieldwork is older than the Panelbase and YouGov polls at the weekend. The important finding though is the big shift towards YES. As we’ve seen, over their last three polls YouGov have shown a steady and substantial movement from NO to YES, turning the race from a roughly a 40-60 split to one that’s broadly neck-and-neck. Panelbase didn’t pick up the same pattern, but had consistently been showing a very close race anyway. TNS had been showing large steady leads for NO, similar to those from YouGov, and now they too are showing the race tightening right up so it’s broadly neck-and-neck. The YouGov and TNS polls are conducted using very different methods – YouGov an online panel, TNS a traditional face-to-face quota sampled survey conducted in people’s own homes, yet they are showing the same pattern. Any idea that the narrowing is something to do with online methodology goes out of the window, it looks as if the race genuinely has tightened right up.

The thing to watch now will be how the Scottish public react to the tightening of the race. It’s easy to imagine people recoiling from the risk of voting YES as it becomes something that might really happen. It’s equally easy to imagine a surge of enthusiasm for voting YES as it becomes clear it’s something they can really make happen.