Two types of outlier

This morning TNS released a new poll showing figures of CON 28%, LAB 35%, LDEM 6%, UKIP 18%, GRN 7% (tabs). The seven point Labour lead is striking, and out of line from the broader trend. Usual caveats apply, but for once I haven’t seen many people over-react to it. Most have sensibly enough seen it as the just the other side of the coin to the ICM poll earlier this week showing a solid Tory lead – two outliers in opposite directions. However, it is worth looking at the different reasons why these two polls went against the trend.

On average ICM produce figures that are slightly more Con/less Lab than average, but only by a tiny bit, generally they show Labour -v- Tory leads that are much the same as those from other companies. The reason that their poll on Monday showed a four point Tory lead when other companies are showing the race almost neck-and-neck is almost certainly just random sample error. Samples vary from day-to-day, month-to-month and sometimes you get one that’s a bit Laboury or a bit Conservativety in ways that weighting does not correct. I expect ICM’s next poll will be in line with those from other companies.

TNS is a different sort of outlier. The seven point Labour lead may be different from that shown by other pollsters, but its actually in line with TNS’s previous polls. In their previous four polls they showed Labour leads of 6 points, 0 points, 7 points and 7 points. Over on my chart of house effects TNS’s polls on average show a Labour lead three points larger than other companies (and if anything the gap is growing!) This isn’t random sample error, this is a consistent methodological difference between TNS and other companies, and unless they switch methods I expect their future polls will continue to show bigger Labour leads than other polls.

Two outliers, but one is probably just a random blip from a pollster who normally shows the same as other companies, one looks like a typical poll from a pollster who regularly produces polls that show a bigger Labour lead than the pack.

311 Responses to “Two types of outlier”

1 2 3 7
  1. First.

    Always wanted to do that.

  2. Lol TNS.

    Or as the papers would have it, “Tories collapse by 8% in three days!11!!!1!”

  3. @ Anthony,

    Any idea why they have such a huge house effect? I can’t see anything in their methodology that would obviously cause it, although they were notably wrong in 2010, with the Lib Dems way too high, both main parties too low, and the Tory lead 1% too low as well.

  4. Thank you Anthony.
    No papers are running on the TNS outlier.

  5. TNS are members of the British Polling Council, Lord Ashcroft is not. I would strongly question those who dismiss TNS polling, but put strong credence into Lord Ashcroft’s.

  6. No surprise there Chris, although if the Mirror pick it up they are likely to make something of it.
    TNS is the company I work for (though I play absolutely no part in polls, though I did as a fieldworker in 2009-2010). The consistently bigger leads they give Labour give the lie to those who think, for example, that Ashcroft polls are biased towards the Conservatives because Ashcroft himself is an active Tory. TNS is a subsidiary of WPP, whose CEO is Sir Martin Sorrell. Sorrell is a known Tory but no-one could accuse him of manufacturing polls which favour his party. Of course, using the pretzel logic of those who claim that such-and-such a poll is biased in favour of such-and-such a party, one would have to conclude that YouGov polls are simultaneously biased in favour of Labour (since its President, Peter Kellner, is a known Labour supporter, and married to a Labour peer) and the Conservatives (since its major shareholder, Nadeem Zahawi, is a Tory MP). :)

  7. Chris Lane,
    Well they wouldn’t say that would they!

    Very Good!


    BPC rules are mainly about publication rather than methodology, and so are most certainly not a guide to reliability of their published results.

  10. @ Jayblanc,

    Ashcroft publishes his tables, though, and the pollsters he uses are themselves members. It’s not BPC membership that in itself confers legitimacy, it’s the transparency it enforces. If he’s willing to abide by the standards without being a member I don’t see why we shouldn’t trust his results.


    Sorry. My comment re BPC wrt TNS was meant for JAYBLANC.

  12. So surprised there is no Guardian article on this poll. Would have thought it of major importance that we have had a swing of -6% to +7% for Labour in such a short space of time.

    Maybe we could analyze the salient reasons why:
    * further HSBC revelations filtering into the public zietgiest?
    * realisation that the Govt. cannot fix Greece?
    * Ukraine truce being agreed and falling apart?

  13. * TNS did a poll?

  14. Fizzer,

    Usually governments will claim credit for good economics news and deny responsibility for bad economic news. On the other hand, critics of a government will do the reverse. Spinners gonna spin.

    This is not news, so I’m afraid I don’t think that there’s any psephologically interesting to discuss in your question.

  15. Why is Ashcroft not a BPC member?

  16. I can’t imagine Sorrell spends too much time focused on the TNS poll TBH otherwise Labour wouldn’t be 7 points ahead. I was at TNS when they set up the polling but wasn’t involved. As you can imagine, the people who set it up were experienced professionals with no political agenda so it seems weird to me that they constantly show these big Labour leads. It’s got to be something to do with the quality of the sample on their OnlineBus if you ask me.

  17. FIZZER

    @”What about full time jobs? ”

    ” Changes in people in employment between October to December 2013 and October to December 2014, seasonally adjusted”

    Full time- +460,000
    Part time + 148,000
    Total + 608,000

    UK Labour Market, February 2015

  18. Why is Ashcroft not a BPC member?
    Isn’t that like asking why The Times isn’t. He commissions polls, doesn’t actually conduct them. Unless I misunderstand?

  19. Bill Patrick
    “This is not news, so I’m afraid I don’t think that there’s any psephologically interesting to discuss in your question.”

    Yeah, ‘cos non-polling stuff never gets discussed on this site does it……

  20. Bill,

    I know that. I think the measure of unemployment is flawed, as it is done in UK.

    If there was a true recovery, tax revenues would be rising strongly, but they are not. If you replace a full time job with a zero hour contract, ofc tax revenues can’t rise, even though U might fall.

    It is Germany where there is true growth, as tax revenues are rising sharply.

    I think I read somewhere that only 1 in 20 new jobs now are full time. I think UK has more par time jobs than anywhere else now.

    I wonder if people will buy this mantra.

  21. Bramley,

    Actually, it does. What we all try to avoid, I’m sure, is partisan discussion, and talking about the government’s economic record is obviously going to get a partisan discussion going, without helping us understand polling and voting issues.

  22. TNS is certainly breathtaking, outlier though it must be. The Lab – Con gap widens to 9 points (36-27) on the second Q about constituency voting intention. Also very interesting Scottish cross break showing Tories ahead (yes!) of both SNP and Labour, which for some reason none of the usual suspects on this site have posted ;-). Otherwise Labour lead in every other region apart from the SE. Some of the leads are commanding. Are we seeing an affluence effect ? The two wealthiest regions on TNS split are SE and Scotland – and those are the two where Cons lead….

  23. BRAMLEY.
    A long time ago Peter Shore used to say that prosperity is a good time for Labour when fighting a General Election in Opposition, since voters feel safer with the idea of a Labour Government, which does, in its guts, wants to support largesse in public spending.

  24. @ChrisLane,

    I think that’s been the underlying dynamic of the 2015 GE for a long time. Can the Tories get the right blend of voter satisfaction with the direction of travel, and voter anxiety about future economic prospects. It’s a very tough juggling act, and so far the external circumstances have given the Tories a relatively easy ride in this respect.

    Of course it may be that even with the perfect balance of both, the electoral arithmetic cannot be made to work for the Tories/Coalition, whatever they do.


    ” Also very interesting Scottish cross break showing Tories ahead (yes!) of both SNP and Labour, which for some reason none of the usual suspects on this site have posted”

    Dear oh dear please check back on the previous thread where ” the usual suspects” did indeed post about the Scottish cross break but there is only so much one can discuss on a sample of just 49 people.

  26. CHRISLANE1945

    Perhaps check the historical data before making such a political claim.

  27. FIZZER

    @”I think I read somewhere that only 1 in 20 new jobs now are full time”

    “Total people working full time 22,597,000
    Total people working part time 8,299,000
    Total people working 30,896,000”

    UK Labour Market, February 2015
    Summary 3

  28. FIZZER

    @”Colin, think about why tax revenues aren’t rising much. If most of the new jobs were full time, at higher pay, they would be.”

    Not necessarily. Increasing the Tax Free Allowance for so many millions of earners has obviously impacted tax revenues.

    @I am just saying that what Tories are claiming, should be taken with a grain of salt.”

    These are ONS numbers-the Office of National Statistics.

  29. @Fizzer,

    Remember that tax on low wages has been quite drastically reduced, and that the value of in-work benefits (although they have been cut) means that there isn’t always a massive saving for the treasury when someone goes from unemployment to low-paid work.

    It is perfectly possible to obtain a full-time, non-zero-hours job and still not be contributing very much to the Exchequer. That doesn’t mean that the reduction in unemployment isn’t still a hugely good thing, though. As the difference it makes to the employed person and their family, and the boost it gives to the economy as a whole, is far more important than the net gain in direct taxation.

    Eventually, a sustained reduction in unemployment will lead to above-inflation rises in pay, which we are just starting to see (although the temporary drop in inflation has perhaps brought this forward artificially).

  30. FRITZ

    “I am just saying that what Tories are claiming, should be taken with a grain of salt”

    I watched the news at lunchtime and 3 different people were interviewed and gave a positive reaction on the economic news. None were Tories or at least none were Tory MP’s.

    To give the Tories credit they are not jumping up and down and saying all is well and there are problems still out there for low wage earners and long term unemployed.

    All the main KPIs show the economy is on the right tracks but no one is saying “mission accomplished”

  31. Lies,damned lies and statistics is an old and valuable saying.

  32. FIZZER

    Ah-TUC numbers then-I note they they are excluding self employment..

    Yes-that seems to be something they don’t like.

  33. Ignoring outliers for the moment, it would appear we are still firmly in hung parliament territory.
    Constitutional convention in those circumstances indicate that DC, as the incumbent, will be allowed the first opportunity to construct a coalition by which he can command a majority in Parliament.
    It is revealing that using the advanced swingometer (and some recent polls for Wales and Scotland along with the English Averages that have emerged) the only way for DC to achieve such support (with 275 seats) would require SNP support in some way (42 seats). SNP having ruled out any deal with the tories (as I was so firmly reminded recently in another thread) means that SNP support in Scotland could not lead to a tory government unless there is a significantly large English and Welsh vote gap between the Tories and Labour to the former’s advantage.

    In the novel “Nightwatch” by Terry Pratchett there is a poignant line after someone is speared through that and the cynical capatain watches the young colleagues carry of the bosy to a medic that “he was always amazed of the ability to see life in the corpse of a friend”

  34. AC

    Thanks – I should have guessed – you guys are on the alert 24/7 for a new Scottish crossbreak. Still it’s so strange that it was worth posting again on this thread ;-) a possibly unique crossover.

    IMHO the ICM and TNS looked as dodgy as each other, and I wonder if both have inbuilt permanent defects ( not just TNS as implied by AW). One point I recall from looking at the ICM tables is that they suggested Cons were just ahead of Labour in Wales, which is probably about as likely as the TNS crossbreak for Scotland.

  35. Ann in Wales

    “Lies,damned lies and statistics is an old and valuable saying”

    Especially during an election campaign!

    At least, in England, they’ll get a break for a wee while – do you get the same as we have, where the opposition parties are furiously campaigning on issues for the 2016 elections rather than this May?

  36. In fairness I think there are serious concerns over how many of the new “self-employed” are actually in work at all, as there is some evidence that DWP staff point people in that direction as the “in work” benefits that attach to self-employed status can leave people better off than JSA. Plus of course it gets you around the inconvenience of being asked what you’ve done to find a job.

    However, one can imagine that the TUC, which almost by definition exists to further the interests of “employees”, wouldn’t want to encourage alternative ways of life. You can’t go on strike against yourself.

  37. This was ONS’ comment on self-employment trends :-

    “”The rise in self-employment can be accounted for by fewer people leaving self-employment than in the past. About 886,000 people who were self-employed in 2009 had left by 2014, compared with 1.3 million who were self-employed in 2004 leaving by 2009,” the ONS said.”

  38. Old Nat,
    I would be a renowned soothsayer if I could predict what will be happening in
    2016.Presumably if the Tories get in they will be campaigning for the referendum.Or another general election,groan.

  39. Fizzer

    You asked about employment statistics, the data in the news, so it’s dishonest to try to change the subject to tax receipts.

    There is nothing wrong with UK employment statistics: they are produced by the ONS, follow an OECD standard and are audited by Eurostat.

    The release says:
    73.2 per cent of people aged 16 to 64 were in employment, highest since 2005. Unemployment at 6-year low at 5.7%. Total average weekly earnings 2.1% up on a year before.

    You might think that this data is pretty comforting to the government. And no doubt they will try to spin it that way. But the reality is that for a recovery from a recession, the news is not so great. Unemployment is not yet back at pre-crash rates (though we will get there eventually). Wages must eventually stop falling in real terms and start growing. So the fall has ended, big deal. The Chartered Institute of Personnel and Development polled employers – it seems like the expected average annual wage rise next year will be between 2% and 2.5%. So people will still be worse off a year from now than they were in 2010.

    So here’s a slogan for Labour, based on the data:
    ‘Are you better off than five years ago?’
    For the majority, the answer is no.

  40. @Colin,

    That’s reassuring, although I suppose if your self-employment is a fiddle to start with, you’re less likely to leave it?

    No doubt when the Sword Of Truth enters Downing Street in May the real situation will become apparent.

  41. Ann in Wales

    “Lies,damned lies and statistics is an old and valuable saying”

    Indeed and another is “Politicians use statistics in the same way a drunk uses a lampost: more for support than illumination”

  42. Colin, Neil A,

    I am perfectly aware of the economics of it, and I am not saying a fall in U is bad, however the full picture is not revealed by just saying there was a fall in U.

    I am not saying vote Labour to fix it-just that it is a fact.

  43. NEILA


  44. When we get economic figures, can we please not have a display of people trying to put politically spin on them. Talk about their implications on public opinion (if any) by all means, but we can do without the “interpretation of the figures in a way that helps the party I support”.

    And if others do fall into that trap, can the rest of us please ignore them rather than argue or respond with sarcasm. It takes two to have a partisan argument.

  45. Colin
    Your statement is simply untrue. There are problems with the data from the Department of Employment but not from the ONS.

    I remember when Tony Blair claimed that the the price statistics were being manipulated by the Major government. Then I got inside the CSO (as the ONS was then) and saw that the price statistics were in such a state that no-one could manipulate them!

    The ONS has the legal right to respond to erroneous claims about their statistics…


    “Thanks – I should have guessed – you guys are on the alert 24/7 for a new Scottish crossbreak.”
    Haven’t you uploaded the (OLDNAT crossbreak alert app) onto your smart phone yet?

    Interestingly I would say the Tories stand a much better chance of catching Labour in Wales than in Scotland but the TNS crossbreaks are a bit comical in the sense that the Tories are leading in Scotland.

    I did have fun with my app though. ;-)

  47. AW

    My comment at 1:38 pm on the economy should set the gold standard for non partisanship.

    I’m not looking for a lollipop or anything but I am testing out my new trumpet. ;-)

  48. people believe government statistics are accurate if they support the party in power,if they don’t then government statistics are lies.
    on the polls,I asked the telegraph journalist who excitedly claimed on his front page the tories had a big surge in support if he was going to put todays tns poll on his front page also. strangely I haven’t had an answer yet.
    maybe they’re still in mourning over peter obornes departure.

  49. @WB
    I don’t see why there is ever a need for politicians to be dishonest with the statistics. If the data does not support your story, maybe you should rethink what you want to say.

    With the current employment data, both main parties have something to say. The Tories might have a slightly easier sell, but nothing contradicts the Labour line on earnings (which why it was chosen in the first place).

  50. Allan Christie

    “Haven’t you uploaded the (OLDNAT crossbreak alert app) onto your smart phone yet? ”

    Sadly, there was that time that I misread the data, and published wrong information.

    Fortunate that politicians in Scotland never to do that and have to hastily remove their tweets and videos! :-)

1 2 3 7