Every fortnight or so there is an Angus Reid poll for PoliticalBetting, and every fortnight or so there is the same discussion in the comments here about whether they are right or wrong. Why do Angus Reid consistently show a lower level of support for Labour?

There probably isn’t one simply answer, I’ve seen several people spot a potential difference in what Angus Reid do, and decide on that basis that this *must* be the cause of the difference. Realistically, there are a lot of differences between what different pollsters do, and all these things can have an impact on the result. So –

Angus Reid word their question differently (from all the other companies)
Angus Reid do not have voting intention as their first question (unlike all the other companies)
Angus Reid do fieldwork online (unlike ICM, MORI, CR and Populus)
Angus Reid use their SpringBoardUK panel (unlike YouGov)
Angus Reid weight using newspaper readership (unlike ICM, MORI, CR and Populus)
Angus Reid weight using past vote (unlike MORI and YouGov)
Angus Reid weight using data collected at the time of the survey (unlike YouGov)
Angus Reid do not factor false recall into their past vote target (unlike ICM, Populus and CR)
Angus Reid use a squeeze question (unlike ICM, Populus and YouGov)
Angus Reid do not filter or weight on likelihood to vote (unlike ICM, MORI, CR or Populus)
Angus Reid do not reallocate Don’t knows (unlike ICM and Populus)

This is not to suggest that Angus Reid are more different than other companies – you could come up with lists of exactly the same length for any of the other polling companies. It’s just to demonstrate how variable the methodologies used by pollsters in voting intention questions are. For an outside observer it can be very difficult to isolate a variable and see what it’s effect is, especially since many of the differences are inter-related and have effects that cancel one another out.

Anyway, with those caveats aside, what are the most likely explanations for the difference? We can largely discount the effect of the question wording – my understanding is that Angus Reid have done parallel testing of the different question wording and it makes no difference to the result. We can also ignore the lack of a filter/weighting on likelihood to vote, since the experience of all other pollsters is that introducing this would make Angus Reid’s Labour figure even lower. We can also dismiss the squeeze question – Angus Reid’s tables show their results prior to the squeeze question being asked, and it does not normally make any difference to the level of Labour support.

Angus Reid’s decision not to ask voting intention as the very first question has the potential to skew responses (which is the reason no one else does it!), but I would be surprised if it had much effect in this case – most important issue is a non-partisan question that should not skew answers.

One factor that almost certainly makes a difference is Angus Reid’s weighting. The other three companies who weight by past vote target figures that assume a certain level of false recall (in practice, this means Labour are weighting to a higher level and the Lib Dems are weighted lower). In contrast Angus Reid weight their past vote to figures based on the actual shares of the vote in 2005, meaning their samples have a higher proportion of people who said they voted Lib Dem and a lower proportion of people who said they voted Labour than ICM, Populus or ComRes. They also weight people to who say they did not vote in 2005 to a lower level – this too is probably a factor, since the Conservative lead amongst those people seems to be lower.

Angus Reid’s tables do not provide a cross break by past vote, so we can’t work out for certain the difference it makes, but we can get a good idea of the potential impact. If you look at the last ICM poll and take their answers before likelihood to vote filtering and their topline adjustment, it gave shares of the vote of CON 39%, LAB 31%, LDEM 20%. Now, if you crudely reweight the past vote to the figures Angus Reid use you get CON 41%, LAB 30%, LDEM 21%. As we’d expect, Angus Reid’s weighting helps the Lib Dems and Conservatives and punishes Labour.

However, this doesn’t explain the whole difference, since Angus Reid do not in fact show higher levels of Conservative support , but they do show higher levels of support for “others” – the minor parties like the BNP, UKIP and Greens. What I suspect is that there are two factors at play here – *something* is pushing up their level of other support, and the remaining support is being skewed against Labour and towards the Conservatives by their past vote weighting.

As to what that *something* causing a higher level of support for others is, I really don’t know. I can’t see any obvious cause. It could even come down to Angus Reid’s sample, or the make up their panel – things it’s not really possible to judge from outside.

This brings us to the question of who is likely to be right. Several people on here have said that you can’t judge at all until an election, others have said that a pollster who is consistently different from all the others is more likely to be wrong. I wouldn’t necessarily agree with either. In 1997 ICM were consistently different from other companies and turned out to be correct. In principle it can be the outlying company that’s right and everyone else who is wrong, but an important caveat is that in 1997 we knew from the previous election that there was a problem, ICM turned out to have solved it and others hadn’t. In 2005 all the pollsters did pretty well, so for Angus Reid to be correct everyone else (despite their differing methods) would have had to have developed some problem with their polling since 2005 (or be unable to cope with changes in public opinion since then…)

The other question was whether you can judge a pollster before an election. In some cases you can. If, for example, a new pollster came along who weighted their sample so it was 75% male and 25% female we wouldn’t need to wait for an election to know they were doing something pretty seriously wrong. The failure of Angus Reid’s past vote weighting to account for false recall seems demonstratably wrong in the same way – the evidence of false recall is solid. Since Himmelweit et el first described false recall of past vote based on panel studies in the 1960s and 1970s it has been repeatedly tested in academic panel studies and we can be relatively certain that there a systemic skew in how people report they voted at previous elections. Weighting recalled past vote to the actual shares of the result should underestimate Labour support.

My view therefore is that Angus Reid are likely to be wrong in terms of the gap between Labour and the Conservatives. However – other polling companies have tended to overestimate Labour’s position, so we may yet find that Angus Reid are right for the wrong reasons. It’s also worth noting that the academic evidence for false recall is largely based upon face-to-face surveys, theoretically people could be more accurate in their recall in online polls. In terms of the other big difference between Angus Reid and the other pollsters, their higher level of support for “others”, I have little guidance I can offer you – I don’t know what the cause of the difference is, so can’t begin to say if they are right or not. Only time will tell.

Of course, this is just my opinion – I’m sure Angus Reid would disagree. No pollster produces figures that they themselves think are wrong, especially on voting intention, figures upon which reputations are built and destroyed. I normally try my best to explain the potential differences between polls rather than say which one is wrong or right – after all, I work for YouGov, so I have a natural bias in that direction. I think one can come to a legitimate view on which poll will be right ahead of an election based upon their methodology… but even then, you won’t know if that view is right until the day afterwards.

(BTW – there will indeed be a brand new ComRes poll for the Indy on Sunday later on tonight)


54 Responses to “Why do Angus Reid polls show a lower level of Labour support?”

1 2
  1. @Smokey Jones

    http://www.surveysystem.com/sscalc.htm

    Shows how to calculate sample sizes. I might be doing something wrong, but I have put in figures of 44m for population, 99% confidence level, and 2% confidence interval, This gives a sample size of 4160. I could do the calculations myself, rather than trust the site, but I’m too tired tonight. I’d be interested in learned statisticians views on the figures.

  2. Let’s start off with, I’ve never been asked! Now if the polsters ask the same people for their voting preference and they get a different answer from one month to another then we have to ask, how? Now if they ignore the first group and ask another group then they wont know if the first group have changed their minds. How can anyone be persuaded to change their minds, their only source of info is the media, who are biased towards WHO? and you DO know the answer to that.

  3. @ Alec
    Glad to see you have been looking into things before replying.
    This is not a financial forum, as you well know; there is no scope for uploading images and using up bandwidth on a discussion that veers considerably from the OP.

    I’m quite happy for you to not agree with me – the more the merrier for me. It does however, have political implications.

    Inflation is necessarily going to rise because of a number of global factors that I’m sure you can imagine: energy; unresolved debt; population/climate/pollution issues; etc…,

    interest rates must rise to keep on top of inflation; which means that mortgages (especially buy-to-let ones) will become unaffordable and and creating ideal conditions for an avalanche of selling and housing pricefall, mitigated (though not exactly helped) by immigration.

    Pressure will come onto any government to put measures in that effectively substitute in all sorts of complicated ways for the large council housing sector that used to exist before the US went off the gold standard and the UK deregulated the banks. I’m sure you know all this.

    It’s not simply to do with the “chimps in the zoo”, but the zookeeper (i.e. the government), and their attitude to control of the financial sector.

    Not all metrics are of equal relevance or value, which I’m sure you also know.
    True, putting it in simple terms that the man on the street can relate to has it’s drawbacks, but it does not make the core issues implied any less true.

    There’s no use trying to blind with science or attempt to conflate financial traders with systems engineers and muddy the waters either, the systems analysis perspective is not a narrow financial one, and obviously includes all systems. As I have effectively stated, the external input to the UK system came from America in 1971.
    There are clear and very tangible levers that were pulled that triggered the current divergent phugoid; a few interrlated example steps:
    1. financial deregulation by Labour governments (plural)
    2. emergence of consumer credit
    3. reduction in the council housing stock
    4. expansion of mortgages and unsecured loans
    5. entry of China into the global market as a resource-to-goods converter
    6. expansion of containerisation of ships

    it could go on, nothing to do with obtuse and occult theoriesm but simple and specific events and their tangible input into the system.
    …of most recent significance is Labour’s response to the Asian economic crisis (part of the China blindness effect), being (just like Callaghan) to keep interest rates far too low for short term electoral gain.
    I do not care for the Tories or LibDems, but it’s perfectly clear to me that Brown is responsible for the global crunch because of the huge leverage London has had over so much of the world, Brown pretended to be an economic whizzkid, and surfed a boom built on debt and a hands-free approach to financial sector regulation as bad as any Tory (remembering it was Heath who first let go of the reins).
    Nothing at all to do with twaddle like “climax types”.

    Labour caused the mess; the Tories can’t fix it; the LibDems can’t pick the right leadership team; and the minor parties are under constant assault: it’s an appalling mess. When it’s all taken in context, the AR results look far more plausible than any of the others, which frankly, are painting a fantasy picture of where the electorate is at.

  4. @seal pup – we’ll just have to disagree. To blame Labour for all this deregulation, but omit to mention the very Thatcherite Big Bang, and Callaghan for creating a boom when the Heath government’s politically inspired ‘Barber Boom’ was the cause of much of the problems in the late 70’s tells me you don’t know much about recent economic and political history. I’ll leave you in peace to massage your fugoids.

1 2