The polls have been largely static for the last three months, showing Conservative leads between 13 and 24 points.

Written like that, it doesn’t sound so static does it? Say the position was closer between the main parties, no one would describe a situation where polls were showing results varying between a 6 point Labour lead and a 4 point Tory lead as consistent.

The reality though is that they are. My first sentence did rather deliberately make things look volatile by focusing on the lead, which since it includes variation in both the Labour and the Conservative share of support falsely magnifies small differences. It’s more accurate to say that recent polls have shown the Conservatives at 41%-48%, Labour at 24%-29%, the Lib Dems at 15%-20%.

Over at PoliticalBetting Bob Worcester puts it all down to normal sample error. Looking at those bands of support that seems to fit, especially if you dismiss the one Populus poll that had the Tories down at 41% as an outlier. The reality is however that this is an overly simplistic explanation for the variation.

If it was all down to random variation we’d see some YouGov polls showing a low Tory lead, some Populus ones showing a high lead. It would – as you would expect – be random. We don’t see that. We see some pollsters consistently showing a lower share for the Conservatives and higher share for the Lib Dems than other pollsters.

Taking polls since the start of June:

YouGov average is CON 46.4%, LAB 25.8%, LDEM 16.6%
Ipsos MORI average is CON 46.7%, LAB 26.3%, LDEM 16%
ComRes average is CON 45.2%, LAB 25.7%, LDEM 16.8%

All very close, but…

Populus average is CON 43%, LAB 26.7%, LDEM 19%
ICM average is CON 43.8%, LAB 27.4%, LDEM 19%

If the difference was down to sample error, which is as likely to go one way as the other and affects all pollsters, then given enough polls the average from each pollster should be much the same. Clearly they are not – Populus and ICM are showing a lower level of Conservative support and a higher level of Lib Dem support. This is only three months data, so could easily be chance, but regular readers will know that these are actually long term trends and suggest that the contrast in the polls is due to methodological differences between the pollsters.

Part of this is because ICM and Populus are measuring slightly different things from the other pollsters. YouGov, MORI and ComRes’s figures are based on how people say they will vote. ICM and Populus’s also take into account how ICM and Populus think people who say don’t know will vote, and at this moment in time this cuts the Conservative lead. The rest is likely to be down to differences in their respective weightings.

That doesn’t change the central point that the polls are NOT all over the place, they have been very stable over the last couple of months. The broad picture we have is not, however, of polls all agreeing with each other with differences explained solely by sample error, they have slightly different ways of doing things that produce slightly different figures…albeit, each set pretty stable when compared to themselves.


20 Responses to “Not as simple as that…”

  1. Although I say this with no evidence to support the contention, it has always seemed to me that polling never changes smoothly – there hasn’t been a gradual shift upwards in the Conservative position and downwards in the labour position. There was a sudden substantial shift (or possibly two/three such shifts) driven by real events.

    This links to a further feeling which is that people tend only to change their voting behaviour in response to matters that are really personal – which is who inflation is more politically damaging than unemployment and why issues such as the NHS (for Labour) and immigration (for the Conservatives) seem not to have the purchase polling often suggests.

    This points towards Labour’s problems coming from: 1) the credit crunch; 2)the lost personal date (remember every child benefit recipient recieved a grovelling letter – 7 million were worried by this cock up; and 3) the abolition of the 10p tax rate. Perhaps someone cleverer than me can see whether these events do correlate (as I suspect they will) with the shifts in manifest support for labour and Conservatives.

    For the Lib Dems these are however the worst run of polling figures since the early days of Ashdown and must be very worrying.

  2. Does this mean the new LD leader Nick Clegg has established himself at the helm of the party and thereby reestablished their status and stability in the polls?

    I think there will be a lot of eyes on his performance at the LD conference because it will be his first – it will announce his arrival to much of the country and will set the tone of his whole tenure. A barnstorming speech could give a boost to LD fortunes going into the next election and provide some much needed momentum, while a weak and miserable exhibition could signal that they’ve reached their ultimate ceiling, though a modest triumph will indicate the likelihood of continued incremental progress.

    My feeling is that the poll stability proves we are now holding our breathe for the LDs to show us their colours – to see who they’re prepared to upset first or whether they can successfully bring together a new coalition of interests and redefine the political landscape in their favour.

    From my point of view they seem more credible than ever previously (with the emergence of Vince Cable), they are clearly fighting more seriously than ever before (with more incumbent MPs than previously) and they have grown a well-spring of in-built sympathy by showing they are more in touch with the public on issues like Iraq and more in touch with reality by dealing with their internal problems in an adult and mature manner.

    If ever they have the opportunity to make a big impression on the public and challenge the dominance of the big two parties then they do now.

    There is a definite case to made that this year the LD conference could be the most significant of the three and potentially their most significant in a generation.

  3. Or it could to be a damp squib.

  4. Simon , that is utter drivel . Ashdown became leader in 1988 and for most of the time since then LibDems have had worse polling figures than those in the past few months . In 1995 to 1997 for example the LibDems polled from 9 to 19% . In the whole of the 1997-2001 parliament except for a brief period in Sept 2000 they were polling worse than now . In the last parliament the LibDems were polling at current levels until mid 2002 and even in Feb/March 2005 the polling range was not a great deal higher than now .

  5. Anthony,

    Your thesis appears well reasoned. Have you challenged Bob Worcester to respond to the thesis?

  6. Mark:

    In the real polls which matter, isn’t it the case that LibDems are actually much less volatile Jeremy Thorpe’s era apart?

  7. Not sure John,

    The Liberal vote per candidate was 24% in February 1974 (so the highest apart from the Alliance in 1983).

    Their 1979 vote of 14.8% per candidate was lower than in 1964, and only marginally above their vote per candidate in 1970.

    Basically, they started contesting many more seats, hence the higher vote.

  8. I don’t think Nick Clegg has to do anything this side of the next GE. Other then wait that is.

    Project New Labour is on it’s last legs. Following the eventual Conservative victory Labour will revert to form. [Just look at Ken Livingstone’s new job for proof of that.]

    So 2011 will offer an opportunity for the LibDem’s to redefine the centre-left polity within the UK. I’d expect to see a few Blairites joining them as reality finally hits home.

    As to your point about the polls Anthony all very interesting. With all in stasis there is very little room for much fresh analytical comment.

    So why not generate comment by saying something controversial. Bob’s comments are like Nick Beale’s moving average: some agree, others don’t.

  9. For ICM and Populus to try to guess how the don’t knows will actually perform at a general election seems to me to be fraught with danger. If we do indeed have a new species of the shy voter this time apparantly in the form of a Liberal Democrat-and its true that they often pick up votes during a campaign but I don’t know if that links up-then I think we need to know a bit more about the reasoning behind this assumption.
    Personally I don’t entirely buy into this scenario. I think that for very obvious reasons there are far more shy Labour voters than Liberal Democrats around at the moment.

  10. Very interesting variant on the BW analysis Anthony.

    It looks like a settled picture for Labour, with the Con lead depending upon the truth about LD support.

    I agree with Fluffy about their strategy-wait for New Labour to reach it’s sell by date, then try to assume control of the centre left position.There would be no point pursuing Cameron on his ground if the punters had just voted him in.

    If Labour goes left this will leave an open goal for Clegg-but I cannot see the Blairites giving up without a battle.

    Will Labour split?-would the Blairites join the LibDems rather than the traditional Left?

    I think the Labour Conference is critical for them-will it be a platform for retrieving voter support, and if so what can they say which could conceivably bring that about?

    -or will it be a de-facto acceptance of looming opposition & the start of the battle for ownership of brand Labour?

  11. As they go to great lengths to be ambivalent about almost anything that is important LibDems are always well placed to take advantasge of Labour or Tory unpopularity but it never goes any further than that and they have virtually no long term committed support

  12. Nick, the comparative difference between Labour and the Conservatives is affected by the reallocation of don’t knows. The higher level of Lib Dem support is probably more down to the weighting.

  13. ‘As they go to great lengths to be ambivalent about almost anything that is important LibDems are always well placed to take advantasge of Labour or Tory unpopularity’

    I really dislike it when this poll site is a repository for these sorts of comments; generalist, obviously not able to be argued in lengths less than a book and showing the writer’s beliefs rather than logic. This site should be a place for discussion of polls, methodology, implications of polls and local knowledge where polls may not work.

    People who want to have political arguments based on broad generalisations should be elsewhere. (And no, I don’t vote Liberal.)

  14. I believe the YouGov poll for the Daily Telegraph normally comes at the last Friday of the month. Will this be tomorrow now?

  15. What I meant was, it normally comes out on the last Thursday of the month, and is printed in the Daily Telegraph the following day.

  16. My last comment is not a broad generalisation. It is a statement of fact in the context of unreliability of the LibDem vote, see “implication of polls” etc.
    And no, I certainly don’t vote LiDem either.

  17. David, are LD voters unreliable?

    My impression is that their ratings are volatile because the majority of their support is entirely conditional, whereas Labour and Conservative are mostly tribal.

    I’d have thought it was more sensible to take a contingent position as a voter and be able to switch the party you support – I certainly would not support a donkey in a rosette!

    FWIW I describe myself as an independent liberal because I make my own mind up on questions as they are presented to me. I have no inherent objection to supporting any party except in the way the issues are framed.

    If you do have some principled objection to voting for any particular party (which appears to be the case) then by definition you ARE generalising far too broadly to be accurate.

    Each of the major parties do represent different philosophical traditions and behave differently for this reason, so of course it is entirely consequent that all ideologically-defined tribal supporters will fail to rationalise the poll fluctuations of their opponents accurately enough to help themselves.

  18. Anthony

    I think you have missed the point of what Bob Worcester was trying to say. Of course it is true that that different polling methodologies can and do cause systematically different results, and Bob would certainly not disagree. There are consistent differences between the results of the different companies at the moment. But they are small ones.

    Nobody is suggesting that these differences are no more than random variation – Bob does not “put it down to normal sample error”, he simply points out that the differences that exist are no bigger than would have to be expected if it were no more than normal sampling error. None of the polling companies use simple random sampling for their polls, so the statistical probability laws and the standard “margins of error” do not strictly apply in any case. In fact we use different sampling methods, we interpret the data in different ways, and for that matter we are not necessarily trying to measure the same thing.

    You would think that we ought to be all over the shop – but, oddly enough, we are not. Although it is possible to detect systematic differences caused by the differences in method over a number of months, that difference is so small that it is within the compass of what we would consider the “margin of error” if were using simple random samples. Within any one cycle of polls, the differences between the various polls are not generally more than we might expect if they were all random samples and measuring something much less complex than voting intention; there are occasional outliers, of course, but the number of outliers from the whole group of polls over a period of months is no more than would be expected if they were all random samples (even though the different poll series do not have the same mean). Even though it is possible, knowing which company conducted which poll, to detect systematic differences between them, taken as whole they don’t look very different from what you would expect if they were all random samples.

    What this means is that, despite their differences in methodology, all the polls are telling us the same thing. Two polls can only be considered to disagree if their results are so far apart that there is no possible “true” result within the “margin of error” of both of them. Bob’s point is that, except for the occasional outliers (and there are only about as many of them as one has to expect in any sampling exercise), none of the polls are disagreeing with each other in this sense. That is so even though with non-random sampling these margins of error are more a rule of thumb than something we can strictly expect to apply on statistical grounds (and even though the conventional plus-or-minus-three points margins of error which Bob is using assume a sample size of 1,000 when nearly all the polls have effective sample sizes considerably lower than that).

    Furthermore (although Bob does not go into this in detail in his article for Political Betting), the polls are not static. If you take the record back to last year, before Gordon Brown became PM, you can find this same degree of consistency between the polls even when the state of public opinion was very different. Month after month the polls have been sufficiently in step as Labour support has risen and fallen that almost all have been within this narrow margin of the running average.

  19. Hi Roger,

    I don’t imagine that Bob would disagree with anything I’ve written (in fact, I’m sure I remember you or Bob writing a very similar article pointing out that different pollsters are measuring slightly different things in a pre-election column- I looked for it when writing this but couldn’t find it on the new website!).

    I think Bob’s original article did carry the implication that this the only reason for the difference – doubtlessly unintentional since I know he doesn’t think that. Take that difference into account and the consistency of the polls is even more notable.