Back to those Lib Dem private constituency polls they were so keen to brief the press on last month. When we left the story it had emerged that the polls had been done as purely field & tab, so there was no British Polling Council requirement to release them (that is, Survation just made the phone calls, the actual method and analysis was done by the Lib Dems themselves. In the same way, for example, ICM or Populus’s call centres are sometimes commissioned to do the phone calls for other companies and it doesn’t make it an ICM or Populus poll). Now the Lib Dems themselves have released one of the polls, in Hornsey and Wood Green – tables are here.

Hornsey and Wood Green is the seat of Lynne Featherstone and has Labour in second place. Lord Ashcroft’s polling of Lib Dem seats shows them doing far worse in LD -v- Lab seats than in seats where they are up against the Tories. In Hornsey and Wood Green itself Ashcroft found the Lib Dems trailing by 13 points. The Lib Dems own poll shows them only behind by one point.

As ever, one should be particularly suspicious of polling produced by political parties – certainly one should ignore them unless they cough up the actual tables so we can see what they’ve been up to. In this case the Lib Dems have done just that, so we can see for ourselves. As we knew from Survation’s earlier statement, the polls were not conducted using Survation’s own methodology, but using a methodology directed by the Liberal Democrats. There were a couple of specific factors:

  • The first was that the poll was weighted to actual past vote. There are essentially three approaches to past vote weighting in constituency polling. One is to assume the problems of false recall and people moving constituency are insurmountable, and not to use past vote weighting at all. This is what Survation do in their own polls. Another is to estimate some level of false recall and factor that into your weighting targets, which I think is what Ashcroft does in his constituency polls. The third approach is to assume no false recall and weight to the actual shares of the vote in 2010, which is what the Lib Dems did.
  • The second is that the question itself included the names of the party candidates, which is generally thought to boost the Lib Dems. This is not necessarily a bad thing – certainly I have grave doubts about polls done in Lib Dem constituencies that just ask a standard voting intention question. The alternative approach taken by Lord Ashcroft is to ask a two stage question, asking people a standard voting intention question and then to think about their specific constituency. This also gives the Lib Dems a significant boost.
  • The third was question order. The Lib Dems did not ask voting intention first, instead they asked people to rate Lynne Featherstone first. It is possible that this boosted her personal vote in the subsequent voting intention question.

All three of these methodological choices probably helped the Lib Dems. However that doesn’t necessarily mean they are wrong. I wouldn’t personally have made the same weighting decisions as them, or asked a question before the voting intention question, but that’s the beauty of publishing the tables for people to see: we can judge for ourselves what to believe. The candidate naming question is perfectly reasonable – the fact is, because constituency polls have historically been very rare we don’t really know what the right approach is. It may be that a raw standard question actually gets the best result, it may be that the two-stage question with a constituency prompt gets the best result, it may be that prompting with candidate names gets the best result, it could even be that prompting people to rate their MP first gets the best result (though my own guess is that it is forcing respondents too strongly to consider the candidate). Unless someone is nice enough to commission and release lots of constituency polls using different approaches very close to the election, we will never know.

With all that said, I’ll still be giving no real weight to the Lib Dem private polls for completely non-methodological reasons. I would advise people to ignore them because of potential publication bias. The Lib Dems have commissioned about 120 of these polls, they’ve released one. I don’t think it overly cynical to ponder whether they may have chosen to release one putting them in a relatively good position, rather than one showing them getting a soaking. With sample sizes of 400 or so people, the Lib Dem polls have a margin of error of +/-5% (and that’s on each figure, so on the lead between one party and another it’s double that). By definition, normal random sample error means some of those polls will be overstating the true Lib Dem position by as much as 5 points, others understating it by that much. We have no way of knowing whether there are many other private Lib Dem polls that show the party doing even worse than in the publicly available polls, which they have chosen not to publish. And, to be fair, short of publishing all their private polling there is no reasonable way the Lib Dems can prove it either.


41 Responses to “Lib Dem private polls again”

  1. @Anthony

    You’ve taken a lot of words to basically say that we shouldn’t take a blind bit of notice of these private Lib Dem polls!

    :-)

  2. Lynne should still keep her seat though as she’s been an incredible minister.

    In case I get accused of being partisan though, I will point out that I think it’s obvious that it would be helpful for us to see all the fieldwork.

    And first.

  3. What a thoroughly professional and objective analysis, Anthony – and all the more devastating because of that.

  4. Ah, but it’s important to know WHY you shouldn’t take any notice of them. We can’t just go round ignoring things whilly-nilly.

  5. @Zak P

    “And first.”

    Afraid not. See above!

    :-)

  6. YouGov/Sun poll tonight – Tories ahead by two: CON 33%, LAB 31%, LD 8%, UKIP 15%, GRN 6%

  7. YouGov/Sun poll tonight – Tories ahead by two: CON 33%, LAB 31%, LD 8%, UKIP 15%, GRN 6%

  8. Anthony

    “We can’t just go round ignoring things whilly-nilly.”

    Come now! That’s what lies behind many posts on here – “I have no evidence for it, but I’m really scared that X will happen.”

  9. If we saw enough of these polls they might be interesting. They would probably give some indication as to whether the Lib Dems still have a chance in these constituencies or if they are definitively toast.

  10. YG at least definitely seem to be showing that “something’s changed”, although the “big two” are quite low in that poll.

  11. Looking forward to the two horse race bar chart in Hornsey and Wood Green.

  12. No evidence of any great Tory surge and UKIP holding very steady. However, what is starting to become clear is that Labour are sinking.

    And sinking rather fast.

  13. Night All.
    Number cruncher says first tory hat trick since October

  14. @Neil A

    ‘YG at least definitely seem to be showing that “something’s changed.”‘

    Not yet. It’s not Thursday. That’s the key poll.

  15. It’d be noble of the Lib Dems if they agreed to release all of the polls after the election. Can’t do any harm then, and society would benefit from having all of those methodological test cases.

  16. Crossbat

    Purely looking at publication bias actually tells us a lot. The LDs are in serious trouble. I have no idea how good their best poll should be out of 120 but they will have picked the best sample with regard to lead not just VI, which has an even higher volatility, we expect 6 samples to lie outside the quoted MOE.

    I really feel I should run a bootstrapped model and find out what the average excess lead of the best sample out of 120 is. I wouldn’t be surprised if it were over 10%, funnily enough in line with Ashcroft.

  17. Chris Green

    If they did it’d be like a donation for medical science!

  18. @ CROSSBAIT

    I’m not sure Labour are sinking rather fast. I think its still neck and neck which the next two nights YouGovs will show.

  19. @ Anthony Wells

    Choosing the result that suits you from a dataset, and ignoring others is perfectly acceptable in most academic journals
    Publishing something because you found something interesting in your data (while you ignored a number of other interesting things) and hence not matching your original research design is perfectly acceptable in most academic journals
    Not declaring how many iterations you did your factor analysis is perfectly acceptable in most academic journal
    Getting a job in Stanford for a single paper with a blatantly obvious methodological error (and not being fired when found out) is acceptable.

    Aren’t you putting the hurdle a bit too high for the LibDem’s private poll? :-)

  20. March YG Average:

    Con 34 Lab 33.

  21. @Norbold

    If we really start to see Con/Lab even again tomorrow with Lab ahead again on Thursday, that would be a very odd indeed!

  22. @norbold we’ve been getting bar charts for years… but one of the reasons this poll was released is that Mark Pack is integral to Featherstone’ s campaign. Also, she is one of the Lib Dems most important MPs – supposed to be on the coalition negotiating team. It will be tighter than Ashcroft (polling done before conference season) predicted.

  23. Interesting stuff. What kind of effect do you think the Budget will have on VI, if any? It does tend to be in the news for a few days. But then so was the debate fiasco.

    This is the first general election I’ve followed in such detail, so I’m not sure.

  24. Chris Green – I bet Mark Pack would actually go for that, but I doubt he’ll be able to swing the party behind such a donation!

  25. I think the clear purpose in releasing this poll is that the race in this seat is close, but to win they need to squeeze the Tory vote to Lib Dem. If they manage to do that, they have a chance of holding it.

    It is a campaigning tool, not meant to be a serious poll.

    But they wouldn’t waste money doing a poll like this if they were not close, they would be using their money elsewhere. So this is one where they think they may be able to hold on.

    Lynne has been one of the Lib Dems using Labour’s anti immigrant leaflet to great effect in her seat that has a large BME population. It would be interesting to know if that is working in moving voters from Labour to Lib Dem. I’ve been looking at those Populus monthly consolidations, and also the Ashcroft mega poll, and no indication there that the BME vote is moving away from Labour.

    Unfortunately this poll doesn’t have ethnic weighting or cross breaks neither does Ashcroft’s.

    Zack, any idea?

  26. ” I would advise people to ignore them because of potential publication bias. The Lib Dems have commissioned about 120 of these polls, they’ve released one. I don’t think it overly cynical to ponder whether they may have chosen to release one putting them in a relatively good position, rather than one showing them getting a soaking”
    __________

    Excellent advice AW.

  27. BME?

  28. I would like to see some research on the different methods of asking the constituency question. I quite like the named candidate idea as it’s closer to what the voters will see on their ballot papers, which has a nice feel to it. I don’t know of any evidence that it actually works better in reality, but it would be nice if it did – parsimonious.

    Obviously it hits a stumbling block if some of the parties in a constituency haven’t named their candidates yet and the incumbent’s advantage will probably decrease as the election approaches and voters learn more about the others.

  29. BME = black and minority ethnic

  30. Pete – Black and Minority Ethnic

  31. PETE
    BME?

    Black and minority ethnic. Or Body Modification Ezine if you’re into poking things through/chopping bits off your body.

  32. ‘Boring Methodological Exchange’ – where people discuss polls they like/dislike.

  33. Bloody Minded Electors.

  34. I think you pretty much summed it up when you said it was a campaigning tool – you’re right though; they wouldn’t be going to any effort unless they considered it winnable.

    I think it’s no secret that at any London seat in the North of London, it’s pretty much being suggested that we go help Lynne.

    She’s made huge moves on ending FGM, and was the major driver to gay marriage becoming law – I think she deserves the help. :)

  35. Just on observation, of the main three parliamentary parties, I’d say the LibDems are the most susceptible to confirmation bias. A scout thru’ LibDem voice will give good examples, but a fairly glaring example has been their failure to properly acknowledge and take in their wipeout at the Euro Elections nearly a year ago.

    Whilst pollsters are showing the LibDems anywhere between 5% and 12% lately, I think their presumption of 29 seats or so this coming May is over-confident. I think they should consider themselves very lucky if they have 25.

  36. RIP Lib Dems.

    If this is the only thing they can release, then just think of the carnage they aren’t telling/ hiding from us.

  37. I suggest that Lib Dems do better where they face the Tories because many of their votes come from tactical voting by Labour supporters in seats where Labour are unlikely to win (and to a lesser extent by Tory supporters in Labour seats). Their true support at the 2010 election may well have been a half or less what they actually achieved hence the apparent dramatic collapse now. They will probably poll well above their projected levels with tactical voting.. Without that, they will be virtually wiped out.

  38. worrying if thats the only poll they’re confident in showing…

  39. Yes here in Hornsey and Wood Green the local Liberal Democrats tend to be rather ‘creative’ with facts and figures. Opinion polls of course have sampling errors but there is a set of methodologies that really are standard practice for respected polling organisations such as Survation. The interpretation of this particular poll is entirely down to the Lib Dem local organisation I think Survation should distance themselves from it. Finally the local Lib Dems are promoting these questionable results big time and even with their partisan use of the polling questions they still come out trailing Labour. I have always considered opinion polls as at best snapshots but in this case it is preferable to ignore the poll altogether.

  40. Actually, the error on the difference would be 7%.

  41. A political party doing something with the hope of presenting itself in a favourable light?

    Unthinkable! How dare such a party be allowed to operate. Xenophobes, fascists, separatists and ageing paramilitaries we can handle, but good grief, not spin!

    Seriously though, unlike the other differences wrt Ashcroft polling that AW pointed out, naming MPs is probably the right way to go as part of an ‘in your constituency’ question. Personal votes are by definition votes for a person in spite of preferring another party, and are difficult to fully capture without naming the person.