Some of the internet got very excited over a LibDemVoice poll earlier this week showing 46% of Lib Dem members don’t want Nick Clegg to stay on as party leader at the next election.

The question itself was rather more nuanced than some of the comment upon it suggested – it gave respondents options of Clegg staying for the election, stepping down just before the the election or stepping down sooner than that (and also separate opinions for stepping down as leader and deputy PM). Most of the 46% of Lib Dem members that wanted Clegg to go were happy for him to stay on for now – 32% of respondents wanted him to step down as party leader at some point, compared to only 14% who wanted him to step down in the next year. It suggests to me that this is more about Lib Dem members thinking Clegg is probably not the leader to get them votes at the next general election, rather than a sign of unhappiness or opposition to him per se.

While I’m here I should write quickly about how representative the polls on LibDemVoice are. Stephen Tall and Mark Pack don’t make huge claims about representativeness and are always quick to stress that they can’t claim they are representative. This is admirable, but is sadly not a carte blanche, as however much the person doing a poll hedges it with caveats and warnings these are rarely picked up by third parties who report a poll and are more interested in making it newsworthy than reporting it well.

That said, I think they are actually pretty worthwhile. They have the huge advantage of being able to actually check respondents against the Liberal Democrat member database so we can be certain that respondents actually are paid up Lib Dem members and not entryists, pissed off former members, other parties supporters causing trouble, etc. LDV also have access to some proper demographic data on the actual membership of the Lib Dem party, so while their sample is unrepresentative in some ways (it’s too male for example), they know this and can test to see if it makes a difference. They have also compared it against some YouGov polling of Lib Dem members which had very similar results, and actual Lib Dem party ballots, which had excellent results in 2008 and rather ropey ones in 2010. Mark Pack has a good defence of them here.

Of course, there are caveats too. The danger for such polls is if they end up getting responses disproportionately from one wing of the party or another, from supporters or opponents of the leadership. I am not a Lib Dem activist so such things may be over my head, but from an outside perspective the LibDemVoice website doesn’t seem to be pushing any particular agenda within the party that might skew the opinions of their readers or which party members take their polls. If reading LDV does influence their opinions though, it could obviously make respondents different to the wider Lib Dem party (for example, here Stephen suggests Nick Harvey’s increase in approval ratings could be the effect of making regular posts on Lib Dem Voice, which would indeed be a skew… but not on a particularly important question!)

I do also worry about whether polls that are essentially recruited through online party-political websites or supporter networks get too many activists and not enough of the armchair members, or less political party members (not an oxymoron, but the type of party member who joins for family or social reasons, because their partner is a member or because they want to contribute to their local community through being a councillor and the party is really just the vehicle).

All that said, while they aren’t perfect and Mark and Stephen never claim they are, I think they are a decent good straw in the wind and worth paying attention to, especially given the verification of whether respondents are party members.

Lord Ashcroft has released some more polling on gay marriage, asking a question on whether people would be more or less likely to vote for a party that legalised same-sex marriage.

As regular readers will know, I have an awful lot of reservations about would X make you more or less likely to vote Y questions. To tick them off quickly –

(a) people tend to use the question to register their support or opposition to a policy, regardless of whether it would actually change their vote
(b) people are extremely poor at understanding the drivers of their own voting behaviour anyway
(c) if it asks about a specific party, people who are already voting for that party regardless say it makes them more likely to vote Y, people would would never vote for them anyway say it makes them less likely to vote Y. Neither of these groups matter
(d) by singling it out it gives the issue being asked about a false prominance, when actually lot of other equally or more important issues would be there influencing people’s votes
(e) more or less likely is a pretty low bar. It isn’t saying people definitely would or would not vote Y if X happened, just more or less likely. It’s pretty easy to tell a pollster that to indicate your support or opposition to a policy, it can be a more difficult decision when it comes to an actual ballot box

Despite these problems, more or less likely to vote are much beloved of campaigning and pressure groups as it makes whatever pissant little issue they are campaigning on seem like something incredibly important that will decide elections.

Anyway, this isn’t to particularly criticise Ashcroft’s question, since they’ve done all they can to try and get a decent question out of it – they gave people the option of saying they supported or opposed gay marriage, but that it wouldn’t affect their vote and they looked separately at current Tory voters and potential Tory voters.

Overall, Ashcroft found people in favour of gay marriage by 42% to 31%, with 27% saying they had no real opinion either way. People who were opposed to gay marriage were more likely, however, to say it would affect their vote – overall 10% of people said they were more likely to vote for a party that supported gay marriage, 12% said they were were less likely.

It becomes more interesting when we look at the crossbreaks. Amongst people who voted Tory in 2010 and would still vote Tory today the vast majority say the issue makes no difference – 6% say it would make them more likely to vote for a party, 9% less likely to vote for a party. Amongst lost Tory voters, who voted for the party in 2010 but wouldn’t now 26% say supporting gay marriage would make them less likely to vote for a party and only 4% more likely – this fits nicely with a support that the Conservatives have lost to their right and UKIP.

However, there are two sides to the equation. Looking at the votes the Conservatives have gained since the election 15% say they are more likely to vote for a party that legalises gay marriage compared to 11% less likely. Looking at those who are not voting Tory but may consider it, 12% say they are more likely to vote for a party that supports gay marriage compared to 9% less likely.

Of course while Ashcroft and his pollsters have done their level best to write a good question, most of the caveats above still apply – questions like this give undue prominance to an issue of low saliance and even wording like this it probably grossly overestimates the importance of the issue in voting intention. It does however, as Ashcroft concludes, demonstrate that the effect of gay marriage on voting intention is not all one way.


The Boris bandwagon rolls on, and an ICM poll for the Sunday Telegraph tonight apparently has another question trying to measure whether the Conservatives would do better with Boris Johnson as leader. There are two things to consider with hypothetical “who would you vote for if X was leader” questions.

The first is that they need to be exactly comparable. The difference between voting intention with two different people as leader of a party is often only a few points. However, adjustments like weighting by likelihood to vote or reallocating don’t knows can also make a couple of points difference, so if you want to be confident the difference is due to the leader the need to be done exactly the same way. If the main figures are weighted or filterted by likelihood to vote, they need to be weighted by likelihood to vote (ideally asked separately), if there is a squeeze question or don’t knows are reallocated in their main question, the same needs to happen in the hypothetical questions.

Trickier to control for is the question itself. Normal voting intention questions don’t mention the party leaders, so if asking how people would vote with Boris as Tory leader increases the Tory vote by 2 points we can’t conclude that he’d do better than Cameron without checking mentioning David Cameron as Tory leader in the question wouldn’t do the same. This is why when YouGov run the questions they ask a control question including the names of the current party leaders.

The second thing to consider is quite how hypothetical these questions are! In many cases we are asking about politicians who the general public know very little about – apart from very well known politicians like party leaders and Chancellors of the exechequer many other ministers – even cabinet ministers – are almost complete unknowns to the majority of people. Even when a politician is relatively well known, like Gordon Brown pre-2007 or Boris Johnson now, people answering questions like this don’t know what they would do as a party leader, what sort of mission and narrative they’d set out, what policy priorities they’d follow, and all these things could change how they are viewed.

However, that doesn’t necessarily mean questions like this are never useful. Back before Gordon Brown became Labour leader polls like this consistently showed him doing less well than Tony Blair. At the time I made all the same caveats as above, but said in the specific context of Gordon Brown it probably was showing that Brown would do badly because of why people gave him negative ratings. The polls said people saw him as competent and efficient and capable… but they didn’t like him. If people had seen Brown as incompetent or inexperienced he could have changed impressions in office, but those were already positive. The polls were telling us that his problem was a negative that was difficult to change, just not being likeable.

So to Boris. What can we tell from hypothetical polls about him? Well, I haven’t seen the ICM poll yet, but YouGov have done two hypothetical polls about him. The first in May showed Boris doing basically the same as David Cameron. The second a week or so ago had Boris doing 5 points better than Cameron, presumably because of the effect the Olympics has had on how Boris is seen. We shall see if ICM shows the same sort of pattern.

Is this really meaningful? Well, as Gordon Brown seemed to do badly simply because people didn’t warm to him personally, Boris Johnson seems to be an opposite case – he seems to do well because he is likeable and eccentric. It’s an open question to what extent that would transfer were him to become Prime Minister or Conservative leader – a politicians ability to come across as likeable and to connect to people seems to be innate to some degree, so would probably benefit Boris in any role. On the other hand, being seen as a bit of a buffoon is not necessarily on the job description of PM. Would something that seems like a wizzard prank in a hypothetical opinion poll seem rather less funny in an actual election? We don’t know.

A more concrete caveat to keep in mind is to remember that all these Boris quesions are being asked in the midst of the London Olympics, Boris’s big moment in the sun. Before the Olympics the polls didn’t suggest Boris would do any better than Cameron. I’d wait until the publicity around the Olympics fades before drawing any long term conclusions…

A month ago I wrote a guide to How Not to Report Opinion Polls. I have a history of starting regular features and then failing miserably to deliver them, but I am at least going to try to come back to it at the end of every month and highlight particularly poor reporting in the weeks just gone by.

For July I’m going to start with this report from the Independent, claiming that a ComRes poll shows “Dramatic change as two-thirds now support GM crop testing” and that “Public opinion appears to be shifting in favour of the development of genetically-modified crops, according to a ComRes survey for The Independent”.

ComRes found 64% of people agreed with a statement that “Experiments to develop genetically-modified crops should be encouraged by the Government so that farmers can reduce the amount of pesticides they use”. However, the article doesn’t mention any past results that it can be compared to in order to justify the claim that there has been a dramatic turnaround in support for GM crops.

Historical polling on the issue by MORI here does indeed suggest a much lower level of support for GM crops, but these differences could easily be explained by the question wording – MORI was asking things like “How strongly, if at all, would you say you support or oppose genetically modified food?” while ComRes’s statement specifically links the development of GM crops to a positive outcome of reducing the use of pesticides.

To see the impact asking different questions could make, look at this more extensive polling on the issue by Populus. Asked a generic question on whether or not GM food should be encouraged 27% of people say yes, 30% no. However, if you look down the survey to page 38 it asks specifically about whether people are supportive of using GM wheat to repel aphids and reduce the need for pesticides and finds 58% of people are supportive of this specific use. It seems plausible that the reason ComRes found such high support is not because of some great shift in support, but because their question specifically mentioned a popular potential outcome from GM.

If we want to see whether or not public support for GM actually is growing we need to have a question that has been asked consistently over time. This is surprisingly difficult to find – MORI don’t seem to have asked the question above again since 2004. The best I can track down is the Eurobarometer polling here, which every 3-5 years has asked if people agree that GM food should be encouraged. As you can see from the table on the first page, there is no obvious trend in the UK’s answers, support for encouraging GM food has moved by 45% to 25% to 35% over the years. Certainly the picture it shows is not one of a strong trend towards people supporting GM food.

(The Populus poll, incidentally, asked a similar question to the Eurobarometer question, but they can’t be directly compared either, not least because 43% of people told Populus they “neither agree nor disagree”, an option that the Eurobarometer did not offer. This, in turn, was misreported by the Daily Mail back in March.)

Once again, the lesson is to look at the polls in the round, not to take a single finding out of context, especially when it that question is one that is likely to put an issue in a particularly good or bad light. If you are looking at trends over time, you should always compare apples with apples. If two significantly different questions give different results it is as likely to be down to different wording as it is to a change in opinion, especially in cases like this.

Olympic poll boosts

No – don’t get excited – tonight’s poll doesn’t show one. Today’s YouGov poll for the Sun has perfectly normal topline figures of CON 34%, LAB 42%, LDEM 10%, UKIP 6%. Once again it is well within the range of the 9-10 point leads that YouGov have been showing for the past couple of months.

As yet there is no sign of any Olympic effect. I wouldn’t necessarily expect one, but I wouldn’t rule one out either, in the same way we saw a (brief) Jubilee effect straight after the Jubilee weekend. Two things to remember:

First, why these things happen. After the Jubilee I saw several comments saying how absurd it was that it affected the polls. Why would someone think “Oh look, the Queen has been there a long time, better vote Tory”? Well, that would be absurd, but the reasons things like this can affect the polls is more straightforward. First there is a general feel good factor – if people feel generally more positive about the country and the way things are going they may be more likely to support the incumbent government. Secondly, and in my opinion probably more significant, is the absence of bad news – looking back over recent months an average month for the government has at least a few party rows, a couple of controversial policies, a spattering of bad news, perhaps a rebellion and, on recent form, a U-turn or two. With the Olympics totally dominating all news coverage the next couple of weeks will have significantly fewer of all those things, so you can imagine how the absence of bad news may have an effect.

Secondly, remember that if there is an Olympic effect on the polls it will be probably be temporary. The Jubilee effect, if it ever existed, only lasted a couple of days. A positive feeling from a big national event fades; once the Olympics and silly season are over the normal news agenda and the trail of bad news stories that governments have to cope with will resume. If the Olympics does have an effect, it is unlikely to be long lasting.

UPDATE: I haven’t seen it mentioned on twitter or by the Indy, but the voting intention fgures from ComRes’s monthly poll have now appeared on their website here. Topline figures, with changes from their previous telephone poll a month ago, are CON 33%(nc), LAB 44%(+2), LDEM 10%(-3), Others 13%(+1). Certainly no sign of any Olympic boost there either!