In the last few days a couple of people have asked about local government by-elections and whether we can tell anything about national levels of support from them, and I promised I would write a post about it. I have always been dubious in principle about whether local government by-elections can be used to gauge wider public support. The proportion of people who vote in them is vanishingly low and it seems likely that they will be strongly influenced by local issues, the personality of the candidates and the effort the local parties put in than the standing of the national parties. However, they are often put forward by people claiming to mean something, they are an expression of support and they do involve thousands of people a month, so I thought it worth crunching the figures properly and seeing if we can find a connection.

Local by-elections have, for many years, been carefully archived by Keith Edkins here, using the weekly info from the Association of Liberal Democrat Councillors. What I’ve done to test if we can do anything with local government by-election to predict actual elections is take the local by-elections in months running up to the last two general elections, and see if it is possible to build a model that predicts the general election result.

I can see two approaches to trying to relate local by-elections to national levels of support. We can either look at the average changes in the vote in local by-elections – who is on the up or down, or we can look at the aggregate shares of the vote and treat all the votes cast as akin to a big opinion poll.

Taking the changes in vote first, what if you take the average change in the vote for each party in each by-election and use that as a prediction? Obviously we can’t take whatever swing this produces and apply it to the last general election, since the dates don’t line up (if you had a by-election in 2004 in a ward last fought in 2003, you couldn’t apply the swing to the 2001 election, or you’d miss any change in support from 2001 to 2003). Instead we need a baseline for when the seat was last fought.

As many readers will know, Professors Rallings and Thrasher produce a notional figure for nationwide support at each local elections, which takes a large number key wards and projects them across the country. The soundest method I can find in theory, therefore, is to take the change in the vote for each party since the council ward was last fought, apply those changes to the notional nationwide figures from that set of local elections so you’ve got a projected national share of the vote for each individual by-election…then average them all. Unfortunately it doesn’t actually work.

In 2001 it would have given us CON 37% LAB 29% LDEM 30%
In 2005 it would have given us CON 38% LAB 29% LDEM 31%

In both cases it underestimates Labour, overestimates the Conservatives and grossly overestimates the Lib Dems. Of course, some local by-elections are obviously not good indicators. If Labour put up a candidate in a seat they didn’t contest last time, it doesn’t actually mean they have gone up by 15%. If a strong independent or minor party candidate suddenly stands, or ceases to stand, it have have a dramatic effect on the vote. By next plan therefore was to take only local by-elections where the 3 main parties had both stood at the by-election and at the previous election. I also tried excluding by-elections where a minor party had changed its level of support by more than 10 percentage points.

In 2001 (all three parties) CON 36% LAB 30% LDEM 29%
In 2001 (all three parties, little minor party impact) CON 35% LAB 30% LDEM 30%

In 2005 (all three parties) CON 37% LAB 34% LDEM 29%
In 2005 (all three parties, little minor party impact) CON 35% LAB 32% LDEM 29%

As you can see, its a bit closer, but still bears very little resemblence to the actual general election results. In both cases it underestimates Labour support and grossly overestimates Liberal Democrat support. All this suggests one of two things. Either Labour are disproportionately bad at getting their supporters out to vote in local by-elections, while the Lib Dems are disproportionately good OR lots of people who support Labour at national elections vote Lib Dem in local by-elections.

Once we know that though, can we factor it in and make a model based on local by-elections? If the bias against Labour and towards the Lib Dems was constant, maybe we could – at the crudest level we could take 10 points off the Lib Dems and add it to Labour! But as we’ve seen, in 2001 it was about 10 points, in 2005 it was about 6 points. Using this method even the movements in the projection don’t reflect changes in General elections support – from local by-election result’s we could have predicted that the Lib Dems would do *worse* in 2005 than they did in 2001, when actually they increased their support by 4 percentage points. Labour were doing better in local by-elections in 2005 than they had in 2001…yet their support fell 6 points at the 2005 general election. A model based on change in support in local government by-elections therefore seems to be a non-starter.

So, what about the superfically cruder method of adding up all the votes and looking at the shares?Where by-elections happen is, of course, random, so it is possible that the luck of the draw will produce by-elections all in safe Labour wards, or all in Tory wards. However, normal probability means it won’t normally happen. With a decent sized sample of by-elections we should have a good spread across the country.

Starting at the simplest level, if you take the total of all the votes cast in those elections then it certainly doesn’t reflect the levels of support in the general election. In 2001 it showed the Conservatives winning, in 2005 it showed Labour in third place. Any method must be more complex that that. If again we take only contests where all three of the main parties stood, the results are:

Sum of local by-elections Jan-May 2005: CON 33%, LAB 26%, LDEM 31%
Actual general election result 2005: CON 33%, LAB 36%, LDEM 23%

Sum of local by-elections Jan-Jun 2001: CON 32%, LAB 30%, LDEM 26%
Actual general election result 2001: CON 33%, LAB 42%, LDEM 18%

Sum of local by-elections Jan-May 1997: CON 28%, LAB 37%, LDEM 30%
Actual general election result 1997: CON 31%, LAB 44%, LDEM 17%

The Conservative support in local by-elections here isnt actually that far off, but Labour are once again underestimated and the Lib Dems grossly overestimated. Neither, alas, is there a steady relationship between the two sets of numbers – the Conservatives were doing 5 points better in local by-elections in 2005 than they were in 1997, but their vote at the general election was only two points higher. The Lib Dems did 4 points worse in local by-elections in 2001 than 1997… but increased their general election support. Here too, there really doesn’t seem to be any sort of strong relationship between local by-election results and general election performance.

None of this is new territory of course- Professor Rallings and Thrasher have been trying to crack this particular nut for many years, they’ve had some success at predicting local election results using it (though they’ve had some failures too) and did call the 1997 election right using it, but as far as I’m aware, they’ve never been able to get a model using local by-election results that consistently predicts general election voting intention – the simple fact is that Liberal Democrats always do better in local government by-elections than elsewhere, Labour always do worse, but the amount Labour do worse and the Lib Dems do better isn’t constant, rendering them of very little use in predicting general election support… unless you know better…

The last three polls, from ICM, Populus and ComRes, have shown slight variation but have all been reporting the same trend: Labour collapsing further and the Liberal Democrats benefitting. The Sunday Times’s YouGov poll however paints a far more static picture. The topline figures, with changes from YouGov’s last poll, are CON 44%(+1), 32%(nc), 14%(-2).

So where ICM, Populus and ComRes have Labour shifts to the Lib Dems, YouGov have pretty much no change (the 2 point drop in the Lib Dems would probably mean nothing anyway, prior to their last poll YouGov had the Lib Dems at 14%-15% for 10 polls in a row).

This obviously begs the question of who is right. Are the Lib Dems up or not? Well, some pollsters are more volatile than others, but ICM and YouGov both produce very consistent figures. We are used to ICM and YouGov showing divergent Lib Dem figures – there are various possible reasons for this but one can normally expect a couple of points difference in their reported level of support. They do tend to show the same sort of trends though – when ICM show the Lib Dems high by ICM’s standards, YouGov show them high by YouGov’s standards (and vice-versa). Besides, eight points is well beyond the normal difference between the two companies.

Normally I would be very suspicious of any change that doesn’t have an obvious explanation – public opinion doesn’t magically move by itself, people react to events. If a party jumps up by a third in the polls when they haven’t really done much or got significantly more publicity than usual, it should ring alarm bells. A six point change in the polls is the sort of thing that normally only happens in either rogue polls, or in response to something big.

For that reason, if it was just the two polls my expectation would be that the ICM poll would turn out to be a freak result, and the YouGov one correct…but in this case the trend in ICM is supported by similar findings from ComRes and Populus. It might not have an obvious explanation, but as I said in a comment on a previous post, perhaps it’s just that Labour’s continuing collapse in support was starting to eat into those people who would never consider voting Tory and see the Lib Dems as their natural alternative. Looking at the details of ICM and ComRes part of what also appears to have happened in both of them is that people who voted Lib Dem in 2005 are much more likely to vote Lib Dem now than a month ago (Lib Dem “voter retention” has gone up from 64% to 72% in ICM, 69% to 84% in ComRes).

Of course, it may just as well turn out that the Lib Dem bounce proves illusionary and the YouGov poll is shown to be right. We won’t really know for sure until we see what happens in the next ICM (should be this week in the Guardian) and YouGov (probably in the Telegraph at the end of the month) polls. We could even, theoretically, find that the difference persists for some methodological reason… but I doubt it very much, if it does we’ll worry about it when it happens!

The Populus poll yesterday provoked rather a lot of comment on support for “others”. They are up to 13% in this month’s poll, from around about 9% last month. Most of the comments about it are pondering whether this is UKIP and the BNP (in fact, it’s provoked a bit of a barney over on LabourHome.)

Does it mean anything at all? Well, yes and no. The graph below has the support for other parties in Populus’s poll since Sept 2007 when they made a slight change to their methodology for measuring “other” support (the graph is based on the figures from Populus’s table before their topline adjustment, since they don’t always provide adjusted figures for “others”).

You’d have to be a better man than I to pick any obvious trends out of that (and to people who occassionally ask why I don’t do graphs for other parties – this is why). As you can see, support for “others” in polls bounces about from month to month (within their own tiny margins) and there are no obvious trends there. The BNP is one of the parties to have benefited this month, but not to an extent that it necessarily means anything at all, they were just as high last summer.

Separately, the levels of support for others don’t mean much at all. Things are slightly more interesting though if you view them together. From November 2007 to October 2007 Populus had the total support for other parties at between 11% and 13%. Then for three months from November 2008 to January 2009, roughly co-inciding with Gordon Brown’s bailout bounce, it fell to between 8% and 9%. Now the bounce has subsided the others are back up to 13%.

My guess is that that high others score contains lots of otherwise Labour voters who are annoyed with the government, went back to backing Labour after the bailout, and have now drifted off again.