
The difference political weighting makes
Both Mike Smithson and I regularly criticise the BBC’s Daily Politics for commissioning political polls that are not politically weighted. The vast majority of political polls include a voting intention question and (with the exception of MORI, who don’t believe past vote is suitable for weighting in the first place) these are always weighted to be representative politically as well as demographically, normally with reference to how people claim they voted back in 2005.
In contrast, the BBC invariably commission polls without voting intention questions (since their producer guidelines say they need special permission to do so) and these are invariably NOT politically weighted, presumably because past vote weighting requires an extra question and therefore cost more. Does this matter, does it make a difference? Well, for this week’s Daily Politics ComRes asked exactly the same question in their non-politically weighted poll as they did last week in a politically weighted poll for the Indy.
In the ComRes poll for the Independent, which WAS weighted by past vote, Brown & Darling were most trusted by 35%, Cameron & Osborne were most trusted by 33%, Clegg & Cable by 7%.
In the ComRes poll for the BBC’s Daily Politics, which WASN’T weighted by past vote, Brown & Darling were most trusted by 32%, Cameron & Osborne were most trusted by 23%, Clegg & Cable by 4%.
As you can see, while Labour are much the same, there is a huge 10 point difference between the two Conservative scores. It’s possible, of course, that confidence in Cameron & Osborne has massively slumped in the 7 days between the two sets of fieldwork, but this doesn’t seem likely given subsequent national polls haven’t shown this drastic slump. In fact I’m certain it’s because one was politically weighted and one wasn’t. Past vote weighting of phone polls invariably involves making the sample more Conservative and less Labour and the cross-breaks of the ComRes poll for the Indy shows that answers to this question are very closely correlated with voting intention – 72% of Tory voters answer Cameron & Osborne and 80% of Labour voters answer Brown & Darling.
If one believes that political weighting is necessary to get a representative sample for voting intention, then it should also be necessary for other polls where people are asked to compare the political parties or where answers are likely to be aligned to party loyalty.
UPDATE: Andrew Hawkins of ComRes has asked me to link to his comparison a month or two back of a poll he did when political weighting did not make a difference, and I happy to do so here. He has also let me that political weighting made a 2 point difference to the question when it was asked in the Independent (I’m not sure if that is the Conservatives 2 points higher, or the lead 2 points different, or whatever). Presumably recalled past vote was not asked for the BBC version of the question, so we can’t tell how much difference that would have made to the answer.