Ipsos MORI are still working on the review of their polling methods they launched after deciding their London polls had indeed over-estimated Labour support in the mayoral race. Avid watchers of polls will probably have spotted that their political monitor never surfaced last month – I understand they carried it out, but aren’t publishing any figures until the review is completed.
In the meantime Mike Smithson has the lowdown on the first adjustment MORI are planning on making – having reviewed their samples MORI have found they have been over-representing public sector workers.
As regular readers know, for reasons that no one knows for sure – though it’s probably something as simple as how busy people’s lifestyles are or a simply attitudinal difference towards answering the door or phone to a stranger and giving up 20 minutes to answer impertinent questions for no reward – raw samples tend to over-represent Labour voters.
ICM and Populus get round this by weighting samples according to how respondents voted last time, which we call “past vote” or “political weighting”. While on paper this is a simple task (look up the 2005 election result, weight the sample to match. Finished), in practice it isn’t, because it has been proven that people do not accurately recall how they voted at the last election. When deciding what figures to weight past vote to, ICM and Populus therefore have to guess what level of “false recall” to factor in.
MORI have always recoiled from this form of weighting because of the “false recall” problem – they worry that the level of false recall could change suddenly in response to events, and weighting by past vote could end up dampening down genuine changes of opinion. The problem is that without some form of political weighting the samples remain biased to Labour, the standard demographic weights on age, gender, tenure and so on don’t correlate strongly enough with voting intention to cancel it out.
If their samples were over-representing public sector workers, then this may be a solution. It’s an area where people should be able to give accurate answers, the proportion of people working for the public sector doesn’t change vastly from month to month, it should be possible to get relatively reliable targets to weight to and – most importantly – it is a factor that appears to correlate with voting intention, so correcting it would serve to reduce or eliminate the Labour bias.
We await the full results of MORI’s review to see what effect it has. It will also be interesting to see if this skew towards public sector workers in present in other companies samples. It is far from a given that quota sampled face-to-face samples will have the same biases as quasi-random phone samples – they are obtained in different ways, but it is at least worth a look.
Filed under: Methodology