Time to dish out another of the much sought after UKPollingReport “Crap media reporting of polls” awards. The Daily Record is proudly reporting that “Support for Lib Dems in Scotland down to just 3%, according to new poll”. I thought for a moment I has missed a new Scottish poll (hardly unlikely), or that Scottish commenters on UKPR had missed a new Scottish poll (much less likely). Alas not – the Daily Record has, in fact, precariously balanced the headline upon the Scottish cross-break in a single Angus Reid survey, consisting of 161 respondents.
The Daily Record’s article does at least mention this later in the article, and puts it in the context of Scottish cross breaks from other polls that, er, don’t show the Liberal Democrats at 3%, rather suggesting that they’ve cherry-picked a single outlying figure from extremely volatile cross-breaks to make a good headline. Cue lots of politicans cherry picking their own favoured cross-breaks to show how wonderfully they are doing. It’s down to whichever beleaguered toiler was on the Lib Dem press office that day to give the only sensible comment in the piece “It would be ridiculous to draw any meaningful conclusions from this poll given the minuscule Scottish sample size” (and that, one cynically observes, is probably only because no cross breaks look pretty for the Lib Dems).
I can only repeat the same thing I’ve repeated a million times – for fine tuned questions like voting intention, where are couple of percentage points make all the difference, individual regional cross-breaks are pretty much worthless. Sample sizes are typically very low, meaning large margins of error (in the case of this poll, about plus or minus 8 points) and numbers will jump about wildly from one poll to the next.
Equally, polls are normally weighted to be representative overall, rather than within individual crossbreaks – a poll that is representative of GB overall may have, for example, too many Tories in one region and not enough in another. Hence while aggregating lots of crossbreaks together will get you over the hurdle of small sample sizes (and is certainly better than looking at the crossbreaks in individual polls), it won’t magically transform aggregated crossbreaks from a GB poll into a properly sampled and weighted Scottish poll.
(*Well, strictly speaking they could be, since the whole point is that we don’t have any recent proper Scottish polling to judge it upon)