Last week there was some fuss over the Times’ decision to make their frontpage headline a story about a Populus poll that put David Davis ahead of David Cameron amongst Conservative voters. The actual number of Tory voters asked the question in the poll was only 122, meaning that the margin of error was 9% and, therefore, David Cameron could actually still be ahead.

The problem is that, while a poll of 1,000 has a small enough margin of error (3.1% in fact) to draw reliable conclusions, if you look at the detailed figures of what, for example, male respondents said, you need to take just the number of men who did the survey, meaning you get a bigger margin of error. If you want a really atrocious example – take a look at the BBC today.

The BBC have commissioned a poll from ICM on faith. The poll found that 14% went to a religious service once a week or more, while 40% went once a year or less. 72% thought it quite or very important that British society continues to be based on Christian values.

Asked about how much people thought they understood about the main religions in the UK, 51% thought they understood a lot about Christianity, with a further 36% saying they understood a little. Most people were less familiar with all the other faiths in the UK – only 18% of people thought they understood a lot about being a Muslim, with 35% saying they didn’t understand it at all. The figures for Judaism were almost identical, followed by Hinduism and Sikhism, which was the least understood religion – only 11% thought they understood a lot about it.

Finally ICM asked about British attitudes to Islam – 33% of people said they had a positive image of Islam, while 20% said they had a negative image. 19% said the London bombings had made their perception of Islam more negative.

This is all well and good – the problem is the BBC report here, which breaks down the survey to include such findings as “37% of Muslims said they knew nothing about Judaism” and “significantly, 31% of Jews said they knew nothing about their own faith”.

The sample only included 28 Muslims and 5 Jews, so the respective margins of error are 19% and a whopping 44% – in other words, the breakdowns of the answers by religion (other than for Christians) are entirely worthless. The moral is that, while it’s a jolly good thing to look at the tables on the websites of the polling companies and draw your own conclusions from the tables, do remember to look at the base for each demographic break to see if the sample is actually big enough to be meaningful before jumping to conclusions…especially if you are a major public sector broadcaster ;)


Comments are closed.