Ipsos MORI have released a fascinating chunk of polling on how people’s perceptions differ from reality – report here and full toplines here.

A lot of the poll won’t be anything new to regular readers. We’ve all seen previous polls showing that the general public underestimate this or overestimate that, especially on figures that we really have no reason to expect the average man in the street to know. The public underestimate the average salary, they overestimate the proportion of children living in poverty and the number of over 65s. They underestimate turnout at the last election, and overestimate the proportion of crime that is violent. In all these cases though, why would we really expect people to know – they are the sort of thing you look up if you need it.

A few cases are also probably cases of people answering slightly different questions to the ones MORI asked or intended to ask. For example, on average people thought 34% of people in Britain were Christians, whereas 59% of people identified themselves as Christian in the census. Did people get it wrong? Well, we don’t really know, as while 59% describe themselves as Christian in the census, we know most of them don’t go to church and polling shows many don’t even believe in a God…so exactly how were respondents defining “Christian”? Respondents said that on average that 28% of people in Britain are single parents (my emphasis), when the actual figure is 3%. 28% would be absurdly high once you factor in the proportion of people in Britain who are children themselves or don’t have children… but my guess is that many people have probably mentally parsed that as having asked what percentage of parents are single parents (I suspect they’ve still overestimated it, but not by so much!)

In a few cases though people really do give quite barking answers. For example, on average people think that a quarter of the British population are Muslim (the actual figure is 5%). 5% of people are under the impression that over half of the British population is Muslim. People think that 15% of girls aged under 16 get pregnant each year, when the actual figure is 0.6%.

Some of this is what I think of as “hell in a hand basket” responses. Whatever the reality and whatever the statistics show, people do normally tend to say that things are getting worse and heading the wrong direction. I’ve previously mentioned crime figures and the public’s firm opinion that crime is rising, regardless of the firm evidence that crime has been falling for twenty years or so. MORI tested some true and false statements along those lines in this survey, and found the usual pattern – if there was a false negative statement, people belived it. If there was false positive statement, people didn’t believe it, and vice-versa with true statements. The only true, positive statement that a plurality of people believed was that MRSA infections in hospitals were falling. I expect things like people overestimating rates of teenage pregnancy stem from similar causes.

As well as the familar litany of public ignorance though, MORI also asked some fun questions I hadn’t seen before. One of the statements asked what proportion of people in Britain today are immigrants (that is, were born outside the UK). On average people said 31%, when the actual figure is 13%. MORI then went back to those people who said the proportion was 26% or more (that is, those who were wildly out), told them the official census figure and asked them why they had thought it was so much higher. 23% gave the honest and straightforward answer that it had just been a guess, a further chunk of people said they’d be answering based on what they saw in the media (19% TV, 16% papers – people could give more than one answer) or in their local area (36%). The biggest chunk clung to their view anyway, with 56% saying the census must be missing out on illegal immigrants, and 46% saying they still believed it was more than 13%. People are very keen to defend their faulty opinions, even when, as in the this case, it is probably just an on the spot guess they’ve made in an online survey.

Another question in the MORI survey repeated a question that YouGov asked for the TUC last year, asking people what proportion of benefits are claimed fraudulently. The actual answer is about 0.7%, but on average people said 24% (very similar to the 27% that YouGov got). MORI then asked people what sort of fraud people were including when they answered the question. 42% of people said they were including people from abroad claiming benefits, 34% were including people claiming benefits who hadn’t paid tax, 32% were including people having children just so they could claim more benefits. So at least part of the reason for the discrepancy is that when some people say people are fraudulently claiming benefits, they are apparantly including “people I don’t much like claiming benefits in a perfectly legal and honest way”.

Another fun question was that after asking people whether the economy was in a good or bad shape, MORI then asked them what figures they based their answer on. The top answer was unemployment (52%), followed by inflation (40%), government debt (38%) and high street shops (32%). GDP figures were 23%. Now, I should add that while this is interesting, the answers are almost certainly complete nonsense – most of us really don’t have a very good idea of how we actually reach decisions and what we actually base our beliefs upon. A glaring ommission from the options, whether or not people would have consciously picked it, is “whether the TV, radio and newspapers tell me the economy is getting better”. Nevertheless, it would be an interesting experiment to take economic optimism figures for the last forty years and plot them against unemployment, inflation, growth and so on and see which, if any, does correlate the most. I suspect we wouldn’t see a consistent pattern, as the measures the media and political world have fixated upon have changed over time, back in the 1960s and 70s they got het up about the balance of payments, then levels of inflation – these days it’s GDP that seems to be the measure the media focus the most attention upon.

All in all, the poll is a reminder of how public perceptions of the economy, crime, immigration, social issues and, I am sure, many other facets of social policy can have little or no resemblence to reality. When we talk about how improving standards here, or the government missing targets there, might affect people’s votes, remember that people’s perception of those standards, those targets or those changes may be completely at odds with reality.

148 Responses to “The ignorance of crowds”

1 2 3
  1. The real issue that should be debated here (it won’t be anywhere else) is what effect knowledge, as opposed to induced propaganda, debases the democratic principle of one man one vote.

    If voters don’t understand what’s what, even remotely, what value does their vote have?

  2. @ Roger Mexico

    I got 5/10 and I think Anthony gave me 4 of those 5 answers! I didn’t know 16% of the population has twitter accounts and don’t see why I should know!

    @ Others

    I see quite a few disputing the figures. I appreciate some of the figures are to a certain extent debatable (especially stuff like child poverty which is debatable based on an official definition) but the point is you (one respondent above) are arguing 0.7% may not be correct for benefits fraud and you may be right but if the average people are thinking is 24% then that 24% is clearly much more incorrect than 0.7%!

  3. What annoys me is the misleading headlines, I swear it getting worse. The actual newspaper articles are often not badly balanced but the headline is pure propaganda. I can’t say if the left wing press also engages in this cos I only read rightwing papers

  4. Howard – the general public might be less informed about the facts, but if we’re honest, we’re only interested in those that back up our original thoughts.

  5. One thing that no one has picked up yet is there was no control question to make sure folk actually understood percentages

  6. @PaulCroft,

    “How did they treat you in prison?”

    Contrary to popular media portrayals (i.e. the Daily Bigot), I found prison to be rather hellish. I am relieved to be back.

  7. @Roger

    “The real problem comes when the overwhelmingly London-based media (and it’s a bias getting worse) looks around itself and thinks that London is representative of the UK in total.”

    Yes, that was what I meant, but forgot to add that bit.

  8. @Ringo

  9. Drat, misclicked, and posted before I’d finished writing…

  10. @Ringo

    Who is to say only 0.7% of benefits are fraudulently claimed?

    Statistics can say. You already know what the claimant level is. You use statistical measures and survey sampling of the population, to give a good estimate of what the claimant level should be according to that data. Then you use anomaly detection to identify suspect cases, as well as the cases positively identified as fraud. If the anomaly detection and known cases of fraud come within margin of error of the difference between the survey estimate and actual expenditure, you have a confident idea of what the real fraud level is.


    You cheated with that post. You actually know stuff about …. that kind of stuff!

    Ergo, you are not representative of the great British public.

    Ergo, your knowing things disqualifies you from posting on a site devoted to studying the opinions of those who know sod all about anything.

  12. I should note that anomaly detection isn’t really good at identifying specific suspect cases, just suggesting the existence of suspect cases.

    As an example, let’s take the case of pension claims continuing after the death of the claimant. You could use up to date actuarial tables to estimate how many people over 95 there should be in any particular county. If there are substantially more people over 95 claiming pensions in a county than actuarial tables say there should be, either the tables are wrong or there’s some level of fraud occurring.

    The same practice is how casinos know *when* and *where* to start looking for cheats, but it doesn’t tell them *who* is cheating.

  13. Jay,

    Is that how the 07% figure was arrived at?

  14. @Neil A

    I don’t work for the DWP, and I think that the briefing documents on this that would explain what they do to get this figure are confidential, so I can’t say for sure. But it’s what I would do.

  15. I suggest that ignorant people are much more likely to think they know all the answers than better-informed people. The ill-informed will say “25%” when the wise would say “I don’t know.” Therefore, the sample on which these surveys are based is biassed towards the ignorant and their value is correspondingly less.

  16. AW uses the term ‘crowd’. Oh, yes, the madding crowd. We now have, I read, ‘crowd funding’ for their desired porno film.

    We had better be grateful for the increasingly shallow politics that they have in Australia. Perhaps Murdoch did us a favour. The Crowd will never vote for a madman. I’m OK with Blue Peter presenters.

    Bring it on. A brief election campaign with voting online.

  17. Neil A – the DWP report it’s from is here, the method is explained therein.

    Basically though it’s taking a sample of claims, and going and examining them, and getting professional fraud investigators to look at any that look dodgy. As far as I can tell it only includes fraud they detect, I don’t think they include any estimate for undetected fraud.


  18. @Jay,

    In my experience just because something should and could have been done a particular way by officialdom doesn’t mean that it was.

    I am particularly suspicious of statistics about things which seem inherently secretive, or inherently subjective.

    Sadly, I don’t care enough (or am too lazy) to go and find out for myself so I am reduced to asking others.

  19. Thanks AW. Lazy Neil will now read the report you’ve fetched for me..

  20. Yup looks like you’re right. DWP take a lot of trouble to try and identify cases of fraud, but it is only the proven cases of fraud that they include in the figures. They include this interesting proviso;

    The estimates do not encompass all fraud and error. This is because fraud is, by its nature, a covert acti
    vity, complex official error can be difficult to identify and some suspicions of fraud on the sample cases
    cannot be proven. For example, unreported earnings in the informal economy will be much harder to detect
    than those in the formal economy.

    It seems to me that the reviews will probably only catch the low-hanging fruit – those who inexpertly cheat the system hoping it will never be checked.

    So I’d say it is not unreasonable to assume that the real rate of fraud is significantly higher than 0.7%. Of course it’s hard to know how much higher, but given the number of cash-in-hand jobs that are carried out every year, I’d guess quite a bit. People replying to an opinion poll are much more likely to do a back-of-the-fag-packet calculation based on the people they know who’ve fiddled the system, than to appraise themselves of official statistics.

  21. Ambi


    “How did they treat you in prison?”

    Contrary to popular media portrayals (i.e. the Daily Bigot), I found prison to be rather hellish”


    Mmmm…. I am quite sure you enjoyed Sky Sports all day and a life of jollity, made even better by not bein’ allowed to vote or post on UKPR.

    As for hell – that reminds of a joke about the bloke who was shown around hell and told he could choose his own hell-hole. He found a room that seemed by far the best, where people stood up to their ankes in warm sh*t and drank tea.

    “I’ll pop in here ta very much” he said.

    “In you go mate” said the warder “and tea break’s over so back on yer heads everyone.”

    Now that’s what jail SHOULD be like.

  22. Benefit fraud is costing £5.2 billion… £1.6 billion down to administrative errors. Set against the 16 billion of wtc, council tax benefit, pension credit etc, which goes unclaimed.

    Compare that with £17 billion in tax evasion (a fraud level of 3%), and £70 billion tax avoidance.

  23. Actually if you look at the report Anthony linked to, it suggests that the topline figure of 0.7% probably is an underestimate. The reason for this is that the benefits total being looked at includes the State Pension which makes up 48% of the total expenditure. This shows zero loss to fraud, an estimate based on surveying 7 years ago. However this is always going to be a benefit that has low fraud – people’s circumstances don’t change (they remain old) and very few people bury granny in the back garden and keep drawing her pension. Even if they do she’s probably been drawing her pension for a lot longer than you do before you get found out.

    If you look at the Continuously Reviewed Benefits (the ones they sample for fraud every year): Income Support, Jobseeker’s Allowance, Pension Credit, Housing Benefit, these have a higher rate of fraud. But it’s still only 1.8%, compared to MORI’s mean of 24%.

    Using the mean is a bit misleading though, because the figures get skewed a small number of people who say a ridiculously large number. An estimated median would be around 16%. Of those who gave a figure 38% guessed 1-10%. This seems to be a pattern in a lot of the questions – an estimate of a small number being boosted by a minority who give a very large reply

  24. ”Benefit fraud is costing £5.2 billion.’


    The real scandal is [insert thing about which you are incensed.]

  25. @Paul Croft,

    “Mmmm…. I am quite sure you enjoyed Sky Sports all day and a life of jollity, made even better by not bein’ allowed to vote or post on UKPR.

    As for hell – that reminds of a joke about the bloke who was shown around hell and told he could choose his own hell-hole. He found a room that seemed by far the best, where people stood up to their ankes in warm sh*t and drank tea.

    “I’ll pop in here ta very much” he said.

    “In you go mate” said the warder “and tea break’s over so back on yer heads everyone.”

    Now that’s what jail SHOULD be like.”


  26. I’m still pretty sure if you asked the general public what a quarter is when expressed as a percentage that about half of them would get it wrong

  27. That wasn’t as clear as I meant. What I was trying to say was that, when asked about the benefits budget, people will not think of it in terms of including state pensions, even though it makes up nearly half. So the estimate that they are aiming at will be a bit higher.

    It still doesn’t mean that it’s a more valid way of assessing the true value of benefit fraud though. Neil A may suggest that you can make an estimate from the people you now, but he forgets that he knows a lot of criminals. In any case people usually think things in their immediate locality are better than in the country as a whole – there’s a whole section of questions showing this in this very survey.

    Howard – I assume Anthony is riffing off the phrases “The Wisdom of Crowds/the Crowd”:


    which a fairly fashionable concept at the moment (it’s behind that “poll” that ICM due for the Sunday Telegraph for example)>

  28. Alex Harvey

    I’m not enamoured with royalty – does that count? I appreciate I belong to a statistically insignificant group though.

  29. frank upton

    “I suggest that ignorant people are much more likely to think they know all the answers than better-informed people.”

    It’s a variation on Catch 22. Many people think questions MUST have answers.

    The solution is a simple “No thickies” caveat at the outset.

  30. I have never thought benefit fraud was the problem, it’s the sheer level of benefit the state is paint out.

  31. Paying, darn ipad.

  32. RM

    As I am sure you guessed, I was not aware of your reference.

    On madding crowds, I have read all of Hardy’s novels though, bloody depressing stuff, and it all began with Julie Christie (sigh).

  33. @ Rich

    I agree – let’s start with abolishing pensions and introduce workhouses.

    Actually I do think that unemployment benefit is immoral – providing that the government guarantees jobs/retraining to a HIGHER level.

  34. Just in, and fully agree with one of @Howard’s earlier posts. Pretty much all of the examples that AW quotes from this poll are areas where there has been sustained misinformation, misreporting, or willful focus on negatives and neglect of the positives in the media.

    The responses to issues like like crime, immigration, welfare and teenage pregnancies has been fed by a media incapable of accurately providing balanced news that accurately reflects real life. This has a direct impact on people’s lives, as it distorts the entire functioning of democracy.

    We learned quite a lot about how bad our media was in the hacking scandal, but there is something far more damaging and insidious that they are still inflicting on society today, and the quality and veracity of what they produce is an area where we are still blighted.

  35. @Rich

    That’s the problem when you build an economy full of jobs which don’t pay enough to live off, significant levels of unemployment and basic amenities like housing, water and power left in private hands to set at prices they please.

    Or, are you going to argue, against all evidence I’ve seen, that UK benefits are generous?

  36. I have a lot of issues with this polling.

    First off, with roughly half of marriages ending in divorce, it’s not much of a stretch to assume that half of those divorced couples have kids (ie 25-28%)

    Secondly “One of the statements asked what proportion of people in Britain today are immigrants.

    Well it depends on your view of who is an immigrant. The government currently follows this liberal policy of if you’re born here you’re British and that’s where the 13% figure comes from, but actually a lot of people myself included, take a view that actually no the fact you happened to be born here, doesn’t make you British, it just makes you X Nationality who happened to be born in this country. If my child happens to be born in Spain for whatever reason, the government takes the view they are somehow Spanish, but ordinary people see behind this false definition, and that actually they would just be a Brit born in Spain.

    You can pass a law saying everything with four legs is a chair, but we all know at the end of the day that a table is a table.

    And yes illegal immigration is a big factor, the policy wonks in their ivory towers may have some model to predict illegal immigration numbers, but the truth is they don’t know, how can they?. It’s the ordinary people on the street, the ones being polled, who can see, these figures aren’t right.

  37. @ MiM

    I know some people here in Liverpool who consider commuters from the Wirral aliens.

  38. I’d hazard a guess that DWP’s figures for benefit fraud are nearly as fat off the mark as those given by voters.

    I would expect cash in hand work to be the most common form of fraud – it’s also particularly difficult to prove, or estimate statistically, which presumably means the DWP figures only really cover claims where the claimant is not entitled to claim for other reasons.

    Incidentally, this is also a problem with tax evasion and avoidance figures – there are lots of easy ways for folk to funnel money round the taxman without him knowing -.for example, a mate of mine bought a car from his boss (who owned the company). rather than pay out of taxed earnings,
    he didn’t get paid for most of a months worth of hours. It ‘saved’ him income tax and NI, and his boss the employers NI. This sort of crime is near undetectable, and again I can’t see how it could easily be deduced by statistical processes either.

    Incidentally, the figures banded about over tax evasion and avoidance are somewhat misleading. The evasion figures tend to include things like money owed by bankrupt companies (usually not really evasion as such, and normally it’s money HMRC are never going to see), as well as unsettled court cases over avoidance schemes, where if HMRC eventually win, money is paid, and if they lose, it wasn’t owed in the first place.

  39. I note that AW when talking about Christians, has included the people that call themselves Christians but don’t believe in God but has not included the far larger proportion of people who believe in God but don’t call themselves Christians. I believe that this would make up a largish percentage of the population. (I am judging from the people I know and I know quite a good cross-section of the British people.)

  40. Also forgot crime.

    How much crime goes unreported?

    The first two times my family were victims of a crime, we reported it to the police and they were absolultely useless, so when it came to the 3rd and 4th time we didn’t even bother reporting it.

    Also attempted crime can often not be reported, if someone tried to steal from you but failed, the police don’t care, they wonder why you’re complaining, you still have your stuff.

    Then of course we have the strange ways the government counts crime, similar to how they calculate unemployment, there’s all these little caveats that mean “violent” crime isn’t as high as it really is.

    Same with unemployment, so fed up with being unemployed you feel like giving up, well then suddenly your not unemployed.

    Been unemployed for 2 years and have been put on an evening course 1 night a week for 2 hours, suddenly your no longer unemployed your in part time education/training.

  41. I have an issue with the way pregnancies in girls aged 13 – 15 are reported. These girls have been the victims of a crime but the reports generally seem to blame the girls for their situation.

  42. mim

    “If my child happens to be born in Spain for whatever reason”

    It would almost certainly be because that’s where the child’s Mum is at the time.

  43. @Paul

    But that’s your view. In this hypothetical scenario most people would recognize that child actually would be British but legally classed as Spanish. Then articles like this would just laugh at the supposed ignorance of the people.

    Also I’m a little uncomfortable with this view of how the public are somehow the ignorant unwashed masses, because they don’t agree with massaged statistics.

    To me this article just proves the age old sayings of you can get a statistic to prove anything, and also, there’s lies, damn lies, and statistics.

    If I’m on a sinking ship, and the waters coming up, past my ankles all the way up to my neck, forgive me if I don’t believe your statistic saying I’m actually getting dryer since the boat started to sink.

  44. Sun Politics [email protected]_Politics 5m

    YouGov/Sun poll tonight: Labour lead down to 5 points. CON 32%, LAB 37%, LD 11%, UKIP 12%. Is Falkirk beginning to bite?

  45. mim

    Re born in Spain – it was a joke but not that funny, even if you had got it cos I forgot the LOL!

    As for this >>>>>>

    “If I’m on a sinking ship, and the waters coming up, past my ankles all the way up to my neck, forgive me if I don’t believe your statistic saying I’m actually getting dryer since the boat started to sink.”

    I never said any such thing and I dint even know you were on a ship, never mind a sinking one.

  46. Bugger: forgot me LOL! again.

    Oh well, lol anyway.

  47. @Katie

    Best to see the Scottish cross-break for a start (although it’s not representative of Falkirk in itself).

    I’ll guess at the IMF’s economic growth news being reflected in London / RoS and Mid being the reason (if it’s not an outlier).

  48. Whatever the reason – if there is one – it’s not helping the tories at all. 32% seems their recent average so, wherever Lab VI is going it must be going somewhtere else.

  49. Amazing how quickly a discussion of how ‘the general public’ can be way of the mark on their estimates of figures, turns into a ‘but I figure that the real amount of fraud must be tons higher’ speculation.

    As said, the DWP used statistical sampling to identify potential fraud cases. And then they referred those to investigation to discount false positives. Those are the firmest figures we have.

    Now you could very well say they *must* have then missed some from the sample, but really it’s actually very *very* hard to hide benefit fraud once someone is investigating it. Those “cash in hand” jobs will be the first to get spotted by the men in vans with cameras. And once tax fraud via unreported income is suspected, bank details can be compelled. You’d have to both be a master of subterfuge and a top notch criminal accountant to get away with it while being actively investigated.

    Now it may be that there’s a high level of super-genius criminal masterminds fiddling benefits in a way that’s undetectable by investigators… But I doubt it.

  50. @Paulcroft

    My guess is Don’t Know/Won’t Vote churn.

    Incidentally, anyone noticed the timing of announcement of the MP Pay Rise recommendation is quite fortuitous for Ed M? It’s almost like he controlled when the Union funding story would be released to the press, so it would be rolled over by the MP Pay Rise. What a coincidence, eh?

1 2 3