I posted these a couple of weeks before the election, but I see more and more of them cropping up in the comments, so I think it’s worth reposting it for newcomers.

1) The polls are ALL wrong, the real position is obviously X

Er… based on what? The reality is that opinion polling is pretty much the only way of measuring public opinion. We have some straws in the wind from mid-term elections, but they tend to be low turnout protest votes, don’t tend to predict general election results and are anyway quite a long time ago now. Equally a few people point to local government by-elections, but when compared to general election results these normally grossly overestimate Liberal Democrat support. If you think the polls are wrong just because they “feel” wrong to you, it probably says more about what you would like the result to be than anything about the polls.

2) I speak to lots of people and none of them will vote for X!

Actually, so do pollsters, and unless you regularly travel around the whole country and talk to an exceptionally representative demographic spread of people, they do it better than you do. We all have a tendency to be friends with people with similar beliefs and backgrounds, so it is no surprise that many people will have a social circle with largely homogenous political views. Even if you talk to a lot of strangers about politics, you yourself are probably exerting an interviewer effect in the way you ask.

3) How come I’ve never been invited to take part?

There are about 40 million adults in the UK. Each opinion poll involves about 1,000 people. If you are talking about political voting intention polls, then probably under 100 are conducted by phone each year. You can do the sums – if there are 40,000,000 adults in the UK and 100,000 are interviewed for a political opinion poll then on average you will be interviewed once every 400 years. It may be a long wait.

4) They only interview 1000 people, you’d need to interview millions of people to make it accurate!

George Gallup used to use a marvellous analogy when people raised this point: you don’t need to eat a whole bowl of soup to tell if it is too salty, providing it is sufficently stirred a single spoonful will suffice. The same applies to polls, providing an opinion poll accurately reflects the whole electorate (e.g, it has the right balance of male and female, the right age distribution, the right income distribution, people from the different regions of Britain in the correct proportions and so on) it will also accurately reflect their opinion.

In the 1930s in the USA the Literary Digest used to do mail-in polls that really did survey millions of people, literally millions. In 1936 they sent surveys to a quarter of the entire electorate and received 2 million replies. They confidently predicted that Alf Landon would win the imminent US Presidential election with 57% of the popular vote and 370 electoral votes. George Gallup meanwhile used quota sampling to interview just a few thousand people and predicted that Landon would lose miserably to Roosevelt. In reality, Roosevelt beat Landon in a landslide, winning 61% of the vote and 523 electoral votes. Gallup was right, the Digest was wrong.

As long as it is sufficent to dampen down sample error, it isn’t the number of people that were interviewed that matters, it is how representative of the population they are. The Literary Digest interviewed millions, but they were mainly affluent people so their poll wasn’t representative. Gallup interviewed only a few thousand, but his small poll was representative, so he got it right.

5) Polls give the answer the people paying for it want

The answers that most clients are interested in are the truth – polls are very expensive, if you just wanted someone to tell you what you wanted to hear there are far cheaper sources of sycophancy. The overwhelming majority of polling is private commercial polling, not stuff for newspapers, and here clients want the truth, warts and all. Polling companies do political polling for the publicity, there is comparatively little money in it. They want to show off their accuracy to impress big money clients, so it would be downright foolish for them to sacrifice their chances with the clients from whom they make the real money to satisfy the whims of clients who don’t really pay much (not to mention that most pollsters value their own professional integrity too much!)

6) Pollsters only ask the people who they know will give them the answer they want

Responses to polls on newspaper websites and forums sometimes contain bizarre statements to the effect that all the interviews must have been done in London, the Guardian’s newsroom, Conservative Central Office etc. They aren’t, polls are sampled so they have the correct proportion of people from each region of Britain. You don’t have to trust the pollsters on this – the full tables of the polls will normally have breakdowns by demographics including region, so you can see just how many people in Scotland, Wales, the South West, etc answered the poll. You can also see from the tables that the polls contain the right proportions of young people, old people and so on.

7) There is a 3% margin of error, so if the two parties are within 3% of each other they are statistically in a dead heat

No. If a poll shows one party on 46% and one party on 45% then it is impossible to be 95% confident (the confidence interval that the 3% margin of error is based upon) that the first party isn’t actually on 43%, but it is more likely than not that the party on 46% is ahead. The 3% margin of error doesn’t mean that any percentage with that plus or minus 3 point range is equally likely, 50% of the time the “real” figure will be within 1 point of the given figure.

8 ) Polls always get it wrong

In 1992 the pollsters did get it wrong, and most of them didn’t cover themselves in glory in 1997. However, lessons have been learnt and the companies themselves have changed. Most of the companies polling today did not even exist in 1992, and the methods they use are almost unrecognisable – in 1992 everyone used face-to-face polling and there was no political weighting or reallocation of don’t knows. Today polling is either done on the phone or using internet panels, and there are various different methods of political weighting, likelihood to vote filtering and re-allocation of don’t knows. In 2001 most of the pollsters performed well, and in 2005 they were all within a couple of points of the actual result, with NOP getting it bang on.

9) Polls never ask about don’t knows or won’t votes

Actually they always do. The newspapers publishing them may not report the figures, but they will always be available on the pollsters’ own website. Many companies (such as ICM and Populus) not only include don’t knows in their tables, but estimate how they would actually vote if there was an election tomorrow and include a proportion of them in their topline figures.


675 Responses to “REPOST: Too frequently asked questions”

1 9 10 11 12 13 14
  1. @EoinClarke
    “The most unexplored aspect of Cameron’s ideology is his foreign policy.”

    Judging by his track record (Burma floods, Georgian crisis, Europe) my judgement is that he is less driven by ideology than by his determination to appear decisive. I expect the FCO would put him straight if he were to have to tackle issues as head of government and deal with them in a considered way, in consultation with other international/regional players.
    When GB had to do so DC characterised it as “dithering”

  2. @GreenG,

    Dithering rocks! Less people seem to die :)

  3. Thinking about the campaign so far, I agree with Eoin opinion that this smacks of 92, certainly at the start – an unpopular government, but no great enthusiasm for the alternative. I think the way the the country embraced the Lib Dems backs this up, picking up disgruntled labour supporters who could never stomach backing the Tories, and also capturing those voting Tory in protest rather than enthusiasm for Cameron. But will it play out the same as ’92 with a late rush back to labour – maybe not enough for a majority (although that cannot be ruled out) but certainly enough to deny the Tories outright victory.

    The Lib Dem surge obviously changes things, but as polling day draws closer I suspect the defectors will start to return to their original party. As the ‘shy’ voters from each side start to contemplate the serious possibility of a Tory majority/Hung Parliament (depending if they were red or blue)the Lib Dems poll percentages will slowly fall, stabilising on around 25-26% – their pre-campaign figure plus the new voters they picked up.

    The other obvious difference to 92 is that we are considering shy labour voters – will they flock back to their party at the last minute the way the Tories did? Brown has been playing the “trusted pair of economic hands” card pretty much all the way through the campaign, but will economic worries motivate his supporters in the same way it did Majors? It’s playing to a different demographic so impossible to call. Should Labour start closing the gap on the Tories Tuesday/Wednesday we can guess they will, but it might be a last minute change, again as in 92.

    Based on the above, my prediction for YG is pretty much the same as yesterday, maybe the Lib Dems down a point but all MOE stuff. I don’t think yesterdays debate was a game changer, and any drift back won’t start in earnest until after the weekend.

  4. Welcome to the site Karl – as you can see, there is a diversity of political opinion here so not everyone will agree with your take on the debates ;) but I think we can all agree that if the debates have improved the political engagement of younger voters like yourself, that’s a very positive development. :)

    With regards to your concern about UK politics becoming too image-based – well, images have mattered for a long time and much as we’d all like every policy to be appraised in a cool, reflective, completely unbiased way – it ain’t gonna happen! ;) But I do see grounds for optimism in that the legacy of the expenses scandal has created a mood for reform that all the parties have picked up on – even the Conservatives now want to achieve consensus on an elected House of Lords…

  5. @PeteG,

    we now know for a fact that Bashful Browns exist.

    Gillian Duffy saind, “I am ashamed to say I vote Labour”

    Should we change it from bashful Browns to reluctant reds?

  6. @PeteB
    “all the cases I can recall were perpetrated by people not indigenous to this country.”

    Could those be because they were the ones most reported in the media?

  7. Average of the last four YG polls over the past week.

    Con: 33.5%

    LD just over 29%

    Lab 28%

    Pretty close I’d say.

  8. I’ve just been looking at comments on the Guardian website (something I rarely do and won’t be doing again in a hurry – this place is an oasis of sanity in comparison). Anyway, what struck me is how many people seem to believe that poll results are doctored to fit the agenda of whoever has commissioned, who see no difference between a voodoo poll and a professional poll, and who seem to think that the Guardian is betraying Labour by publishing ICM’s findings on last night’s debate.

    It’s a mad mad world.

  9. Momentum and perception is a HUGE thing and today after listening and watching the media I think that the leak/trickle will turn into a flood. The public like to be seen to back the winner and all the time the Tories are out in front they’ll jump on board IMHO… I fear the Labour vote is crumbling, the LD’s are an unknown entity (their time may come).
    I really think that a slight increase in the Tory lead may lead to the floodgates opening and I predict they will get a healthy majority.

  10. Kepp your eyes peeled for a 6pm Guardian ICM poll. If i am not mistaken, there was one this last two weeks at that time.

  11. @ Pete G

    “The other obvious difference to 92 is that we are considering shy labour voters”

    I’d say the crucial difference is the leader. Major had replaced Thatcher and, unlike Brown, was relatively unknown. People had effectively had a change of government and were willing to give the new PM a shot.

  12. Karl
    Welcome, and good first post – you’ve endeared yourself to the Lab leaners here (including me).

  13. @polly ticks,

    Have I been nice enough to you this last wee while to get a chance to ask you one question sir?

    What constituency are you….?

  14. Polly Ticks “I fear the Labour vote is crumbling…”

    Lol

  15. Eoin..

    Of course my friend..ask me anything…

    I live in Locks Heath which comes under Fareham….Susan Bayford is the Lady I have leafleted for and Mark Hoban is my Local Tory Candidate

  16. PTicks:

    “I fear the Labour vote is crumbling”

    I’m sure you’ll get over it.

  17. @EOIN CLARKE

    -I just logged on.
    -Is there any word on today’s polls?

  18. @Polly Ticks

    I notice you only whine about partisan when the Conservatives are doing badly…

  19. Mike N…

    We’ll see……..I do think i’ll have the last laugh…

    But I’m of the mind that what will be will be……

    Cheers

  20. Latest oddsclegg 14/1 most seats
    Cameron 1/8 most seats
    Brown 5/1 er sorry just changed while typing now 6/1
    yeah looks like them dumb bookies got it wrong again,expecting a deluge of labour supporters treading on each other to get those odds on brown ,
    yeah ok i’ll do the jokes

  21. Steve
    DC is having a big push for Ed Ball’s seat! General feeling in West Yorkshire is that he’s building up some real momentum here!
    **************************************************
    To borrow a Kevin Keegan phrase, I would love it, just love it, if Cons took Morley.
    My brother used to live there, I know the area quite well. Probably best described as nearer working class, but up & coming aswell, with some areas of quite nice big old slate built terraces, likely to be more middle class.

  22. Polly Ticks

    I was and still am laughing at your “I fear…”

  23. Paul..you’re right… :-)

    Billy…get over it..because you don’t do you? ;-)

  24. Did anybody notice this is yesterday’s post-debate ICM poll:

    Sample of people who have changed their minds about who to vote for:

    Tory 15%
    Labour 33%
    Lib Dem 38%

    What do we make of this?

    ???

  25. @grem3 and Yakobs

    You are the last straw with your tribal comments. I am leaving this site until after the election ( Sue and Amber will be delighted).

  26. I wish that was the latest poll (;-)

    Sample of people who have changed their minds about who to vote for:

    Tory 15%
    Labour 33%
    Lib Dem 38%

  27. Billy:

    They’re not doing well though – don’t get carried away with the momentum about hype following artificial, non-representative, post-debate polls.

    The fact is they’re averaging just 34% after thirteen years in opposition.

    On these figures their only real chance is that Labour and Lib Dems end up bashing each other up and letting them in that way.

  28. @Geoff

    That accords with the AR polling data – while NC was not thought to have won the debate amongst viewers as a whole, he does seem to have come across better to undecideds/floaters.

    What I would keep an eye out for in the coming polls is whether DC’s performance has managed to gather a few % of UKIP voters back to the Tories. Just 1 or 2% could make a significant difference to the number of seats he wins.

  29. @Roger H
    Major replaced Thatcher in November 1990, the election was May 1992 so he was hardly a newbie, he’d also been Foreign Secretary and Chancellor and had been vilified on Spitting Image as being a pea loving grey man.

    I think people may well be surprised on Thursday at the similarities between 2005 and 1992

  30. Thanks Pankot.

    One small request – would you mind using the term ‘floating voters’ and not ‘floaters’? It’s puts an image in my head that I would rather was not there.

    Thanks.

  31. @ Geoff

    Not sure, does it mean that 15% are now going to vote Tory or that 15% of people who originally were going to vote Tory are now switching to other parties and the same for the other two?

  32. @James Ludlow

    The thing that strikes me most about the comments on the Guardian website is how heavily they’re in favour of the Lib Dems. The main gripe that they seem to have is that despite carrying out a sort of crowd-sourcing operation where they asked their online readership which party they would like the paper to support (1500 odd replies, mostly encouraging them to go Liberal), the paper has been producing a series of opinion pieces and scaremongering headlines trying to shore up the Labour vote. This has riled it’s Lib Dem supporting readers no end, and has resulted in angry comments on numerous articles. The ICM poll, having put Clegg in 3rd place in the last debate, has become a target for this ire.

  33. @Andrew

    Sorry, I didn’t explain it well really.

    That is the voting intention of people who have changed their mind about who to vote for. So, of the people that have changed their minds, 15% will now vote Tory, 33% Labour, etc… it seems to indicate that Labour and Lib Dem should be gaining on Tory as more people are moving to them than to the Tories. That said I’ve probably missed out some fundamental reason why this is not the case.

  34. Mike N

    I fear because like the Tories in 1997, I think Labour will take a massive beating next week. They need to go away again and re-group etc.

    I’m of the opinion the country needs a strong opposition and the Tories were not that for at least 8 years after they lost. I also think if DC wins next week which I think he will now, that LD will push Labour all the way to 2nd.

  35. I’d like to hear some general views on the Ipsos Mori ‘Worm’ used by the BBC Ten O’Clock News providing real time analysis of the leaders’ debates. If you haven’t seen it, a studio audience (not sure how they are selected) electronically choose a preference for the speaker which is then displayed as a wiggley line. The line rises or falls in real time according to how much they like what the speaker is saying. Admittedly it might be interesting to see the analysis over the full 90 minutes – say online – but does it work within a few minutes on the news? I’m, no pollster unlike you lot- but it didn’t do much for me.

  36. @Geoff

    w w w.guardian.co.uk/politics/2010/apr/30/tv-debate-cameron-brown-gaffe-poll

    According to that article, those numbers relate to how the 15% of respondents who’s minds were changed by the debate will split between the three parties. As Pankot says, it points in the same kind of direction as the findings in the AR poll. What it doesn’t say is what the intentions of that 15% were before the debate.

  37. @IKEADDY

    Looked to me that the number of people involved was very small and (dare I say it!) didn’t exactly look like a good cross-section of the voting population as a whole (lots of students! – this might indicate why Cameron’s immigration speech did not go down well). Also, if they are all doing this in their seat then they can all see what everyone else is doing and so may be influenced by them.

    It’s all very pretty to look at, but not sure it is much use to be honest.

  38. @Geoff,

    Sorry…. and lol…

    “floating voters” from now on, I promise! :)

  39. How dare anyone criticise the Guardian poll! The percentages match also exactly those I posted here after the debate.

    So the poll proves incontrovertibly that . . . I am a Guardian reader.

    Which I am.

  40. I’m standing by my earlier Yougov prediction:

    CON 34%
    LAB 26%
    LDEM 33%

  41. Sorry “almost” not “also”

    Must stop rely on the spellcheck

  42. @Roger,

    I have been surprised how anti-Brown the Guardian have been. Also not too anti-Tory either.

    When I was at Uni in the 90s, the Guardian was pretty left wing and very anti-Tory.

    rich

  43. @Geoff
    Thanks for that, I’ve taken a look at the poll data and the next two questions after the one you mentioned may give some indication

    Brown polled 36 to Cameron’s 35 on who would make the best PM after the debate and 45% thought Brown would make the right decisions when the going got tough compared to Cameron’s 32% with Clegg at 23 and 17 respectively.

    Maybe that’s some indication as to why people have changed to Labour but doesn’t explain why more have shifted to the Lib Dems!

    So who know’s we’ll find out on Thursday I guess :-)

  44. @IKEADY
    -I watched the entire debate with the MORI/IPSOS worm.
    -I did not think the 10:00 news analysis of the worm’s movement was representative of the fluctuations I saw during the debate.
    -EXAMPLE..At first GB had only slight reactions in either direction.
    -But, as the debate continued; he had more positive reactions.
    – His closing statement garnered negative feedback.

  45. Ricard O

    The Telegraph and Mail have not been that pro-Tory

  46. @ Richard O

    I have noticed the same drift with the Guardian over the last 18 months (its the only broadsheet I can abide paying for)

    Blair poisoned the well for them, and they reluctantly recognise the Brown was a great chancellor and a diabolical PM.

    I guess they imagine they are better inside the tent peeing out, and so are being reasonably friendly to the Tories.

  47. @Andrew

    The AR data may give a clue here:

    On questions like “most Prime Ministerial” or “best able to take tough decisions”, labour voters lined up behind GB, and conservative voters behind DC. But the LD voters lined up only partially behind NC – many of them preferred GB!

    Probably this is an area where NC’s being “new”/”fresh” doesn’t help him.

    So in other words, there are a sizeable portion of voters who shall we say have some regard for GB’s qualities, but still won’t vote for him (say on policy grounds)

  48. @KEADY
    -It was interesting to learn the audience was selected because they were UNDECIDED voters

  49. Geoff,

    Yes, I spotted this. There are some very interesting findings underneath the surface in the polls.

    ICM also have Brown narrowly ahead as the best prime minister.

    The headline “who won” the debate numbers in no way tell the whole story.

    I still think there will be no clear winner in this election, except perhaps those looking for a referendum on PR.

  50. I had an interesting conversation with a nurse today. She has been a life long Tory voter, However, due to the unkown cuts the Tories plan and the effects this will have the NHS, she is changing her vote to the Lib Dems as she feels she cannot trust DC.

1 9 10 11 12 13 14