I posted these a couple of weeks before the election, but I see more and more of them cropping up in the comments, so I think it’s worth reposting it for newcomers.

1) The polls are ALL wrong, the real position is obviously X

Er… based on what? The reality is that opinion polling is pretty much the only way of measuring public opinion. We have some straws in the wind from mid-term elections, but they tend to be low turnout protest votes, don’t tend to predict general election results and are anyway quite a long time ago now. Equally a few people point to local government by-elections, but when compared to general election results these normally grossly overestimate Liberal Democrat support. If you think the polls are wrong just because they “feel” wrong to you, it probably says more about what you would like the result to be than anything about the polls.

2) I speak to lots of people and none of them will vote for X!

Actually, so do pollsters, and unless you regularly travel around the whole country and talk to an exceptionally representative demographic spread of people, they do it better than you do. We all have a tendency to be friends with people with similar beliefs and backgrounds, so it is no surprise that many people will have a social circle with largely homogenous political views. Even if you talk to a lot of strangers about politics, you yourself are probably exerting an interviewer effect in the way you ask.

3) How come I’ve never been invited to take part?

There are about 40 million adults in the UK. Each opinion poll involves about 1,000 people. If you are talking about political voting intention polls, then probably under 100 are conducted by phone each year. You can do the sums – if there are 40,000,000 adults in the UK and 100,000 are interviewed for a political opinion poll then on average you will be interviewed once every 400 years. It may be a long wait.

4) They only interview 1000 people, you’d need to interview millions of people to make it accurate!

George Gallup used to use a marvellous analogy when people raised this point: you don’t need to eat a whole bowl of soup to tell if it is too salty, providing it is sufficently stirred a single spoonful will suffice. The same applies to polls, providing an opinion poll accurately reflects the whole electorate (e.g, it has the right balance of male and female, the right age distribution, the right income distribution, people from the different regions of Britain in the correct proportions and so on) it will also accurately reflect their opinion.

In the 1930s in the USA the Literary Digest used to do mail-in polls that really did survey millions of people, literally millions. In 1936 they sent surveys to a quarter of the entire electorate and received 2 million replies. They confidently predicted that Alf Landon would win the imminent US Presidential election with 57% of the popular vote and 370 electoral votes. George Gallup meanwhile used quota sampling to interview just a few thousand people and predicted that Landon would lose miserably to Roosevelt. In reality, Roosevelt beat Landon in a landslide, winning 61% of the vote and 523 electoral votes. Gallup was right, the Digest was wrong.

As long as it is sufficent to dampen down sample error, it isn’t the number of people that were interviewed that matters, it is how representative of the population they are. The Literary Digest interviewed millions, but they were mainly affluent people so their poll wasn’t representative. Gallup interviewed only a few thousand, but his small poll was representative, so he got it right.

5) Polls give the answer the people paying for it want

The answers that most clients are interested in are the truth – polls are very expensive, if you just wanted someone to tell you what you wanted to hear there are far cheaper sources of sycophancy. The overwhelming majority of polling is private commercial polling, not stuff for newspapers, and here clients want the truth, warts and all. Polling companies do political polling for the publicity, there is comparatively little money in it. They want to show off their accuracy to impress big money clients, so it would be downright foolish for them to sacrifice their chances with the clients from whom they make the real money to satisfy the whims of clients who don’t really pay much (not to mention that most pollsters value their own professional integrity too much!)

6) Pollsters only ask the people who they know will give them the answer they want

Responses to polls on newspaper websites and forums sometimes contain bizarre statements to the effect that all the interviews must have been done in London, the Guardian’s newsroom, Conservative Central Office etc. They aren’t, polls are sampled so they have the correct proportion of people from each region of Britain. You don’t have to trust the pollsters on this – the full tables of the polls will normally have breakdowns by demographics including region, so you can see just how many people in Scotland, Wales, the South West, etc answered the poll. You can also see from the tables that the polls contain the right proportions of young people, old people and so on.

7) There is a 3% margin of error, so if the two parties are within 3% of each other they are statistically in a dead heat

No. If a poll shows one party on 46% and one party on 45% then it is impossible to be 95% confident (the confidence interval that the 3% margin of error is based upon) that the first party isn’t actually on 43%, but it is more likely than not that the party on 46% is ahead. The 3% margin of error doesn’t mean that any percentage with that plus or minus 3 point range is equally likely, 50% of the time the “real” figure will be within 1 point of the given figure.

8 ) Polls always get it wrong

In 1992 the pollsters did get it wrong, and most of them didn’t cover themselves in glory in 1997. However, lessons have been learnt and the companies themselves have changed. Most of the companies polling today did not even exist in 1992, and the methods they use are almost unrecognisable – in 1992 everyone used face-to-face polling and there was no political weighting or reallocation of don’t knows. Today polling is either done on the phone or using internet panels, and there are various different methods of political weighting, likelihood to vote filtering and re-allocation of don’t knows. In 2001 most of the pollsters performed well, and in 2005 they were all within a couple of points of the actual result, with NOP getting it bang on.

9) Polls never ask about don’t knows or won’t votes

Actually they always do. The newspapers publishing them may not report the figures, but they will always be available on the pollsters’ own website. Many companies (such as ICM and Populus) not only include don’t knows in their tables, but estimate how they would actually vote if there was an election tomorrow and include a proportion of them in their topline figures.


675 Responses to “REPOST: Too frequently asked questions”

1 8 9 10 11 12 14
  1. If postal voting fraud is so widespread, should the pollsters adapt their methodology to build in corrections for it?

    It’s difficult to know what question they could ask though….

    “Are you going to commit postal fraud and if so, for which party?”

  2. @Rowan

    ‘The post-debate polls are starting to look meaningless’

    I agree and to think so much is being made of them when they have a prosperous, broadsheet, older, slightly more Conservative and sometimes more male-dominated bias than the voting population as a whole.

    Wonder why this is only being pushed on the C4 site and not BBC, ITV et al?

  3. Sorry that should be:

    Eoin

    I sort of agree with your political analogy.

    But doesn’t the comparison with their Victorian predecessors make our current politicians (esp Cameron) look lightweight?

  4. YG prediction

    C 35
    L 27
    LD 29

  5. @Pankot
    “Unfortunately (following up a post I made back in this thread) the YG debate data now up in their archives doesn’t have a breakdown by party id, so I can’t compare with the AR data.”


    For reasons noone has explained, YG weighted to those “absolutely certain” to watch the debate so even if you did have party ID you would not be able to compare with AR. How many were “fairly certain” or even uncertain and still watched?
    What effect YG’s weighting has is anybody’s guess. I’d be equally interested in their raw data.

  6. @Roger Mexico,

    The most unexplored aspect of Cameron’s ideology is his foreign policy. In Northern Ireland we have a hint of what it migh tbe liek in his decision to contest seats here.

    For Disraeli see 1877 Empress of India
    and 1880 Sudan/Khartoum
    Oh and see Gladstone’s Midlothian Campaign.

    Wikipedia covers them all.

  7. oops, whilst replying to the Last Fandango’s note on postal voting, I seem to have somehow copied in the encylopedia brittania. sorry!

    my point on postal voting was that this was a real worry, and I would consider just about banning it for all those but the immobile.

  8. My YG prediction
    c 36
    ld 28
    l 26

  9. @eoin – “The most unexplored aspect of Cameron’s ideology is his foreign policy. In Northern Ireland we have a hint of what it migh tbe liek in his decision to contest seats here.”

    Since when has Northern Ireland been foriegn? It is an integral part of the United Kingdom and will remain so until the majority of it’s residents say otherwise. Contesting seat is therefore domestic policy.

  10. Channel 4 are saying that YouGov openly admit that they tend to poll more prosperous, older and Conservative voters.

    Is this true?

    http://blogs.channel4.com/snowblog/2010/04/30/what-the-post-debate-reaction-polls-tell-us/

  11. Richard O
    “my point on postal voting was that this was a real worry, and I would consider just about banning it for all those but the immobile.”

    Completely agree. I don’t see how it can be made fraud-proof.

  12. “Furthermore, is it a truism that all elderly people end up more conservative in their views?”

    I have a theory FWIW (probably not much) that it depends how gregarious you are. Thus urban areas tend to be left wing and more tolerant whereas rural areas are more right wing and conservative. Students are more left wing while older or more solitary people tend towards right wing because they have a narrower range of contacts. I’m not sure it’d stand up to much scrutiny, though!

  13. @Andy W,

    I Agree 100% :) The point still stands though since it provides an insight into how he views those types of issues.

  14. 10) 9 million people watch 3 debates, two of which were dead
    level and one which was won by Nick Clegg, so You Gov
    publish a poll suggesting David Cameron won by 15%

    I know they are an impartial polling company but all main members are voiciferous conservatives and after 3 months of closely monitoring this and other blog sites I now conclude you truly believe we are all mute and stupid.

  15. Pete B
    Richard O
    “my point on postal voting was that this was a real worry, and I would consider just about banning it for all those but the immobile.”

    Completely agree. I don’t see how it can be made fraud-proof.
    ***************************************************
    Any view on which parties would benefit/suffer? I am sort of assuming that independant smaller parties would be more likely to be involved if it does occur?

    Having said that, hasn’t a Labour MP opened and reported some results from ‘samples’ of postal voting on twitter, and now been investigated by the police?!?

  16. @Eoin

    “Blue to take…possibly Manchester whittington since there is not much more room for tactical voting…”

    Firstly, you mean Withington right? Secondly, what are you smoking? A bunch of people here aren’t immensely fond of John Leech, but there is literally no way the Tories will take win his seat.

  17. @ Matt

    I still think the Tories will fall short of an overall majority on May 6th, unless tonight indicates a lead over Labour at least 8% . All the parties are running out of time now.

    ____________________________________________

    I think these polls tonight are crucial. There are no more game changers (you would imagine, anyway). If the Tories dont get the breakthrough now it’s looking like they wont get that overall majority at all.

    FWIW, I think we might see some clear space emerging in the next polls on the strength of the reaction to tv debate III.

    My YouGov prediction:

    Con 36
    Lab 28
    LD 29

    Anything lower than that for the Tories would be a real concern for them and a boost to Lab/LD in the run in.

  18. @ben r

    I am not partisan, and if anything would prefer a change of government which would pave the way for a more representative system

    I thought Clegg did several things last night which will reduce the LD vote

    a) he got involved in quite a few aggressive exchanges, particularly over immigration
    b) the immigration issue he failed to respond to effectively, and it’s a big vote loser if handled badly (I don’t know why he didn’t just focus on the 600,000, that they have been the result of the other two’s policies, and say LDs would still be sending illegal immigrants back etc
    c) he completely failed to answer the ‘benefits cheat’ question – it was crying out for him to say he wouldn’t let people get away with it, but instead he kept repeating the £10,000 tax give away – it was a bad relationship – benefits cheats, give away taxpayers money to low earners
    d) I thought his final speech was too similar to his previous ones and came across a bit trite and non specific.

    we’ll see over the next two days

  19. All,

    I am pretty certain that the polls from tonight and saturday will be very close to the final GE result.

    There are no real game changers left now, and with 95% of the population thinking about a long bank holiday, electioneering will be second on the publics mind until at least Tuesday.

    I think we are nearly there. tonight & Sat”s polls are crucial…

  20. @ Eoin

    NW Region only

    Using 26 Apr regional % Con 20, Lab 35, LD 30,
    Others 6 as given in PoliticsHome regional chart for last week.

    Swings are Lab/Con +5, Lab/LD +9, Con/LD +4

    Constituency factors include, same degree of TV as in GE2005 plus, incumbent (advantage, or even disadvantage), comments on individual constituency pages, involvement of independants, UKIP, BNP, Greens.

    Con gains from Lab: City of Chester,Bury N,Ribble S,Rossendale & Darwen, Blackpool N, Bolton S, Wirral S, Carlisle, Bolton NE, Barrow in Furness,
    Weaver Dale, Sefton Central, Crewe & Natwich. Possibles: Lancashire W, Morecombe & Lunsdale, Blackpool S, Lancasster & Fleetwood.

    Con gains from LD: Nil

    LD Gains from Lab: Liverpool Waverley, Oldham E & Saddleworth, Burnley
    Possibles: Manchester Gorton
    LD gains from Con: Nil

    Lab to either Con or LD (too close to call which): Warriington S, Pendle.

    Before anyone challenges, please go to the constituency, see the majority, see the swing required, access the ‘local’ factors, see the comments over the last 3 weeks, add in any small % gained/lost from since 26th Apr. Then make an unbiassed decision. Then please contact if you want to challenge, but be prepared to give reasons beyond personal party bias.

  21. Richard O
    “Any view on which parties would benefit/suffer? I am sort of assuming that independant smaller parties would be more likely to be involved if it does occur? ”

    I think judging from past fraud cases in Birmingham and elsewhere, all three main parties have been involved, though more Labour than the others. Also, at the risk of being branded a ‘bigot’, all the cases I can recall were perpetrated by people not indigenous to this country. Having said that, I have not done a scientific survey of all past cases, so could be wrong. I would be interested to hear any different reasoned views.

  22. Being cautious I have never done a poll prediction but tonight I will do away with all caution:

    yougov

    1. Conservatives – a percentage of some sort
    2. LibDems – a percentage of some sort
    3. Labour – a percentage of some sort

    Anyone agree?

  23. Conservatives need to be hitting 38% regularly to be within sight of overall majority territory, I think.

  24. FrankG,

    Thank you for these. On these types of gains you must think old bluey has a chance of a majority?

  25. @Eoin Clarke

    Repost of my take on the NE Region, in case you did not get it. Enjoy :0

    FrankG
    @ Eoin
    My prediction for the NE using regional poll consolidation in PoliticsHome as guide and individual seat record for local flavour. By the way the PH chart for the NE is C20,L39, LD35, O 9. Yes, it does add up to 103, but that is what the chart says!

    Con gains from Lab Tynemouth. Possible Stockton S

    LD gains from Lab Durham City of, Blaydon, Newcastle N, Newcastle E, Middlesboro S & Cleveland,Hartlepool,Blyth Valley. Possibles Newcastle Central, Bishop Auckland

    What’s yours?

    April 30th, 2010 at 12:59 am

  26. either that and/or somehow push the Libdems below 25% and/or Labour below 24%. One of these maybe, both, possibly not.

  27. FrankG,

    I’ll print them off and tick them as they come up on the night. It will be interesting to see how we done.

    Care to do Scotland?

  28. Not sure if this has been discussed, but what are your thoughts on the following article in today’s Graun?

    http://www.guardian.co.uk/politics/2010/apr/27/nate-silver-labour-swing

  29. Any other polls due tonight besides YouGov?

  30. Ouch… sorry, that’s from the 28th April. It’s probably been talked about to death, but just found it myself. Apologies!

  31. Guardian online poll on debate winner:

    Cameron: 18.6%
    Brown: 30.2%
    Clegg: 51.1%

    :-)

  32. Effectively extending postal voting to being on demand has been a disaster. It was pushed through by Blair & co in an attempt to increase turnout despite all wise heads (including many in their own Party) pointing out the scope for fraud.

    This duly happened usually to the advantage of Labour – not organized nationally, but because it’s easiest to do in city centre areas with multiple occupancy, fast turnover of population etc.

    What’s worse it’s very difficult to prove and easy to get away with. Even when people have been caught red-handed the police have refused to investigate.

    Whatever happens after the election this needs to be reformed and the powers, effectiveness and independence of the Electoral Commission increased.

  33. RogerH,

    given the article on channle four abou the weigting of the audience of the panels for last night’s polls, this one probably has as much credibility.

    Still a voodoo though! And you know it!

  34. @Eoin Clarke

    ‘Thank you for these. On these types of gains you must think old bluey has a chance of a majority?’

    Not necessarily so, I have not done predictions for other regions yet. IMO Con is probably about at least 300+.

    The problem is you have too look at each individual seat to get a ‘feel’ for your factors. If Con goes ahead of Lab much more then yes, it is indeed a strong possibilty. You notice how little LD had picked up in the NW compared to the NE. That is why regional analysis and not national analysis is so vital.

    Have fun checking them out. Keep you busy until any polls tonight. Cheers :)

  35. @ Eoin

    “Still a voodoo though! And you know it!”

    Surely you wouldn’t suggest that Guardian readers are unrepresentative?

  36. DC is having a big push for Ed Ball’s seat! General feeling in West Yorkshire is that he’s building up some real momentum here!

  37. @YAKOB
    Being cautious I have never done a poll prediction but tonight I will do away with all caution:

    yougov

    1. Conservatives – a percentage of some sort
    2. LibDems – a percentage of some sort
    3. Labour – a percentage of some sort

    Anyone agree?

    It’s blatently partisan projections like this that bring this forum down…

  38. @RogerH.

    Either you are being argumentitative for the fun of it, or you are immune to logic. :)

  39. @Eoin
    ‘Care to do Scotland?’

    Ha Ha, a minefield, but as an ex-serivceman of 33 years service, minefields aren’t quite so terrorfying. Will have a go!

    Don’t ask for NI, served there too many times. My prediction – no change.

  40. @ Eoin

    Another prediction please.
    My constuency is Manchester Withington.

    It is a lib dem/labour marginal. Last GE there was a 17% swing from labour to lib dems -the largest in the country.
    :-)

  41. Eoin

    Thanks re Gladstone

    My knowledge of 19th and early 20th Century political history is not what it should be. Plan to read up a bit. Was Gladstone’s legacy not thought to be partly to blame for the Liberals decline to the benefit of Labour?

    My prediction for tonight:

    Con – 35%
    Lib Dem – 29%
    Labour – 25%

  42. “Either you are being argumentitative for the fun of it, or you are immune to logic.”

    The Guardian is the voice of the person on the Clapham bicycle.

  43. Re Guardian voodoo poll

    Fancy taking notice of such a silly poll!
    The media ‘pollsters’ who apparently all UNANIMOUSLY predict a CON MAJ in the tightest election for 40 years stand to make NOTHING if DC gets in.
    I cannot believe people would even question these hard working saint like impartial reporters.

    The Guardian poll and all others conducted by companies not trying to talk the Tories into government should be ignored COMPLETELY

    Tonights Poll prediction YouGOv and AR 44 18 29
    All independent polling companies 34 29 32

    Happy 1 day birthday to all ther other members of the electorate :)

    Land of hope and glory, mother of the free………………………………. COME ON SING ALONG!!!

  44. @The LAst Fandango,

    No not at all. Lord Salisbury served at least threee terms after him, and Asquith served a coupl eof Admins.

    If you want soemone to blame Lloyd George is your man :)

    (sorry Bill Bob)

  45. @ Roger – that really is shocking. Guardian readers vote Cameron in last place! Who’d ever have thought that could happen :D

  46. I tried to watch these debates with the eyes of a hawk and pay close attention as I care how I’m going to be living my life under whoever is in power next week. I’m still young so I haven’t thought about politics before, however I’ve gotten into it after watching these debates.

    As for performance, I think Cameron did very poor and am very suprised about the yougov polls, he failed to answer Gordons question up to 4 times I think it was, and Clegs question about the imigration cap, Cameron won’t deviate from his scripts which leaves him very vulnerable – so I fail to see how he can convince other leaders in or outside of the EU to do anything if he can’t answer questions infront of a live tv debate and infront of Cleg and the current Prime Minister. I really don’t like the fact Cameron will take money out of the economy, NHS and other public services just to pay off a debt we tax payers didn’t even create, at least Labour isn’t making us pay out the arse for it.

    I thought Brown was amazing in this BBC debate, he really knows his stuff about the economy and you can see how passionate he is about it, he really is a man with experience under his belt and it shows. When I hear people on the news or whatever say Brown did terrible, how and where? cause I don’t see it, I was proud of Brown, neutral about Cleg but completely embarresed for Cameron, once he’s in a corner he gets snippy and creates temporary answers for everything like about the cap, watch the part where Cleg had him on the ropes about imigration cap, seriously that was a disaster for Cameron.

    I hope UK doesn’t get like the US and we end up voting on who looks better in a suit or if they play football or not, we should be voting on what policies will be affecting our own lives and not be brainwashed by the media, we didn’t win two world wars to end up as a gimmick country.

    Going on nothing but policies (ignoring who will be PM) I will be voting for Labour, since they won’t cripple our public services, something Cameron can’t guarantee and he’s already said he’ll be taking 6bn out.

    Performance:
    Brown > Cleg > Cameron

    Policies:
    Labour > Tori > Lib Dems

  47. @Eoin

    Well we all know Lloyd George had other things on his mind most of the time… ;-)

  48. CB:

    Don’t shout: I bloody well was singing along.

  49. Brown should be worried – only 30% from the loyal Grauniad readers……….. ;-)

    If I was DC I’d be thrilled to get 18% for last night from them (last week only 5% of Grauniad readers thought it should back the Tories) 18% is probably worth 40 odd % after weighting ;-)

  50. @scotty dog

    I know the immigration stuff is being run hard, but I thought Clegg dealt with it quite effectively and anyway hasn’t this been debated twice already? Isn’t it factored in to people thinking – it can’t have been new to anyone…

    re: the welfare cheats I think the appeal of the threshold increase outweighs the failure to make the point about welfare cheats – to be honest I didn’t even notice he didn’t

    I admit I’m sympathetic to Clegg but I general try to be realistic.

    @valerie

    Manchester Witherington

    With a big swing to the Lib Dems from Labour nationally, I’d be very surprised if that lost any seat to Labour

1 8 9 10 11 12 14