I posted these a couple of weeks before the election, but I see more and more of them cropping up in the comments, so I think it’s worth reposting it for newcomers.

1) The polls are ALL wrong, the real position is obviously X

Er… based on what? The reality is that opinion polling is pretty much the only way of measuring public opinion. We have some straws in the wind from mid-term elections, but they tend to be low turnout protest votes, don’t tend to predict general election results and are anyway quite a long time ago now. Equally a few people point to local government by-elections, but when compared to general election results these normally grossly overestimate Liberal Democrat support. If you think the polls are wrong just because they “feel” wrong to you, it probably says more about what you would like the result to be than anything about the polls.

2) I speak to lots of people and none of them will vote for X!

Actually, so do pollsters, and unless you regularly travel around the whole country and talk to an exceptionally representative demographic spread of people, they do it better than you do. We all have a tendency to be friends with people with similar beliefs and backgrounds, so it is no surprise that many people will have a social circle with largely homogenous political views. Even if you talk to a lot of strangers about politics, you yourself are probably exerting an interviewer effect in the way you ask.

3) How come I’ve never been invited to take part?

There are about 40 million adults in the UK. Each opinion poll involves about 1,000 people. If you are talking about political voting intention polls, then probably under 100 are conducted by phone each year. You can do the sums – if there are 40,000,000 adults in the UK and 100,000 are interviewed for a political opinion poll then on average you will be interviewed once every 400 years. It may be a long wait.

4) They only interview 1000 people, you’d need to interview millions of people to make it accurate!

George Gallup used to use a marvellous analogy when people raised this point: you don’t need to eat a whole bowl of soup to tell if it is too salty, providing it is sufficently stirred a single spoonful will suffice. The same applies to polls, providing an opinion poll accurately reflects the whole electorate (e.g, it has the right balance of male and female, the right age distribution, the right income distribution, people from the different regions of Britain in the correct proportions and so on) it will also accurately reflect their opinion.

In the 1930s in the USA the Literary Digest used to do mail-in polls that really did survey millions of people, literally millions. In 1936 they sent surveys to a quarter of the entire electorate and received 2 million replies. They confidently predicted that Alf Landon would win the imminent US Presidential election with 57% of the popular vote and 370 electoral votes. George Gallup meanwhile used quota sampling to interview just a few thousand people and predicted that Landon would lose miserably to Roosevelt. In reality, Roosevelt beat Landon in a landslide, winning 61% of the vote and 523 electoral votes. Gallup was right, the Digest was wrong.

As long as it is sufficent to dampen down sample error, it isn’t the number of people that were interviewed that matters, it is how representative of the population they are. The Literary Digest interviewed millions, but they were mainly affluent people so their poll wasn’t representative. Gallup interviewed only a few thousand, but his small poll was representative, so he got it right.

5) Polls give the answer the people paying for it want

The answers that most clients are interested in are the truth – polls are very expensive, if you just wanted someone to tell you what you wanted to hear there are far cheaper sources of sycophancy. The overwhelming majority of polling is private commercial polling, not stuff for newspapers, and here clients want the truth, warts and all. Polling companies do political polling for the publicity, there is comparatively little money in it. They want to show off their accuracy to impress big money clients, so it would be downright foolish for them to sacrifice their chances with the clients from whom they make the real money to satisfy the whims of clients who don’t really pay much (not to mention that most pollsters value their own professional integrity too much!)

6) Pollsters only ask the people who they know will give them the answer they want

Responses to polls on newspaper websites and forums sometimes contain bizarre statements to the effect that all the interviews must have been done in London, the Guardian’s newsroom, Conservative Central Office etc. They aren’t, polls are sampled so they have the correct proportion of people from each region of Britain. You don’t have to trust the pollsters on this – the full tables of the polls will normally have breakdowns by demographics including region, so you can see just how many people in Scotland, Wales, the South West, etc answered the poll. You can also see from the tables that the polls contain the right proportions of young people, old people and so on.

7) There is a 3% margin of error, so if the two parties are within 3% of each other they are statistically in a dead heat

No. If a poll shows one party on 46% and one party on 45% then it is impossible to be 95% confident (the confidence interval that the 3% margin of error is based upon) that the first party isn’t actually on 43%, but it is more likely than not that the party on 46% is ahead. The 3% margin of error doesn’t mean that any percentage with that plus or minus 3 point range is equally likely, 50% of the time the “real” figure will be within 1 point of the given figure.

8 ) Polls always get it wrong

In 1992 the pollsters did get it wrong, and most of them didn’t cover themselves in glory in 1997. However, lessons have been learnt and the companies themselves have changed. Most of the companies polling today did not even exist in 1992, and the methods they use are almost unrecognisable – in 1992 everyone used face-to-face polling and there was no political weighting or reallocation of don’t knows. Today polling is either done on the phone or using internet panels, and there are various different methods of political weighting, likelihood to vote filtering and re-allocation of don’t knows. In 2001 most of the pollsters performed well, and in 2005 they were all within a couple of points of the actual result, with NOP getting it bang on.

9) Polls never ask about don’t knows or won’t votes

Actually they always do. The newspapers publishing them may not report the figures, but they will always be available on the pollsters’ own website. Many companies (such as ICM and Populus) not only include don’t knows in their tables, but estimate how they would actually vote if there was an election tomorrow and include a proportion of them in their topline figures.


675 Responses to “REPOST: Too frequently asked questions”

1 10 11 12 13 14
  1. @ Jones – yes, you’re right, there was sizeable Lib Dem support among the Guardian commenters. That doesn’t surprise me but I am surprised that the Guardian itself hasn’t yet come down firmly in favour of the Lib Dems over Labour. Perhaps there are still too many old school Labour supporters in its ranks for that.

  2. @Pete,

    I think your missing the point, or your ‘friend’ is.

    There needs to be massive cuts across the board by whichever of the parties gets in. At least in the region of £50 billion within 2 years.

    I doubt any of the parties will cut front line nurses, however if I was in admin, or working for IT contractors on new systems, I would be doing my CV.

  3. I read an article earlier (Guardian?) that suggested YouGov were weighting to different demographics in the post-debate polls (those watching the debate rathe than UK demographics?)

    Anyone able to explain what weightings were being used by the pollsters on this occasion?

    p.s. Guardian editorial : “If the Guardian had a vote it would be cast enthusiastically for the Liberal Democrats. But under our discredited electoral system some people may – hopefully for the last time – be forced to vote tactically”

  4. We are watching ITV news and they have just done the three ‘follow the Leaders’ sketches.

    We were first laughing and were now dismayed at the differing coverage. Brown was a car-crash loser.
    Clegg’s coverage was similar – he’s letting in all the undesirables. Cameron’s was ‘new dawn’ stuff, all hope and promise.

    I honestly cannot understand how Labour in particular maintains even a double digit support except that clearly voters mainly ignore this stuff.

    So why do they keep doing it? I suppose it’s worth a % of a % but goodness me!

  5. There’s no doubt that within MoE most pollsters produce reliable results – according to the principles of market research.

    Bearing in mind the pollster8’s comments a few Sunday’s back – they were speculating as to how they thought the position would be on May 6. There would have been no scientific research to their comments – just a ‘gut’ feeling based on their long association with elections etc.

    Whilst I agreed, at the time, that CamCon would get to 331 and have c10 maj – the comments by the P8 were surprising. I’ve said before that reporting the ‘facts’ as they discover on research is what they should limit themselves to rather than seeking to make a stab at predicting – I would assume Anthony Wells’ 9 points of ‘law’ are aied equally at the P8?

    The next few polls are important to understand what trends are likely to emerge in the last 5 or 6 days. I would not guess at the results of polls and – from some of the posts here – it looks as if some are getting quite depressed, possibly unnecessarily!

    I think the reported comments that whoever wins will likely lose the subsequent election (and for some time thereafter) – so there’ll be something in Thursdays result for all!

  6. a lifelong tory voting nurse ?

    lol

    do these people exist ?

  7. Wholly unrepresentative and unscientific polling exercise in the school that I teach at today, but in the absence of new polls to discuss I feel compelled to offer it as evidence of, er, something.

    So, Grammar School in Buckinghamshire. Very safe Conservative seat (Not Bercow’s). Week of (impressive) campaigning by 6th Form students representing the main parties. All students invited to vote for their party of choice. In 2005 massive vote for Conservatives in school election. All anecodotal evidence suggests many vote for the party that their parents support (which is no real surprise, after all, as some of the electorate are only 11 or 12 years old).

    Today’s results were 35% Lib Dem, 25% Cons., 18% Green, 16% Labour and 4% UKIP. Turnout 57% of school.

    Make of that what you will, but it’s a much higher Lib Dem vote than I’d anticipated and is proof of something or other.

  8. @wanderwelshman,

    I met a brave soldier yesterday, lifelong Labout voter who is changing to Conservative as he is worried about cuts in equipment…………

    oops, I admit, I made it up!

  9. @Howard,

    I watched it too. It is an excuse for journalism :)

  10. @Valerie
    @ Eoin Clarke

    Re you post to him about Manchester Withington
    April 30th, 2010 at 5:23 pm

    If Eoin has replied then apologies I missed it. If not maybe this will help

    Seat was taken by LD with a huge swing in GE2005. Boundary changes slightly favoured Lab nence the LD majority is a notional 508. By the national swing from Lab to LD, the LD candidate should be walking it, especially with the advantage of incumbency, but he isn’t according to the local seat blog. Local blog runs into 59 pages, the largest I ahve seen for any seat.
    Seat is riven with local politics intriges, accusations and counter-accusations, LD candidate sees to be running a very low key campaign. Very strong Lab candidate working hard, Independent former LD candidate, but not too effective, UKIP and Greens in the field as well.

    My take – this could be a seat going to Lab, too close to call at the moment. The huge swing last GE2005 may result in a swing back, totally against the national trend. Lkely margin a handful of votes either way as it stand today. The LD candidate has got to get stuck in there or he will lose it!

    You can look at the seat blog (but take some comments with a pich of salt) as follow:

    In Constituency Guide above on this page. Click on seat A – Z. Click on seats E – M. Click on Manchester Withington. Low and behold seats stats, info and huge blog appears.

    Hope above helps

  11. @STATTO
    Well your post is a bit more realistic than a week or two ago?
    Whats happened to the genius of Prime Minister Brown shineing through to victory over the vaucous little twitt “Dave”?

  12. @Howard,

    did it show the car crash near where Labour were campaigning in Hockley Heath? Its only a few miles from where I am.

    Apparently they started a press conference about Labour offering hope etc, and a car almost immediattely ploughed in to a bus shelter, leaving the press with an open goal for, ‘campaign like a car crash’ line.

    You couldn’t make it up. Brown is having a touch of bad luck.

    Rich

  13. @ Howard

    I know SKY have been getting a lot of the stick on political bias (rightly so), but ITV have been a disgrace in their coverage of the election. It’s like watching Fox News. Nothing Brown/Labour say or do is left uncriticised; nothing Cameron/Tories say or do is critqued.

    A complete and utter stitch up by a company that is hopefully going to the wall.

  14. @Statto

    Well, my post still holds true. As James points out, there’s an old guard of Guardian commentators (Polly Toynbee, please stand up) who would tell us that Labour was the only ‘progressive’ party if they were rounding up people earning less than £50k a year and sending them to death camps. Anyway, my post on the subject was at 6.16, and they didn’t release that article until 6.43, so there.

  15. @Valerie,

    I am sorry I did not see our earlier post.

    I’ll go check it out now. My own hunch is that it wont change hands but I will check again for you. :)

  16. @OXO

    Proof of the gullibility of youth. I think.

  17. The guardian is a pretentious rag. Plain and simple.

  18. I live in Manchester Withington.
    The seat blog is frankly a waste of time – dominated by a minor candidate with a grudge against the LDs (she was their candidate in 2001).
    At the beginning of the campaign I thought that the LD would struggle to hold this – but feel the tide is moving in their direction. The LD campaign is far from low key. I have been inundated with LD leaflets. Quite a high student vote – which is probably good for the LD’s. I suspect that Labour are now a little concerned about Manchester Gorton – although they should hold it if they don’t get too complacent.

  19. Eoin Clarke
    The guardian is a pretentious rag. Plain and simple.
    ********************************************
    I quite like the weekend supplement with the TV, film and book reviews in. I think it is called the Guide.

    I make a habit of switching my daily papers to avoid subliminal bias and get a fresh perspective.

    The papers I will buy are as follows; The Times, The Independant, The Guardian & The Telegraph (occasionnally).

    Papers I avoid; Express, Mirror, Mail, Mirror, Sun.

    rich

  20. The most interesting part of the ICM poll, from my partisan perspective of course, is the following passage in teh guardian:

    First it explained that the 3rd debate had had the smallest impact of all 3 in terms of changing anyone’s mind about who to vote for. But goes on

    “The Lib Dems continued to pick up more supporters than any other party, with 39% of the small number of switchers saying they were moving to Clegg’s party. But Labour was close behind on 33%, with 15% moving to the Conservatives.

    Overall, 36% said Brown would be the best prime minister, against 35% who named Cameron, and 23% Clegg. Brown’s lead over Cameron was larger as the leader most likely to make the decisions when the going gets tough, 45% to 32%; Clegg was third on 17%.”

  21. Here is how the parties stand in the North according to YouGov up till the 29th:

    North East
    Con 20.2
    Lab 39.6
    Lib 33.9
    Other 9.4
    103.1

    North West
    Con 29.3
    Lab 35.5
    Lib 29.1
    Other 6.3
    100.1

    York
    Con 28.3
    Lab 32.5
    Lib 31.0
    Other 7.3
    99.1

    These numbers are based on the movement of party % from the 25th to the 29th YouGov polls the applied to the 25th regional YouGov poll. Keep in mind that for some strange reason you added the % in the YouGov regional Ne added up to 103%, and Yorkshire adds to 99%.

  22. @Ben R

    Re Manchester Withington

    I’m loathe to indulge in “Well my friend said…” But I have come across people who wouldnt voe labour here last time cos of Iraq but are doing so this time because they are terrified of what will be in boy George’s budget.

  23. Any polls expected tonight?

  24. Grem 3 and Glen Otto

    My yougov “prediction” was simply an attempt to be humorous – hopefully most regulars saw it as that. But I suppose partisanship can be read into almost anything.

  25. mums net have just given brown and cameron a battering

    lol

  26. Richard O

    Yes it was (the car crash) just around the corner that was the present of metaphor to the journos.

    The driver had to avoid a rubbish cart that was cat-calling the Labour gathering, lost control and forced him to swerve.

    I am still reeling from that item -it’s without doubt the worst i’ve seen to date. All’s fair in love and politics but I do ahve a sense of justice and I do hope my reaction would have been the same whichever way it went and the worst coverage was Brown’s so i think I’ve proved that.

  27. Cricky we’re back to everyone knowing someone who is going to vote X Y Z because of A B or C…..

    I could quote lots for lots of reasons, some will be lying, some will change their mind and some won’t vote. Some will probably give me the answer I want because I have subliminally suggested it to them….

    All rubish, read the AW FAQ.

  28. @Jonty
    I havent seen the seat blog but have received a leaflet from lovely Yasmin. Nothing from the tories.

    I just dont know if the Lib Dem will hold on. I hope its too close to call, but what do I know?

  29. @JOHNTY
    I now believe more than ever Labour and their supporters will have some very rude shocks all over England & Wales. Many of their growing band of supporters on this site keep their spirits up by focusing entirely on UNS and a belief that the Tories need to be on 58.765% to get a majority. Of course there is nothing very favorable about Labours UNS figures, but Liberal strength in certain areas once solid Labour, plus Tory strength in the marginals (not as good as it was but still very handy) will leave Labour trailing in third on the popular vote.

  30. Guardian comes out in support of Lib Dems…

    ” Breaking News …on Sky “

  31. @YAKOBS

    :( seems my humour passed you by too.. I enjoyed your post – welcome lightheartedness!

    :):):)

  32. I’m taking this with a truckload of salt but this just got twoatted:

    ComRes shows CON-LAB-LD as 36-24-36

  33. “There’s no doubt that within MoE most pollsters produce reliable results”

    Do you have second sight or a time machine?

  34. @VALERIE
    re Withington
    I thought it was too close to call a couple of weeks back, but now sense a certain lack of belief in the Labour camp. It is very hard to take a seat against such a strong national swing – and incumbents generally have quite an advantage. The bookies make LD firm favourites.
    Looks to me as if the LD’s will pick up a number of Labour seats unless the polls are totally wrong.

  35. @EOIN CLARKE

    The Telegraph and Mail have not been that pro-Tory

    Thanks for that Mr Clarke, really made me laugh on cloudy, windy evening on my island

    @Goatboy

    I have noticed the same drift with the Guardian over the last 18 months (its the only broadsheet I can abide paying for)

    What about the FT? Other than on Saturday of course when it has not awful magazine!

  36. I am going to get in on this act. I know the chairman of a merchant bank in the city who is rank Labour and wants Ed Balls’s baby. I know a Colonel in the SAS who wants Labour in power for ever. So there.

  37. GPolly
    Does anyone who reads the Guardian not already know this would happen? They’ve treated brown with disdain throught the campaign. He’s received more hostile coverage there than in many traditional Tory papers. Expect the Indy to do likewise.

  38. Roland: “Well your post is a bit more realistic than a week or two ago?
    Whats happened to the genius of Prime Minister Brown shineing through to victory over the vaucous little twitt “Dave”?”

    Eh? Have you got the right Statto?

  39. Grem3

    Thanks for your post. I did wonder…….but then Glen Otto’s post made me wonder again. Clearly less wondering required!

  40. @Skudor,

    If that poll was true, Labour have had it. I can’t see it though.

    I’ll say it again, tonights and Sat’s polls will be very close to GE result.

    Who is going to be watching election stuff over the bank holiday weekend, especially with key footy, snooker etc on.

    Rich

  41. @ Skudor

    “I’m taking this with a truckload of salt but this just got twoatted:

    ComRes shows CON-LAB-LD as 36-24-36”

    Looks like Comres voter intention poll after leaders debate last night not widely reported. Is on Comres website.

  42. @Roland Haines,

    You make me laugh!!

  43. @ Skudor
    “I’m taking this with a truckload of salt but this just got twoatted:

    ComRes shows CON-LAB-LD as 36-24-36”

    That would be a very entertaining result alright but definitely salty – only 4 points for others is a bigger squeeze than I’d expect. I’ll wait and see the flock of polls later.

  44. @Matt/Roland

    I agree with Matt! :)

  45. Just to add, I agree with the Guardian. I’m an Indy reader, so you can probably guess I’m an LD centrist. However, what is interesting is how many people are blaming brown for the rightward drift of Noo Labour, whilst most of this drift occurred under his angelic predecessor. Where was the Guardian then?

  46. @Roland,

    lol.

    I know a 100 year old grandmother who has just received a letter from the Queen, she was a double war widow and also worked in the NHS for 50 years serving the public. She has just changed her vote to Conservative.

    sorry, that was childish!

  47. @ YAKOBS & GREM3

    Apparently SUE & I will be delighted that GLEN OTTO will not be posting on UKPR.

    I can’t speak for Sue, but I have no idea why his “departure” should delight me; if he was simply joining in the fun – then fair play to him. If he was serious, then I am baffled…….

    BTW – I found your comedy duo very amusing :-)

  48. @Skudor,

    Those results were shown on ITV ComRes last night, I believe, after the debate.

  49. Oops, I forgot my username, ah well, try again, last post in moderation:

    @ Skudor

    “I’m taking this with a truckload of salt but this just got twoatted:

    ComRes shows CON-LAB-LD as 36-24-36?

    Looks like Comres voter intention poll after leaders debate last night not widely reported. Is on Comres website.

  50. I don’t think that Comress twitt is anything more than either a joke or a false rumour – but if it were true I’d be ecstatic.

1 10 11 12 13 14