As we get closer to the election there will be more and more attention paid to polls, and sadly there will be more and more comments like those below – look, for example, at the horrors that crop up in the BBC “Have Your Say” section. Clearly none of you, my dear readers, would ever leave comments like these, but just in case you come across them elsewhere…

1) The polls are ALL wrong, the real position is obviously X

Er… based on what? The reality is that opinion polling is pretty much the only way of measuring public opinion. We have some straws in the wind from mid-term elections, but they tend to be low turnout protest votes, don’t tend to predict general election results and are anyway quite a long time ago now. Equally a few people point to local government by-elections, but when compared to general election results these normally grossly overestimate Liberal Democrat support. If you think the polls are wrong just because they “feel” wrong to you, it probably says more about what you would like the result to be than anything about the polls.

2) I speak to lots of people and none of them will vote for X!

Actually, so do pollsters, and unless you regularly travel around the whole country and talk to an exceptionally representative demographic spread of people, they do it better than you do. We all have a tendency to be friends with people with similar beliefs and backgrounds, so it is no surprise that many people will have a social circle with largely homogenous political views. Even if you talk to a lot of strangers about politics, you yourself are probably exerting an interviewer effect in the way you ask.

3) How come I’ve never been invited to take part?

There are about 40 million adults in the UK. Each opinion poll involves about 1,000 people. If you are talking about political voting intention polls, then probably under 100 are conducted by phone each year. You can do the sums – if there are 40,000,000 adults in the UK and 100,000 are interviewed for a political opinion poll then on average you will be interviewed once every 400 years. It may be a long wait.

4) They only interview 1000 people, you’d need to interview millions of people to make it accurate!

George Gallup used to use a marvellous analogy when people raised this point: you don’t need to eat a whole bowl of soup to tell if it is too salty, providing it is sufficently stirred a single spoonful will suffice. The same applies to polls, providing an opinion poll accurately reflects the whole electorate (e.g, it has the right balance of male and female, the right age distribution, the right income distribution, people from the different regions of Britain in the correct proportions and so on) it will also accurately reflect their opinion.

In the 1930s in the USA the Literary Digest used to do mail-in polls that really did survey millions of people, literally millions. In 1936 they sent surveys to a quarter of the entire electorate and received 2 million replies. They confidently predicted that Alf Landon would win the imminent US Presidential election with 57% of the popular vote and 370 electoral votes. George Gallup meanwhile used quota sampling to interview just a few thousand people and predicted that Landon would lose miserably to Roosevelt. In reality, Roosevelt beat Landon in a landslide, winning 61% of the vote and 523 electoral votes. Gallup was right, the Digest was wrong.

As long as it is sufficent to dampen down sample error, it isn’t the number of people that were interviewed that matters, it is how representative of the population they are. The Literary Digest interviewed millions, but they were mainly affluent people so their poll wasn’t representative. Gallup interviewed only a few thousand, but his small poll was representative, so he got it right.

5) Polls give the answer the people paying for it want

The answers that most clients are interested in are the truth – polls are very expensive, if you just wanted someone to tell you what you wanted to hear there are far cheaper sources of sycophancy. The overwhelming majority of polling is private commercial polling, not stuff for newspapers, and here clients want the truth, warts and all. Polling companies do political polling for the publicity, there is comparatively little money in it. They want to show off their accuracy to impress big money clients, so it would be downright foolish for them to sacrifice their chances with the clients from whom they make the real money to satisfy the whims of clients who don’t really pay much (not to mention that most pollsters value their own professional integrity too much!)

6) Pollsters only ask the people who they know will give them the answer they want

Responses to polls on newspaper websites and forums sometimes contain bizarre statements to the effect that all the interviews must have been done in London, the Guardian’s newsroom, Conservative Central Office etc. They aren’t, polls are sampled so they have the correct proportion of people from each region of Britain. You don’t have to trust the pollsters on this – the full tables of the polls will normally have breakdowns by demographics including region, so you can see just how many people in Scotland, Wales, the South West, etc answered the poll. You can also see from the tables that the polls contain the right proportions of young people, old people and so on.

7) There is a 3% margin of error, so if the two parties are within 3% of each other they are statistically in a dead heat

No. If a poll shows one party on 46% and one party on 45% then it is impossible to be 95% confident (the confidence interval that the 3% margin of error is based upon) that the first party isn’t actually on 43%, but it is more likely than not that the party on 46% is ahead. The 3% margin of error doesn’t mean that any percentage with that plus or minus 3 point range is equally likely, 50% of the time the “real” figure will be within 1 point of the given figure.

8 ) Polls always get it wrong

In 1992 the pollsters did get it wrong, and most of them didn’t cover themselves in glory in 1997. However, lessons have been learnt and the companies themselves have changed. Most of the companies polling today did not even exist in 1992, and the methods they use are almost unrecognisable – in 1992 everyone used face-to-face polling and there was no political weighting or reallocation of don’t knows. Today polling is either done on the phone or using internet panels, and there are various different methods of political weighting, likelihood to vote filtering and re-allocation of don’t knows. In 2001 most of the pollsters performed well, and in 2005 they were all within a couple of points of the actual result, with NOP getting it bang on.

9) Polls never ask about don’t knows or won’t votes

Actually they always do. The newspapers publishing them may not report the figures, but they will always be available on the pollsters’ own website. Many companies (such as ICM and Populus) not only include don’t knows in their tables, but estimate how they would actually vote if there was an election tomorrow and include a proportion of them in their topline figures.

(There should be an ICM poll later on tonight as well as the YouGov poll at around 10pm – I’ll probably be in a meeting for the ICM poll at least, but will update once I am free)


182 Responses to “Too frequently asked questions”

1 2 3 4
  1. ICM 8%- seems a solid reflection of what most of the polls have been saying…

    Is that still hung parlaiment territory can someone inform me?

    Also was the poll gathered post cameron interview?

  2. Would be convenient to think so Roland but the fieldwork dates suggest not. Also doesn’t look like the Cameron interview would have been factored in either. Not a great poll for the Labour Party.

  3. oops I see its a nine…

    Tory 40%

    Lab 31%

    That seems very close to what most of the polls record for the big two…

    is there soemthing in this poll to keep both sides relatively happy I wonder?

    Labour over 30% firmly now it seems

    and Tories halting thier ‘aparent’ dip…

    can’t both camps be happy? :)

  4. Eoin, on straight UNS it would be Tory largest party in a hung parliament. If we assume a couple of percent better in the marginals, then its a squeaker for a majority.

  5. I see th efieldwork was pre-Cameron interview, can we expect another bounce?

    The yougov tonight will be extremely interesting

    I predict a

    38 Tory
    32 Lab

  6. Neil A

    Can you explain to me what % the Tories would need then to get around 20 seats etc.

    I always thought 40% ish was the magic figure for them….?

  7. Opinium/Daily Express

    Con 39 +2 Lab 28 -2 Lib-Dem 16 nc

  8. I see Ashok Kumar has died,

    Condolences aside, I wonder what the impact is on the constituency vote when an incumbent suffers a tragedy such as this?

    Historically, do the incumbent party get a bounce?

    I apoligise if this is insentitive but I only ask because we have had more than one fatality among member sof parliament recently….

  9. @EDWARD CARLSSON BROWN
    The 2% “bonus” in the marginals is, I think I am right in saying a YG 2%. There was an earlier AR marginal poll which was roundly poo pooed by a number of posters. If the actual Tory advantage in the marginals was somewhere between the pair of them, things would take a different complexion. Many serious poll watchers believe a figure between YG & AR is a sensible bet in any case.

  10. @ Colin – there’s just no acknowledgement of a reasonable analysis from you is there?

    Yes-definitely Sue ??

    I thought you said it’s “wait & see”.

    I hadn’t got that down as “analysis”.

    I have re-read -& now see you said “I however, don’t really believe news stories or interviews influence the polls much,”

    To be honest that didn’t count as “analysis” either. I disagree with the idea-what else are people supposed to be influenced by?

    Why does Anthony spend so much time putting date lines on recent events to count them before or after polling dates?

  11. i honestly don’t believe small events have much impact on the polls. The vast majority of the public couldn’t tell you who Ashcroft is or even probably George Osborne or why BA are striking.
    They don’t care either.
    Those who ARE affected, usually have pretty strong party allegiance and keep up with current affairs. Ashcroft for instance won’t have changed any of their views. They will still either believe he is a corrupt Tory who is trying to buy seats (Lab view) or a donor like all the others who can do what he likes about his own tax affairs (Con view)
    Trend however, picks up something deeper for me. First, it takes an average view of ALL the polls and “rogue” results 9 or 2 are evened out. Secondly, it picks up on an overall “mood” and I think captures the feel of the election better.

    We are back to wait and see I think.

  12. Actually, what I’m really not looking forward to on election night is seeing all the tory triumphantalists hanging out of the windows at Tory HQ. Say you won’t be there COLIN pleeeeze

  13. Dave in SH,

    On 41/30/20 the Tories get a majority of 18 on UNS. (I stuck this in because I assume marginal effect will help them slightly and hinder Labour slightly, but be counteracted by the LibDem campaign swing).

    Its hard to say, the Others share is a little lower in this ICM poll than recently seen. As its the gap with Labour that is the determinant for a majority (mostly), having less Others means the Tories need a higher share of what’s left.

  14. Cashcroft is such a difficult concept to get your head around. A duck pond isnt, a con-dom seems trivial by comparison. I cannot imagine UNITe or Cashcroft infleunceing th eoutcome of the election.

    More likely to be 2.4million unempoyed and 178£ deficits that will decide….

  15. Eoin Clarke. See Opinium – Labour 28%. Not quite so firm perhaps.

  16. There has been months of slating AR polls as way off the mark, but AR seems to be closer to the other pollsters (ICM etc) than YG is. Although not scientific, the YG results also do not ‘intuitively’ align with the state of the economy.

    Is YG going to end up more embarrased than Brown by the results of the forthcmoing election?

  17. @ GIN
    “Opinium/Daily Express

    Con 39 +2 Lab 28 -2 Lib-Dem 16 nc

    Thanks

    This is getting interesting.

    Can’t wait for Anthony’s analysis of both of these polls.

    I wish Peter Kellner hadn’t said “events” take a week to affect Polls-that really only allows Chilcott & the Afghanistan visit-I have some more recent candidates.

    All 20/20 hindsight of course !

  18. Before slating YG lets see what their poll today shows. Perhaps there has been a slight Tory lift that they will also pick up. I don’t think its anything to do with events, more that the string of bad publicity for the Tories has abated.

  19. Thanks Neil A

    :-)

  20. Howard

    The language of political discourse is always interesting.

    The term “lefties” is one I have never encountered except when I read comments on UK blogs. Is it particularly English, or is it used in Wales as well?

    What precisely does it mean?

  21. @VALERIE
    So glad you are foreseeing the final result clearly. Colin & myself will not be hanging out of widows never fear. We will be totally messed up under a table somewhere.

  22. Interesting swing back to the Cons in the polls.

    I wonder whether Labour is now suffering in the same way that the Cons have. i.e just as people who did not want to see a Con government have falled back in line with Labour, those who can not bear the idea of another 5 years of Brown/Labour are returning to the Cons?

    If you are a UKIP’er, ultimately you are going to have to choose between wasting your vote on UKIP or supporting Lab or Con. Most of this group will probably consider Con to be most sceptical.

    The prospect of another 5 years of Lab may benefit the Con’s by encouraging their supporters to go vote.

  23. @ VALERIE

    ” Say you won’t be there COLIN pleeeeze”

    With pleasure
    I shall be clutching a nice glass of red on my sofa-becoming vair vair drunk. ;-)

    BUT….we are a VERY LONG WAY from that .

    THere is a mountain for Mr Cameron to climb first.

  24. Dave in SH, have you played around with AW’s pretty map yet? Its on the left of the page under “Advanced Swingometer”. Its mesmerising fun and you can check out UNS swings in beautiful colours (well, and red of course)….

  25. @Colin,

    Its getting to the stage where the difference in polling figures is so big that a rumpus is unavoidable.

    Ask yourself this, would th esecond biggest party logically attract 28% of the vote. Step outsdie your bubble and think logically. Surely even you can see a plus 30%.

    nobody is saying 34% is gospel…. I certainly for one don’t think it is. But if you, or opinium, want their opinions to be taken more credibly, they need to tune into reality.

    ICM have others at 9% if I am not mistaken. thats very close to half what AR had others.

    I think an ICM 9% is a solid reflection of the trends we ahve been seeing…

    I think Tinky winky and La La need to get real.

  26. Apologies for my spelling, I should say WINDOWS.

  27. @ ROLAND
    “We will be totally messed up under a table somewhere”

    Chahs !!

  28. @ AL J, SUE

    UKippers who hate Brown move to CON when the lead narrowed to much for them to stomach, is my suggestion/ question?

    I am looking forward to the detail being available so that I can check it out; I’d like to see if no. of don’t knows & won’t votes has changed. The chart gives the impression it’s a direct movement from Other Parties -4 to CON +3 & LAB +1; but that may not be the whole truth of the matter.

    Before LAB folks despair there are 3 things that won’t be affecting this poll (from the dates):
    1. LD potential switch to LAB following coverage of Nick Clegg’s seemingly pro-CON stance.
    2. LD at 20% is high compared to recent YG (17).
    3. LAB still have some climb-back potential. The expectation of a CON government usually galvanises some waverers.

    Let’s see what tonight’s YG brings.

  29. Oldnat,

    In my experience “lefties” is usually, but not always, an affectionate term used by normal people to describe what virulently political people would call “Socialists” (and I mean both the Lefties/Socialists themselves and their opponents use the term).

  30. @amber,

    if there really is a swing back to the Tories, which is far from certain, then I think you’re right. It’s more likely to be a reshuffling of the right than any hearts and minds being won back over from the middle.

  31. @Amber

    others are so low now that you may have a point

  32. I havn’t yet Neil but I’ll have a play later….thanks.

    Got to put the kids to bed now………then try to avoid the red wine

  33. EOIN

    “But if you, or opinium, want their opinions to be taken more credibly, they need to tune into reality.”

    I think that counts as an 8.

    It just the sort of pointless stuff which stretches for miles on here sometimes.

    I have no idea who is right & who is wrong EOIN.

    Yes there is a gap between YG & most othersobviously.

  34. @Eoin Clarke
    “Ask yourself this, would the second biggest party logically attract 28% of the vote.”

    Well it is perfectly plausable. Labour got less than 28 in 1983 and with the large amount of hate towards the main political parties, why is it not plausable for an increased percentage of others.

    However I do expect the others percentage to go down as the election properly kicks in.

  35. @SUE MARSH
    Sue, it might become a bit difficult for you if these results continue. However, some people are constantly attacking “the enemy” over the most ridiculos trivia and its very annoying, – ignore.
    I dont mean a real issue like troops equipment but stupid things no one but a wonk would know.

  36. @Colin

    7-9%… right on the marker of hung parl territory

    it seems three possiblities are now open

    a small Tory majority enough to form a gov.
    a hung parl with Tory the largest party (needing unionists)
    a neck and neck labour tory love in… maybe their do what Merkel and the SPD agreed in germany… then you lot can all be in the one aprty. :)

  37. @Rob

    Thatcher was a conviction politician, whom the public could understand, if not like.

    We are hardly in those times now Rob.

    winter of discontent
    falklands war
    hunger strike
    thatcher taking on the unions

    cameron’s amiguity is unlikely to polarise opinion to that extent…..

  38. Anthony, some of the questions raised are valid. For example look at the London Mayor election, all the polls were quite a way out and it would be reasonable to suggest it was based on those same questions. One poll got it right and credit to them.

    Look at the difference between your polls on your site over a week, Labour support is fluctuating by 10 points…that isnt reliable polling…it is ‘havent got a clue’ polling.

    There is no way in a million years the country would vote for Gordon Brown to stay on as PM, he is Cameron’s best asset. With the exception of the heartlands, he is hated throughout the country. Historically, UK people like change.

    Your site seems to be left leaning…again all that does is help the Tories as more of them will get out on election day and consequently Cameron will get a 100 seat majority…guaranteed.

  39. Old Nat Lefties
    Actually I understand (and this will impress Eoin) that it is used in NI to designate a republican.

    I’ll be quite frank I have no idea why it means something to me (socialists) so it must be locally significant.

    I have no idea what it means in Wales.

  40. @ Eoin

    “cameron’s amiguity is unlikely to polarise opinion to that extent…..”

    Well Thatcher didnt exactly pollarise support either in that election. Her share of the popular vote was down from 79. The main reason for the drop in labour support was the Lib/SDP alliance which achieved 25%. Never say never.

  41. I think the difficulty most of us have with AR & Opinium is the size of their “Other Party” compared to other polls & history of voting in the UK.

    Opinium – based of the figures provided of 39, 28, 16 = Others 17. I would suggest this is a bigger ‘tragedy’ for LD than LAB. 17% of voters abandoning the big 2 & none of them choosing LD. Actually LD losing some of their 2005 to the Other Parties.

    Frankly, I am amazed by these results. If I was a LD supporter, I would be devastated.

  42. @AMBER STAR
    We are in complete agreement regarding concepts here Amber. The difference is the galvinisation, I think perhaps Labour were looking to close for comfort.

  43. @Howard,

    very good howard,

    now would you like to know what it means in the south of ireland…….? Bearing in mind that the most left wing party down there is now considering a coalition with the blueshirts

  44. People are quoting detail from the ICM poll – I looked at its web site and I can’t find that. I got the impression that it only gives detail of teh one before last. Can anyone help?

    Also I noticed that the last two were carried out at nearly the same time -or have I got that wrong as well?

  45. @Rob Hist,

    I normally like you rblogs Rob…. but by muther of all gods, Thatcher polarised opinion…

    Add the other two parties got 53% lol lol lol :)

    Hang your head in shame (tongue in cheek I hope by now you know)

  46. @ ROLAND HAINES

    stupid things no one but a wonk would know.
    ———————————————————
    I agree – when folks post little tid-bits from the FT about obscure economic, trade & market data it makes me :-) As if anybody will look at such things & say:
    ‘Uk down-cycle trade deficit denominated in USD has shown an adverse movement of 5 points on the plonker scale – that’s made up my mind, I’m voting for Esther Rantzen’ ;-)

  47. AMBER – had a half of Amber Ale earlier, brought you to mind :)

    It will indeed be interesting if YouGov detects a slight swing back to the Conservatives. Many people expected this in the light of a couple of recent events; they would probably have occurred within the fieldwork for the poll. Most people however have doubted whether the swing would have any finality about it, if there were to be one. There’s no reason why Labour couldn’t close the gap again in most observers’ eyes, though some (not necessarily all partisan Tories) would disagree.

  48. @rob, the tories got a higher perecntage in 79.

    In 83 Thatcher was just able to rouse the rabble…. No need for BNP/UKIP back then

  49. Amber

    Thanks for that great analysis. I feel you have highlighted some important points. If we also consider our organization skill which is moving into gear at the moment –we should be upbeat. ;-) As you rightly mentioned before, (which was vindicated by further closing of the polls), there will be wobbles along the way. I’m awaiting tonight’s YouGov in anticipation.

  50. I think on the whole, I prefer not to dismiss any of the polls on reflection. If YouGov and Bpix have the gap at 3-4, “others” have the gap at 6-7 and AR and Opinium have te gap at 11-13, then that probably points pretty decisively at “others” as the winners.

1 2 3 4