In this post back in January I wrote about the partisan effects of the different methodologies the different polling companies used, of how some companies tend to show consistently higher or lower scores for different parties. Since then I’ve been meaning to do a reference post explaining those different methods between pollsters. This is that – an attempt to do a summary of different companies methods in one place so you can check whether company A prompts for UKIP or what company B does with their don’t knows. As ever, this is from the published methodology details of each company and my own understanding of it – any mistakes are mine and corrections are welcome!

Phone polls

There are four regular telephone polls – Ipsos MORI, ICM, Ashcroft and ComRes/Daily Mail (ComRes do both telephone and online polls). All phone polls are conducted using Random Digit Dialing (RDD) – essentially taking phone numbers from the BT directory and then randomising the digits at the end to ensure the sample includes some ex-directory numbers, all polls will now also include some mobile phone numbers, though the pollsters have all said this makes little actual difference to results and is being done as a precaution. All telephone polls are weighted by some common demographics, like age, gender, social class, region, housing tenure, holidays taken and car ownership.

Ipsos MORI

Now the most venerable of the regular pollsters, Ipsos MORI are also the most traditional in their methods. They currently do a monthly political poll for the Evening Standard. Alone among GB pollsters they use no form of political weighting, viewing the problem of false recall as unsurmountable, their samples are weighted using standard demographics, but also by public and private sector employment.

MORI do not (as of March 2015) include UKIP in their main prompt for voting intention. For people who say don’t know, MORI ask who people who are most likely to vote for and count that equally as a voting intention. People who still say don’t know or won’t say are ignored. In terms of likelihood to vote, MORI have the tightest filter of any company, including only those respondents who say they are absolutely 10/10 certain to vote.

ICM

ICM are the second oldest of the current regular pollsters, and were the pioneer of most of the methods that became commonplace after the polling industry changed methods following the 1992 debacle. They currently do a monthly poll for the Guardian. They poll by standard demographics and by people’s past vote, adjusted for false recall.

ICM don’t currently include UKIP in their main voting intention prompt. People who say they don’t know how they will vote are reallocated based on how they say they voted at the previous election, but weighted down to 50% of the value of people who actually give a voting intention. In terms of likelihood to vote, ICM weight by likelihood so that people who say they are 10/10 certain to vote are fully counted, people who say they are 9/10 likely to vote count as 0.9 of a vote and so on. Additionally ICM weight people who did not vote at the previous election down by 50%, the only pollster to use this additional weighting.

Ashcroft

Lord Ashcroft commissions a regular weekly poll, carried out by other polling companies but on a “white label” basis. The methods are essentially those Populus used to use for their telephone polls, rather than the online methods Populus now use for their own regular polling. Ashcroft polls are weighted by standard demographics and by past vote, adjusted for false recall.

Ashcroft’s voting intention question has included UKIP in the main prompt since 2015. People who say they don’t know how they will vote are reallocated based on how they say they voted at the previous election, but at a different ratio to ICM (Ashcroft weights Conservatives and Labour down to 50%, Lib Dems down to 30%, others I think are ignored). In terms of likelihood to vote, Ashcroft weights people according to how likely they say they are to vote in similar way to ICM.

ComRes

ComRes do a monthly telephone poll, previously for the Independent but since 2015 for the Daily Mail. This is separate to their monthly online poll for the Independent on Sunday and there are some consistent differences between their results, meaning I treat them as two separate data series. ComRes’s polls are weighted using standard demographics and past vote, adjusted for false recall – in much the same way as ICM and Ashcroft.

ComRes have included UKIP in their main voting intention prompt since late 2014. People who say they don’t know how they will vote or won’t say are asked a squeeze question on how they would vote if it was a legal requirement, and included in the main figures. People who still say don’t know are re-allocated based on the party they say they most closely identify with, though unlike the ICM and Ashcroft reallocation this rarely seems to make an impact. In terms of likelihood to vote ComRes both filter AND weight by likelihood to vote – people who say they are less than 5/10 likely to vote are excluded completely, people who say they are 5/10 to 10/10 are weighted according to this likelihood.

Online Polls

Online poll sampling can be somewhat more opaque than telephone sampling. In most cases they are conducted through existing panels of online volunteers (either their own panels, like the YouGov panel or PopulusLive, or panels from third party providers like Toluna and Research Now). Surveys are conducted by inviting panellists with the required demographics to complete the poll – this means that while panels are self-selecting, surveys themselves aren’t (that is, you can choose to join a company’s online panel, but you can’t choose to fill in their March voting intention survey, you may or may not get randomly invited to it). Because panellists demographics are known in advance, pollsters can set quotas and invite people with the demographics to reflect the British public. Some pollsters also use random online sampling – using pop-ups on websites to randomly invite respondents. As with telephone polling, all online pollsters use some common demographic weighting, with all companies weighting by things like age, gender, region and social class.

YouGov

YouGov are the longest standing online pollster, currently doing daily voting intention polls for the Sun and Sunday Times. The length of time they have been around means they have data on their panellists from the 2010 election (and, indeed, in some cases from the 2005 election) so their weighting scheme largely relies on the data collected from panellists in May 2010, updated periodically to take account of people who have joined the panel since then. As well as standard demographics, YouGov also weight by newspaper readership and party identification in 2010 (that is, people are weighted by which party they told YouGov they identified with most in May 2010, using targets based on May 2010).

YouGov have included UKIP in their main prompt since January 2015. They do not use any weighting or filtering by likelihood to vote at all outside of the immediate run up to elections (in the weeks leading up to the 2010 election they weighting by likelihood to vote in a similar way to Ashcroft, Populus and ICM). People who say don’t know are excluded from final figures, there is no squeeze question or reallocation.

Populus

Populus used to conduct telephone polling for the Times, but since ceasing to work for the Times have switched to carrying out online polling, done using their PopulusLive panel. Currently they publish two polls a week, on Mondays and Fridays. As well as normal demographic weightings they weight using party identification, weighting current party ID to estimated national targets.

Populus have included UKIP in their main prompt since February 2015. They weight respondents according to their likelihood to vote in a similar way to ICM and Ashcroft. People who say don’t know are excluded from final figures, there is no squeeze question or reallocation.

ComRes

Not to be confused with their telephone polls for the Daily Mail, ComRes also conduct a series of monthly online polls for the Independent on Sunday and Sunday Mirror. It is conducted partially from a panel, partially from random online sampling (pop-ups on websites directing people to surveys). In addition to normal demographic weightings they weight using people’s recalled vote from the 2010 election.

ComRes have included UKIP in their main prompt since December 2014. Their weighting by likelihood to vote is slightly different to their telephone polls – for the Conservatives, Labour and Liberal Democrats it’s the same (include people who say 5+/10, and weight those people according to their likelihood) but for UKIP and Green I believe respondents are only included if they are 10/10 certain to vote. Their treatment of don’t knows is the same as in their phone polls: people who say they don’t know how they will vote or won’t say are asked a squeeze question and included in the main figures, people who still say don’t know are re-allocated based on the party they say they most closely identify with.

Survation

Survation do a regular poll for the Daily Mirror and occasional polls for the Mail on Sunday. Data is weighted by the usual demographics, but uses income and education rather than social class. Recalled 2010 vote is used for political weighting. Survation have included UKIP in their main prompt for several years. They weight by likelihood to vote in the same way as ICM, Populus and Ashcroft. People who say don’t know are reallocated to the party they voted for in 2010, but weighted down to 30% of the value of people who actually give a voting intention.

Note that Survation’s constituency polls are done using a completely different method to their national polls, using telephone sampling rather than online sampling and different weighting variables.

Opinium

Opinium do regular polling for the Observer, currently every week for the duration of the election campaign. Respondents are taken from their own panel and is weighted by standard demographics. Historically Opinium have not used political weighting, but from February 2015 they switched to weighting by “party propensity” for the duration of the election campaign. This is a variable based on which parties people would and wouldn’t consider – for practical purposes, it seems to be similar to party identification.

Opinium do not include UKIP in their main prompt (meaning they only appear as an option if a respondent selects “other”). They filter people by likelihood to vote, including only respondents who say they will definitely or probably vote. People who say don’t know are excluded from the final figures.

TNS

TNS are a huge global company with a long history in market research. In terms of public opinion polling in this country they are actually the successors to System Three – who used to be a well known Scottish polling company and ended up part of the same company through a complicated series of mergers and buy-outs by BMRB, NFO, Kantar and WPP, currently their ultimate parent company. At the last election TNS were the final company doing face-to-face polling, since then they have switched over to online. The sample is taken from their Lightspeed panel and is weighted using standard demographics and recalled 2010 vote. TNS do include UKIP in their main prompt, and also prompt for the BNP and Green. TNS filter and weight people according to likelihood to vote and exclude don’t knows and won’t says from their final figures.

Putting all those together, here’s a summary of the methods.

methods2

As to the impact of the different methods, it not always easy to say. Some are easy to quantify from published tables (for example, ICM and Ashcroft publish their figures before and after don’t knows are reallocated, so one can comfortably say “that adjustment added 2 points to the Lib Dems this week”), others are very difficult to quantify (the difference the choice of weighting regimes makes is very difficult to judge, the differences between online and telephone polling even more so), many methods interact with one another and the impacts of different approaches changes over time (a methodology that helps the Tories one year may help the Lib Dems another year as public opinions change). Rather than guess whether each pollsters methods are likely to produce this effect or that effect, probably best to judge them from actual observed results.

UPDATE:
TNS have confirmed they do prompt for UKIP, and also prompt for the BNP and Green – I’ll update the table later on tonight.


Monday polls

I think we have just the three regular polls this Monday – Populus, Ashcroft and YouGov. ComRes this afternoon announced that their voting intention polls for the Daily Mail will be weekly for the rest of the campaign, but the first of those won’t be until later in the week. March’s Survation/Mirror poll is also due sometime this week, but I don’t know when.

  • Populus this morning had topline figures of CON 31%, LAB 33%, LDEM 9%, UKIP 16%, GRN 5% (tabs).
  • Ashcroft meanwhile has figures of CON 33%, LAB 33%, LDEM 8%, UKIP 12%, GRN 5% (tabs).

I’ll update with the YouGov poll later, but with seven polls conducted since the budget I think we can conclude it’s had no effect.


-->

On Friday we had YouGov and Populus polls taken after the budget. YouGov showed a slight movement to the Tories, putting them back ahead; Populus showed a shift to Labour. Neither was anything that couldn’t just be normal random variation. Today we have three more polls too see if there is any sign of a consistent budget effect.

Opinium in the Observer have topline figures of CON 36%, LAB 33%, LDEM 7%, UKIP 14%, GRN 6% (tabs) – a three point Tory lead compared to a two point Labour lead a week ago. Opinium also found 43% thought Osborne had been a good Chancellor, 24% a poor one, and on economic trust Cameron & Osborne now have a 21 point lead over Miliband & Balls. Taken in isolation this poll would suggest a budget boost for the Conservatives, which is how the Observer have reported it

But of course, we don’t have to take the poll in isolation. The second poll of day is from Survation for the Mail on Sunday. They have topline figures of CON 30%(+2), LAB 34%(nc), LDEM 10%(nc), UKIP 17%(-2), GRN 4%(nc) (tabs) – changes are from the most recent Survation poll, conducted a month ago for the Mirror. Here we have a slight shift towards the Conservatives – but thats over a month that has seen Labour’s lead fall slightly anyway, and it’s not enough to stop Labour having a clear lead.

Finally there is the weekly YouGov poll for the Sunday Times. Topline figures there are CON 33%, LAB 35%, LDEM 8%, UKIP 14%, GRN 5% (tabs) – so back to a Labour lead with YouGov. YouGov’s regular economic trackers on whether people think the economy is in a good state and whether their own finances will get better over the next year also look pretty much unchanged since before the budget.

Putting all the five post-budget polls together, I can see no sign of any significant budget boost. If other polls had echoed Opinium’s finding then it would be fair to conclude that the budget had moved votes to the Tories, but so far they haven’t – Survation have shown only a twitch in the Conservative direction, YouGov looks stable, Populus’s Friday poll showed movement to Labour. This all looks to me like normal random variation. I may be wrong, perhaps when we’ve a week of post-budget polls we’ll be able to detect some more subtle movement, but it certainly doesn’t look like it’s been some great game changer.


Seven Weeks to Go

Here are this week’s polls:

Opinium/Observer (12/3) – CON 33%, LAB 35%, LDEM 7%, UKIP 14%, GRN 7%
YouGov/S Times (13/3) – CON 34%, LAB 34%, LDEM 7%, UKIP 14%, GRN 5%
ComRes/Indy on Sunday (13/3) – CON 33%, LAB 35%, LDEM 7%, UKIP 16%, GRN 4%
ICM/Guardian (15/3) – CON 36%, LAB 35%, LDEM 8%, UKIP 9%, GRN 4%
Populus (15/3) – CON 34%, LAB 34%, LDEM 8%, UKIP 14%, GRN 5%
Ashcroft (15/3) – CON 31%, LAB 29%, LDEM 8%, UKIP 15%, GRN 8%
YouGov/Sun (16/3) – CON 33%, LAB 35%, LDEM 7%, UKIP 13%, GRN 7%
TNS BMRB (16/3) – CON 33%, LAB 32%, LDEM 7%, UKIP 17%, GRN 4%
YouGov/Sun (17/3) – CON 34%, LAB 36%, LDEM 7%, UKIP 12%, GRN 6%
YouGov/Sun (18/3) – CON 33%, LAB 34%, LDEM 8%, UKIP 14%, GRN 6%
YouGov/Sun (19/3) – CON 35%, LAB 33%, LDEM 8%, UKIP 13%, GRN 6%
Populus (19/3) – CON 31%, LAB 34%, LDEM 9%, UKIP 17%, GRN 5%

After a wafer thin Tory lead in the UKPR polling average last week, this week Labour and Conservative are neck-and-neck: CON 33%(nc), LAB 33%(+1), LDEM 8%(+1), UKIP 14%(-1), GRN 6%(nc). The consistent small Labour lead we had for most of the year has vanished, but the Conservatives have not managed to open one of their own.

Scottish and constituency polls

Survation put out a new Scottish poll this week (in fact they put out new – a new one for the Daily Record, and an older one for Unison). Both showed the SNP lead in Scotland holding strong, and no sign of any movement back to Labour. There was also a new batch of Ashcroft polling which I discussed here

Budget Week

This week was the most significant fixed event of the year before the general election. We still have the parties’ manifesto launches and (perhaps) a debate or debates of some sort, but the budget was indisputably one of the most important known unknowns that could potentially change public opinion before the election. The commentariat invariably spend the run up to the budget speculating about what rabbits the Chancellor will pull out of the hat to make a major impact on the election – history suggests it very rarely happens. There are examples of government support slumping after bad budgets – one of the biggest shifts in public opinion this Parliament came after the bodged 2012 “omnishambles” budget – but precious few of them of them giving a government a substantial boost.

We are not yet in a position to judge whether or not the 2015 budget had any effect. The initial post-budget polling from YouGov gave it a thumbs up – more people thought it was fair than unfair, more people thought it would make them better off than worse off. Most however thought it made no difference, and even a positive reaction does not automatically lead to any impact on the polls. The two voting intention polls we’ve seen since the budget do not point to any big shift – YouGov showed the Conservatives up, but Populus showed them down.

Beyond the budget there was another tiptoe forward in the debate debate. The broadcasters are reportedly now offering one seven way debate on ITV, a Paxman grilling of Miliband and Cameron on Channel4/Sky and two Question Time style events on BBC – one for UKIP, the SNP, Plaid and the Greens, one with Miliband, Clegg and Cameron, but separately. Cameron has agreed to the seven way debate, but all else seems unconfirmed. The first event – the Paxman grilling – is pencilled in to take place next Thursday though, so one way or another, decisions will be made next week.

Projections

The latest forecasts from Election Forecast, May 2015, Elections Etc and the Guardian are below, as well as the new model from the Polling Observatory team. As usual all the models predict a hung Parliament – most have the Tories with marginally more seats than Labour, Polling Observatory have Labour with a 20 seat lead.

Elections Etc – Hung Parliament, CON 284(-1), LAB 278(-1), LD 21(-1), SNP 41(+1), UKIP 3(nc)
Election Forecast – Hung Parliament, CON 284(-4), LAB 278(+7), LD 25(-1), SNP 40(-2), UKIP 1(nc)
May 2015 – Hung Parliament, CON 277(-4), LAB 268(+5), LD 24(nc), SNP 55(nc), UKIP 3(-1)
Guardian – Hung Parliament, CON 277(-2), LAB 268(+3), LD 25(-2), SNP 54(+1), UKIP 4(nc)
Polling Observatory – Hung Parliament, CON 265, LAB 285, LDEM 24, SNP 49, UKIP 3


The rest of the the YouGov budget polling is now up on their website here. It suggests a broadly positive reaction to the budget and a significant jump in Osborne’s own ratings. Overall 42% think the budget was fair, 27% unfair. Most people think it will make little difference to the country or to their own finances, but of those who do there more think it will have a positive impact than a negative one. 40% of people think Osborne is doing a good job as Chancellor (up 6 from before the budget) and his lead over Ed Balls on who would make the best Chancellor is up to 20 points.

Just because something is approved of though, it doesn’t necessarily mean it has any effect on voting intention. The second of this week’s Populus polls, also conducted entirely after the budget, has topline figures of CON 31%, LAB 34%, LDEM 9%, UKIP 17%, GRN 5%. No sign of a budget bounce there – but I’ll repeat the same caveats I made after the YouGov poll last night. We could just be seeing random variation, and news events don’t have an instant effect anyway – these two polls were after the budget, but many responses would have been before the media reaction (just because something has already happened, doesn’t mean the respondent already knows about it). Wait and see if the broader average moves in the week ahead.