The UKPollingReport Polling Average
Current UKPR Polling Average
Polls currently included in the average
|Ipsos-MORI/Evening Standard||2016-11-14||42||33||10||Con +9||0.33|
UKPollingReport has always shied away from producing a rolling average of polls. However, there are lots of polls of polls out there, lots of rolling averages, weighted averages and so on and, when it comes to a general election, its natural that people who don’t want to worry about which pollster is right or which pollster does this or that to their figures will want a single figure showing where the parties stand. Since people are going to do an average whether I like it or not, I may as well do it how I think it should be done.
The best way to judge the polls is to take the broad picture, not an average, look at the general trends and if they are showing a contrasting picture look at the possible reasons why. I hope that is something that UK Polling Report does for readers. If you don’t have time for that though, and just want a simple overall figure that tells you how the parties are doing, then here is the UKPollingReport Polling Average.
The UKPR Polling Average takes in polls from the last 20 days and gives them weightings based on various factors, including how recently they were conducted, the past record of the pollster producing the figures, the methodology used, the sample size and how many polls have been produced by a single pollster. The detailed figures are at the bottom of the page. These are of course a matter of opinion, particularly my preference for polls that have been politically weighted,
As I would expect – the polling average is not better than the best individual pollster at the last three elections (this is a major reason why I don’t like polling averages – by averaging quality polls will ropey ones you don’t get better figures, you just make the better ones worse) but of course, in hindsight we don’t know who the best pollster will be. Since different pollsters use different methods, factor in don’t knows and likelihood of voting in different ways, there is no statistical reason why an average of polls should give you a better idea of where things stand.
What the average does do, is give a nice shorthand for how the polls are at the moment and it is a prerequisite for giving a projection of what the result would be in an election tomorrow – for that see the UK Polling Report current predictions.
Time: A poll will receive a weighting of 1 on the day the fieldwork is completed – after that the weighting decays by 0.05 a day, so that on the 21th day the poll is given a weighting of zero and dropped from the calculation.
Track record: Track record is based on how close a pollster came to the election result at the last two general elections. The rating is based on the average of all their polls done within 5 days of the last general election, and the previous election carries more weight that than the one before that. If a pollster did not conduct a poll at the previous election it is is given a dummy rating instead, if they conducted a poll at the last election, but the fieldwork was more than 5 days before polling day, their weighting factor is averaged with the dummy score.
Methodology: UKPolling Report believes that the most reliable polls are those which include some form of political weighting, and polls that do not weight by past vote, party ID or similar are given an additional weighting 0.75. This is a matter of contention amongst pollsters, with some companies – primarily Ipsos MORI – believing that past vote is not suitable for weighting. That is a legitimate viewpoint done for good reasons, but it is not one I share, and hence this particular average weights down polls without political weighting. Does that mean this is biased against polls that don’t use past-vote weighting? Yes, it does and if someone else wants to do an average that treats them the same go ahead.
Transparency: A key part of knowing how much trust to place is a poll is the openness of its methodology and results. Companies who are part of the British Polling Council are obliged to release the full tables of their polls and some other companies who are not members will voluntarily provide tables. If a company does not provide any tables and it is not possible to verify how they have conducted the poll, then they would be weighted down by 0.25. In practice this means any poll without tables is given extremely low weighting.
Polls by the same company: different companies have slightly different house effects which mean each company has a tendency to show better or worse scores for each party. In terms of creating polling averages this risks skewing an average if one pollster does a lot more polling than their rivals. For that reason, if a single company has more than one poll in the time period, the second is given a weighting of 0.76, the third 0.52, the fourth 0.27 and so on.
Sample size: Polling averages often weight polls by the size of their samples, giving, for example, a poll with a sample size of 2000 twice as much weight as one with a sample size of 1000. I believe this grossly overstates the importance of sample size and sample error. In fact, sample error is just one possible cause of error amongst many others, it is just the easiest to quantify and pollsters with a typical sample size of 1000 have often outperformed those with a typical sample size of 2000. However, polls with a sample size that is too small should be treated with caution, and for that reason any poll with a sample size significantly (I’d turn a blind eye to 998!) below 1000 will be given a weighting of 0.50.