What are TfL hiding?

Last week Mike Smithson broke the story of a BPC investigation into an Ipsos MORI poll carried out for Ken Livingstone. In the Evening Standard today Andrew Gilligan picks up Mike’s story and has got some comment from John Curtice and Ben Page of MORI.

The story begins back in December with this press release from the Mayor’s office, claiming to show that a poll for Transport for London showed two thirds of respondents were in favour of the new emmissions based congestion charge.

There is no info on when the poll was conducted or what the questions were, but normally pollsters are obliged to release this along with the full tables for polls that are released to the media, either by publishing the tables on their websites, or upon request. MORI however have refused to release the tables for this survey. In the Standard Ben Page explained that they asked TfL for permission, but it had been refused. In response the British Polling Council – the association of polling companies who have agreed to abide by its disclosure and openness rules – has set up in an investigative sub-committee consisting of Peter Kellner, Peter Riddell and John Curtice to look into the case.

So what are TfL hiding? And what are the rules?

At the simplest level, it could be as simple as some of the questions showing a lack of support for Ken’s policies, or even TfL just not wanting to release the data, as is their right. There doesn’t have to be anything dodgy at all here.

Mike Smithson says his understanding is that there were some additional political questions in the poll that shouldn’t have been asked in a taxpayer funded poll. Gilligan cites some anonymous sources agreeing with this, but it is flat out denied by the Mayor’s office – who would say that, wouldn’t they? – and more convincingly by Ben Page who states “There is nothing in those surveys about the elections, or Boris. We have not been paid by the Mayor to ask voting intention questions”. (UPDATE – Mike Smithson makes an important correction to the quote from him in the comments here)

On top of that, I don’t think the BPC disclosure rules would compel MORI to release those questions anyway. The disclosure rules state that if a privately commissioned poll is made public they have to disclose the tables, question wording and so on for the published questions but they can keep private other unpublished findings as long as they don’t contradict or affect the published answers. So in theory a client could easily ask lots of transport questions and lots of political questions, release the first lot and be entirely within their rights to keep the second lot secret.

The other possiblity cited in the Standard is that someone was playing fast and loose with either the interpretation of the results, or with the wording of the original question. This at least would make sense, the wording of the questions hasn’t been announced and if the tables and wording were revealed in full any slight-of-hand would become apparent. The Mayor of London has some previous here – check out reports of this April 2005 poll for the Mayor of London’s free newspaper the Londoner – this time by ICM, which apparently showed 79% support for the Olympics. The questions itself is fine, but if you look at the full survey, you’ll see if was asked straight after a truly shameless question asking if people agreed with Nelson Mandela that London was the best place to hold the Olympics.

The rules of the BPC are intended to promote openness. While membership is open only to companies conducting proper representative polls, there’s no policing of what the right methods of sampling, weighting and so on are. It’s primary purpose is to ensure full disclosure of tables and methodologies so that people can see any sharp practice for themselves – if you see figures from ICM, Populus, YouGov, ComRes or other members reported in the newspapers you can find out whether the questions are biased or the question order dodgy by looking at the raw questions and tables yourself. It acts as a strong incentive for the member companies not to allow clients to get away with such practices because they know they know it would be clear for everyone to see and it would be their reputation that was damaged. It lets us trust the polls because, even if we ourselves don’t examine the tables to look for sharp practice, the fact they are available means that someone else somewhere will, and would pick up any misbehaviour. It is a wholly good and admirable endeavour.

Ipsos MORI’s Ben Page explains their position on this poll in the Standard: “Our position is that ongoing satisfaction surveys and policy surveys of the kind we do for TfL do not constitute a poll and are not covered by the BPC rules. We believe they are covered by the rules of the Market Research Society, which do not require us to publish immediate data breakdowns”. The rules of the BPC refer to “All data and research findings made on the basis of surveys conducted by member organisations that enter the public domain” and that this also applies to any privately commissioned polls that enter the public arena. Obviously it will be down to Peter, Peter and John to look into the details of the case, Ipsos MORI’s full explanation and make a decision on how the actual rules apply in this case. Personally though, whether or not this does fall within the current rules, my opinion is that if there is just one instance when it is overwhelmingly desirable that the rules should apply, it is when polls are commissioned and used by a political figure to support his position in the policy debate.

Once upon a time Bob Worcester told Robert Maxwell where to stick it when the Mirror’s owner wanted to decide whether or not the results of the polls he commissioned were ever released…


9 Responses to “What are TfL hiding?”

  1. Anthony,

    The comment that you refer to from me that it was “my understanding” that there were some additional political questions in the poll was going further than I knew to be the case and I emailed Andrew Gilligan at just after 7pm on Sunday evening to make that clear. Unfortunately Gilligan did not read my message before he wrote his story for last night’s Evening Standard and this quote from me appeared.

  2. Ta Mike

  3. If Gilligan says he didn’t read that e-mail, then we must suppose he didn’t. However, if he had read it, I wonder whether he would have changed/diluted his story? It wouldn’t have been the first time if he hadn’t. It’s a shame Dr David Kelly didn’t have the chance to clarify his statements to Gilligan before they were turned/twisted into a “scoop”.

    This of course distracts from the issue : what is a political figure doing trying to suppress poll results? One can understand someone not wanting to arm the enemy, but this exposure does far more harm than could be done by releasing the tables.

  4. TFL is using public money to commission data to find out what the public thinks in order to justify policy decisions it takes on the public’s behalf. In that sense its data is our data and we have every right to see it and determine whether it has been validly collected and honestly used. It’s very different from a commercial company keeping data secret to protect decisions made for commercial reasons. While this might not be a technical breach of the current BPC rules (or indeed the Market Research Society’s rules) perhaps those rules should distinguish between public and commercial polling. Because polling by public bodies when published can only ever be for (small “p”) political reasons. And in that context it can never be the right of public bodies to mislead or misrepresent the data.

  5. If it is a taxpayer funded poll then the details should be released to the taxpayer.
    What sort of information would it be in the taxpayers interest to hide?

    With all the other media commentary about the Mayor, it would be in the pollsters interests not to be associated with anything that is not seen to be transparent in the usual manner.

  6. If it was a taxpayer funded poll, would not a freedom of information request be granted?

  7. New ICM poll for the Guardian seems to be showing
    con 37 [n/c] Lab 34 [-1] Lib Dems 21 [+1].
    This is compared to the last Guardian ICM poll rather then the last ICM poll.
    We have commented on The Guardian’s variation in the past.
    Not sure waht we make of it.
    Compared with the last ICM for ?? whoever, Labour are up 2.

    http://www.guardian.co.uk/politics/2008/feb/19/polls

  8. Is it possible that these are consultation responses dressed up as opinion polling? Has MORI done the processing and then allowed it to be released in a misleading manner?

    See the online questionnaire and comparison with the statement on the press release.

    Bob Worcester’s successors have, in effect, released their own voodoo poll results. It is therefore misleading to say that ‘The polling was commissioned as part of Transport for London’s consultation on emissions based charging proposals’; although it certainly wouldn’t fall within the BPC’s remit.

  9. GD – nope, the results to the online questionnaire are in this report and neither the questions or the number of responses match.