Penn vs Greenberg

No polls this Sunday apparently, so here’s some articles to keep you busy.

British pollsters used to have an amusing habit of fighting like ferrets in a sack. They’ve calmed down of late, but over at www.pollster.com Mark Blumenthal is chronicling a wonderful spat between the two American pollsters that Tony Blair retained for the 2005 election campaign.

In short, Stan Greenberg was Blair’s pollster in the 1997 & 2001 elections, but in 2005 he was retained for some party polling, but largely surplanted at the heart of the campaign by Mark Penn, another US pollster. Greenberg has recently published a set of memoirs about his time working for Blair, Clinton and other world leaders, in which he slates Penn, describing his methods as “errant”, his message testing “biased” and “rigged” and “lacking transparency”. Penn, naturally, disputes all of this, and says he only appeared less than transparent to Greenberg, because Greenberg was out of the loop so wasn’t given access to the full data Blair, Philip Gould and so on received.

Greenberg then offers further detail here, claiming Penn didn’t use any political weighting right up until a month before the election, over-analysed data with a very small sample size and carried out message tests that Greenberg thinks were biased. Mark Penn replies again here essentially saying it’s all sour grapes from Greenberg because he got replaced. Without seeing Penn’s work, it’s obviously rather hard to know who to believe, but it is fun watching the squabble :)


No UK polls today (I believe Populus’s monthly poll is going to be next month) the focus is naturally on the USA. We know what the polls say – Obama will win, and probably win big. As ever, do be wary about the inevitable leaks of exit poll data. Otherwise, feel free to use this thread to ramble on about the US election as you wish :)

UPDATE: There’s a memo from Bill McInturff, McCain’s pollster, here explaining why the exit polls might well overstate Obama’s lead. While McInturff obviously has a partisan interest – the McCain campaign don’t want late voting McCain supporters in Western states giving it up as a bad job once the exit polls from Eastern states start showing towering Obama leads – he is probably right.

UPDATE 2: And yay, with another hat tip to Danny Finkelstein’s little election gossip box here, here is the Lord and Master of all Voodoo polls. Bow down and worship before the AOL straw poll which shows a stomping great McCain landslide with 493 electoral votes (you won’t be able to vote unless you are an AOL user, because like, then it wouldn’t be scientific would it? But try to and you get the chance to skip to the results. And to laugh at them). Bless.

UPDATE 3: Here’s a BBC summary of most of the state-by-state projections, though it inexplicably ignores the excellent FiveThirtyEight and Electoral-Vote.

CQ Politics: Obama 311/McCain 157 (too close to call 70)
Real Clear Politics: Obama 291/McCain 132 (tctc 105)
Pollster.com: Obama 311/McCain 142 (tctc 85)
New York Times: Obama 291/McCain 163 (tctc 84)
Crystal Ball: Obama 364/McCain 174
FiveThirtyEight: Obama 346.5/McCain 191.5
Electoral Vote: Obama 353/McCain 174 (tctc 11)

Nate Silver of 538 is offering his own warning about exit polls here. In case it hasn’t sunk in yet, be wary of them, especially any leaked ones which won’t be weighted properly yet.

UPDATE 4: Since the BBC looks set to scale new heights of dumbed-down coverage of the election tonight (they’ve got Ricky Gervais on it, you know?) I’ll be flicking between the American networks. For impatient anoraks like me waiting until ballots close and we get proper data to geek over, there’s a nice video clip from CNN here with their director of polling explaining how they actually decide when to call states.

UPDATE 5: Rival exit poll leaks! Gawker claims leaked exit polls show Obama up by 4 in Pennsylvania, which if leaked unweighted exit poll figures were any use (which they aren’t) would be lower than we’d expect. Drudge meanwhile claims that leaked exit polls show Obama up by 15 in Pennsylvania, so in short, we are entirely in the dark :)

UPDATE 6: Woo! The first results are coming in from the polling stations in Kentucky and Indiana that closed at 6pm EST. We can’t tell much – less than 1% are counted and they show McCain easily ahead in Kentucky (as would be expected) and Obama narrowly ahead in Indiana (again, in line with polls).

UPDATE 7: More leaked exit polls, from Huffington Post this time. Obama is ahead everywhere expected, so no surprises there (obvious caveats apply that these don’t mean a huge amount anyway, they may well be the early unweighted ones and US exit polls are no great shakes anyway). Pennsylvania incidentally is alledgedly showing a 15 point lead, so inline with those Drudge numbers.

UPDATE 8: CNN have called Kentucky for McCain and Vermont for Obama. Looking at their exit poll figures on their website (they don’t put up topline figures, but you can calculate them from the male/female split) before they start fiddling about and reweighting them, they are

Vermont has a 31 point Obama lead.
Virginia has a 9 point Obama lead.
Indiana has a 6 point Obama lead.
Georgia has a 2 point McCain lead.
South Carolina has a 6 point McCain lead.
Kentucky has a 14 point McCain lead.

UPDATE 9: The CNN exit polls from the 7.30pm closers are:
Ohio has a 8 point Obama lead
North Carolina has a 3 point Obama lead
West Virginia has a 10 point McCain lead
Fox has called West Virginia for McCain, but no one else seems to have yet

UPDATE 10: The next batch of exit polls:
DC has a 85 point Obama lead
Maryland has a 36 point Obama lead
Delaware has a 32 point Obama lead
Connecticut has a 29 point Obama lead
Illinois has a 27 point Obama lead
Maine has a 22 point Obama lead
Massachusetts has a 21 point Obama lead
New Hampshire has a 20 point Obama lead
New Jersey has a 19 point Obama lead
Pennsylvania has a 15 point Obama lead
Missouri has a 8 point Obama lead
Florida has a 2 point Obama lead
Mississippi has a 1 point McCain lead
Tennessee has a 13 point McCain lead
Alabama has a 15 point McCain lead
Oklahoma has a 22 point McCain lead
The networks have called a lot of those straight off. Mississippi is the obvious surprise amongst them.

UPDATE 11: Just the one exit poll at 8.30pm. Arkansas has a 9 point McCain lead.

UPDATE 12: Last lot for me, some surprising ones here compared to pre-election polls.
New York has a 35 point Obama lead
Rhode Island has a 33 point Obama lead
Minnisota has a 19 point Obama lead
Michigan has a 17 point Obama lead
Wisconsin has a 17 point Obama lead
New Mexico has a 13 point Obama lead
Colorado has a 8 point Obama lead
South Dakota has a 3 point Obama lead

Arizona has a 2 point McCain lead
Texas has a 2 point McCain lead
Louisiana has a 6 point McCain lead
Nebraska has a 11 point McCain lead
Kansas has a 13 point McCain lead
North Dakota has a 15 point McCain lead
Wyoming has a 21 point McCain lead


-->

I haven’t followed the polls in the US presidential race in any detail – there are dozens of polls every day and I could never hope to do half as good as job as Mark Blumenthal over at Pollster.com, with just a couple of days to go however, I thought it worth having a look to se whether it really is all over. With the polls consistently showing Obama ahead, can McCain still win? Let’s have a look at the possibilities.

1) The late swing. The most famous surprise result of a Presidental election was Harry Truman who was able to pose the next day with newspapers that had gone to print assuming his inevitable defeat and hailing his opponant’s victory. What happened then was that Gallup finished polling too early and, in the gap between the end of opinion polling and the actual election opinion swung to Truman. We will have polls up until the day itself so unlike in 1948 we would know. Could it happen?

It is obviously a logical possibility, however strong an Obama lead the polls show, it is only public opinion now, it could change by Tuesday. Since McCain’s convention bounce receeded Obama’s lead has been pretty solid and there is no obvious sign of it. The last slice of data in Zogby’s rolling tracker showed a shift to McCain…but there is no obvious pattern in other national match-up polls.

Besides, even it if did happen, it might be too late for McCain. Projection sites like Electoral Vote and Five Thirty Eight have Obama with a solid lead in states collectively worth almost enough electoral votes to win, in the next tranche of states, those leaning towards Obama, the ones McCain would have to get hold of to prevent Obama winning, five have early voting. In Florida a third of the total number of votes cast in 2004 have already been cast, in North Carolina, Colorado and Nevada over half of the total number of votes cast in 2008 have already been cast. If there was some event that suddenly swung public opinion to McCain in these key states, unless it was a major swing it would be too late, people have already voted for Obama.

2) So if it is unlikely the polls will change in time, what if they are wrong? There are lots of ways they could be. First up is the Bradley effect. This is commonly named after Tom Bradley, but refers to the apparant way that polling in a contest between a black and a white candidate in the USA used to underestimate support for the white candidate, presumably a social desirability bias in the answers because people are worried they may been seen as racist if they declare they are voting against the black candidate. There is some debate about whether this happened at all in US polls in the first place, but there is plenty of evidence of polls in recent years which accurately predicted elections between a black and white candidate. If there was a Bradley effect, it seems to have gone. The only way it could suddenly come back, I suppose, is if there are stronger social pressures in a Presidential race, but I can’t see it.

3) Shy McCain vote. Leaving aside the race question, could the undecideds split in favour of McCain. Indeed might there be a shy McCain vote in the way polls underestimated the Conservatives in this country in the 1990s? The answer is probably not – looking back to 1992 here, ICM found in the aftermath that the don’t knows were disproportionately people who had voted Conservative in 1987. In the USA Mark Blumenthal and Charles Franklin have got hold of a huge chunk of data on undecided voters from Financial Dynamics to analyse. When the don’t knows are pushed for which way they are leaning, they break for Obama. When Franklin analysed the demographics of those who refused to even say how they were leaning, looking at their race, party id and so on, his prediction is they are a pretty even break, in fact, if anything they’ll lean slightly to Obama.

4) Faulty Turnout Model. In the UK pollsters tend to base turnout projections on how likely people say they are to vote. In the USA turnout projections are more complex and factor in whether people have voted in the past (and sometimes various other factors as well). Early voting and current polls suggest a higher level of voting than normal in US elections. Could this help McCain? Probably not, as if pollsters are underestimating turnout it is likely to be black and/or poor voters coming out for Obama who are being understated. It may not make any difference at all: Gallup are producing parallel numbers, ones based on their traditional US turnout model, and ones based on a turnout model we are used to in this country based only on how likely people say they are to vote. In the latest figures Obama has an identical 10 point lead on both measures.

5) Weighting. Tracking how pollsters in the US are dealing with weighting is a bigger task than I can do here. Pollsters use a wide variety of weights by party ID, or in many cases none at all. Whereas in the UK demographic weightings are much the same between pollsters, in the US there is some variety because some target models of likely or registered voters, rather than the demographic make up of the whole US. We do see pollsters with different proportions of African-Americans in their samples for example. The sheer variation in the way US pollsters do things is why I suspect this is not disguising a McCain lead: with such variation, it is unlikely everybody’s method has wrong. Equally, having a black candidate is new and probably having increased turnout is new, so it is reasonable to ask whether polls can cope with these developments. There has not, however, been a sudden massive change in US demographics.

Those are the most likely reasons why McCain could still win, and I don’t think any of them work. If anything, I’m expecting a bigger Obama victory than the polls indicate. Of course, when polls do get it wrong it is normally something that we haven’t predicted and it’s perfectly possible that something completely different will come out of the woodwork. If miraculously the polls are wrong and McCain wins on Tuesday there would be one hell of a post-mortem about why the polls got it wrong, but really, I’m not expecting it.


As well as the voting intention poll for the Indy, ComRes have also published a poll they have conducted for Theos. As with every other UK poll on the US election, it shows British people would overwhelmingly back Barack Obama rather than John McCain were they to have a vote in the US election. 66% would back Obama, 10% McCain.

More interestingly though ComRes also asked a series of questions asking whether people would be prepared to vote for a leader who was black, muslim, gay or from another minority group. The question drew its inspiration from a similar poll conducted in the USA by Gallup last year, which found amongst other things that 5% of American voters said they wouldn’t vote for a black candidate and a majority (53%) wouldn’t vote for an atheist.

ComRes’s poll in the UK found that 5% of British voters said they would not vote for a black leader. For all the concern that American voters are somehow more racist and more likely not to vote for a black candidate, the proportion of people ready to admit that they wouldn’t vote for a black candidate is the same in this country (though naturally, we cannot tell how many other people share those views but were unwilling to admit them to a phone interviewer).

This shouldn’t be a huge surprise as analysis of electoral data shows a racial effect in how people vote. Roger Mortimore of MORI crunched the figures for the 2001 election and found Labour did 2.5% worse than average in seats where a ethnic minority candidate had replaced a white one, and 6.1% better in seats where a white candidate had replaced one from an ethnic minority.

I can’t track down a proper study, but most people with experience of local government elections will be able to reel off anecdotal examples of where there would appear to have been a racial bias in people voting in multi-member wards (see, for example, the two split wards in Bexley in 2006 – Belvedere and Erith here).

In the UK the factor that drew the most opposition was age. 43% of people said they would not vote for an otherwise acceptable candidate for leader who was 72 years of age, almost the same as in the USA where 42% said they would not vote for a 72 year old President. It is potentially possible, of course, that this is partially a reflection that people are more willing to admit discriminating in terms of age than on sexuality or religion. After that came being either gay or lesbian, or being a Muslim – in both cases 23% of people (presumably not the same ones!) said they would not vote for an otherwise qualified candidate in those circumstances. The Gallup survey did not ask about whether people would vote for a Muslim President, but did ask about a homosexual candidate and found 43% of Americans would refuse to vote for them.

Most other groups met with comparatively little opposition. Only 7% of British voters would not vote for a divorcee. (In the US survey, which clearly had Rudy Guiliani in mind, 30% said they would not vote for a thrice-married Presidential candidate). 7% of voters said they would not vote for a female leader – interestingly this was evenly split between men and women. 7% of people said they would not vote for a Christian leader – a question that would perhaps have been more interesting if ComRes has asked about committed or evangelical Christianity to see if there was any truth in Alistair Campbell’s famous “we don’t do God”.

In the US survey, the most electoral objectionable group was atheists, with 53% of Americans saying they would not vote for an otherwise well-qualified Presidential candidate who was an atheist. One would expect that figure to be much lower in the UK, but actually it is still surprisingly high at 20%. One might not have guessed it, but not believing in God would appear to be almost as much of an electoral handicap for a potential leader in the UK as being Muslim or gay.


Today’s Guardian reports an ICM poll from last week that showed that British voters would overwhelmingly prefer to see Barack Obama as the next President of the USA, by 55% to 11% for John McCain.

I’ve seen a couple of polls ask about this (it’s also tracked on the Phi5000 figures on PoliticsHome) and the pattern is pretty consistent. To some extent is will be a result of hostility towards the Republican President Bush, but British people do tend to be considerably less right right than Americans, so it should be no surprise that a majority back the Democrat candidate.

Back in March YouGov did a big parallel study of opinion of people in the UK and the USA, which showed the differences and similarities between public opinion in the two countries. Some of the similarities were actually more surprising than the differences. Despite the USA being a nation of immigrants, there was no particular contrast in attitudes towards immigration – Americans don’t seem to be anymore welcoming to other countries’ tired, poor or huddled masses than British people are: 26% of Brits thought immigration had helped the economy, 25% of Americans did; 70% of Brits thought immigrants had taken jobs that should be being done by British people, 64% of Americans thought similar.

On the environment, the image of the USA as a nation of climate change deniers seems largely unfounded. American respondents were more likely to think that there was no global warming at all (18% compared to 7% of Brits), but British people were more likely to think it was nothing to do with mankind (25% to 19%) so the proportions of people believing in manmade global warming were not vastly different (55% in the UK, 49% in the USA). Asked about environmental policies attitudes toward subsidies for environmentally friendly energy, nuclear power or (amazingly, given the American love of big cars) increased petrol taxes were almost identical – only on airline taxes were the US far more hostile.

The biggest surprise similarity though was the death penalty, which proved only marginally more popular in the USA than the UK. 74% of British people supported the death penalty for some (53%) or all (21%) murders. 76% of Americans supported the death penalty for some (50%) or all (26%) murders.

If those are the similarities, where are we different? Attitudes towards withdrawing troops from Iraq were much the same: 35% of British people wanted troops out this year, 36% of Americans wanted troops out this year. This similarity though masks far more hawkish foriegn policies attitudes in the USA in general. In Afghanistan 42% of Brits wanted withdrawal this year, but only 28% of Americans did. On Iran, 26% of British people would countenance military action to prevent Iran gaining nuclear weapons, 46% of Americans would.

Amongst the public the special relationship between the UK and USA seems rather unrequited. British respondents are equivocal about it – 46% would like it to be fairly close or very close, 46% would like it to be not very close or not close at all. For American respondents 78% would like it to be close, only 9% not very close or not close at all.

There was not a huge contrast in attitudes towards the level of tax and spending (though the US was more polarised) but when asked who any tax cuts should benefit there was against a contrast – US respondents wanted to see then go to the middle classes rather than the poor (35% to 20%), UK respondents would rather they were concentrated on the poor (38% to 21%). On welfare too Americans tended to be more right wing. If people are made redundant British voters think it is the responsiblity of the government (38%) or the company (35%) to look after them, in the US, people tend to think it is the worker’s own responsibility (36%), with only 17% thinking the government should. Another contrast was free trade vs protectionism – British respondents favoured free trade by 52% to 30%, US respondents tended to think free trade was a bad thing, by 55% to 31%.

Some things were less predictable – people in the US seem to be *less* comfortable with big business and successful businessmen, when one might expect it to be the other way round. On the other hand, they were more likely to see the profit motive as a good thing – suggesting is particular businesses and the influence they wield on politics that they have problems with, not capitalism per se.

All the differences above, however, pale into insignificance compared to the biggest difference – attitudes towards religion and moral issues. Only 39% of British respondents believed in God, compared to 80% of American respondents. On top of that, those who are religious are MORE religious – only 21% of that 39% of Brits who believe in God said it was very important to their life, 53% of the 80% of believers in America did.

A paltry 5% of British people think sex outside marriage is a sin, 33% of Americans think so. 54% of Americans believe in hell, only 16% of British people do. 63% of British people accept the theory of evolution, 23% believe in creationism or intelligent design. Only 30% of Americans believe in evolution, 59% believe in creationism or intelligent design. 40% of Americans think homosexuality is a sin, only 26% think it is perfectly acceptable. In the UK the figures are 13% and 46% respectively.

On more political grounds, 76% of British people think abortion should be legal in all circumstances, or with only limited restrictions like a time limit. Only 48% of Americans agree, with 48% thinking it should be totally banned, or banned apart from extreme circumstances such as the life of the mother being in danger.

We shouldn’t be surprised the more left wing of the two candidates in the USA is preferred by British people, since in terms of things like welfare, foreign policy, taxation and, most of all, moral and religious issues, America is far to the right of the UK.