Monday, January 18, 2010


Do Senate Polls in Blue State's really underestimate Dems

Yesterday, Nate Silver sought to provide statistical hope to the burning reck in Massachusetts, better known as the Martha Coakley campaign for United States Senate. The piece makes the point that the margin between Martha Coakley and Steve Brown could be overstating Brown's lead. Silver points out that polling in close (margins of 10 or less in the polls) Senate elections since 2000 in deeply blue states (as measured by the Cook Political Partisan Voter Index) has by an average of 3.4 points underestimated Democratic candidates' margin of victories. In deeply red states, on the other hand, polling has underestimated Republican margins by 1.9 points.

Though he used a average in 2008 and Real Clear Politics average in 2006 and 2004, Silver believed he could only use a simple average of all non-partisan polls conducted in the final two weeks compiled by in 2002 and 2000. It would seem did not know that a Real Clear Politics average is also available for 2002 and 2000.

I was interested what, if any, effect substituting the Real Clear Politics averages in 2002 and 2000 would have on Silver's results. Therefore, I decided to create a new dataset modeled after Silver's, but using the data in 2008 and Real Clear Politics average* in 2006, 2004, 2002, and 2000.

Using these new rules, we get the following:

The underestimation of Democratic margins in blue states stays the same at 3.4 points. This result is not surprising, as the blue states part of the dataset is small, and only three results are available from 2000-2002. The underestimation of Republican margins in red states drops from 1.9 to 0.9 points. The overall average of underestimation drops from 2.3 to 1.5 points. In only 7 of 23 contests was the underestimation above 3 points. In no contest did the polling average incorrectly predict the winner due to underestimation of Democratic candidates in blue states or Republican candidates in red states. The only incorrect winner chosen was in the South Dakota (a red state) 2002 race when the polls predicted a victory by Republican John Thune.

The bottom line is that perhaps the polls are overestimating Brown's margin in Massachusetts. The limited data involving closely polled elections in blue states suggest that Coakley might do better than the polls suggest, but when you look at the larger dataset of red states, Coakley should not expect a bump.

-*In some cases (such as Alaska 2004), no Real Clear Politics "average" existed. I just averaged all the polls listed on Real Clear Politics (and in the case of Louisiana 2002) conducted in the final two weeks. I, unlike, Silver use internal polls in these cases... modeling myself after's inclusion of them.

EDIT (1/20/10): Mark Blumental of, quoting from an email I sent him, provides some additional thoughts.

Comments: Post a Comment

Subscribe to Post Comments [Atom]

<< Home

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]