As the polls and expert analysis for Senate races in 2012 begin to come in fast and furious, one question we all should be asking is "do the polls and analysis actually mean anything?" To help answer this question, let us first look at polling data from early on in the last Presidential year Senate campaign (2008). As we will see, one should proceed with caution when examining Senate polling data this early in the cycle. In 2008, Democrats gained 8 Senate seats from Republicans. By the end of 2007, only 12 eventual match-ups had been polled (according to Pollster.com). I have taken an average of all the polls for 2007 in these races. I have converted the Democratic and Republican percentages to two-way (i.e. allocated undecided and third party votes to the two major candidates proportional to their initial unaltered percentage). Of the 12 match-ups polled, the average error between the early polls and actual margin was 10.9%, while the median error was 8.3%. That is about double the average pollster error seen in Senate polls in the final 21 days of the campaign from 1998-2008.
More interesting is the direction of the error. In the Democratic year that 2008 became, Democratic candidates over-performed their early projected total by an average of a little over 4.1%.
Of the 12 races polled in 2007, Democrats over-performed in 9 of them. Of the 9 over-performances, Democrats actually won 3 (Minnesota, North Carolina, and Oregon) races they were polled to lose. The early polling correctly projected that they would pick up 4 (Colorado, New Hampshire, New Mexico, and Virginia) other seats. The 8th seat pick-up, Alaska, was not polled this early in the cycle.
What about non-partisan analysis? Non-partisan analysis this early in the cycle in 2006 and 2008 (2010) under-predicted Democratic (Republican) gains.
At this point in 2006, the invaluable Cook Political Report placed 9 previously held Democratic seats in the likely, lean, or toss-up column, while only placing 6 Republican seats in these designations. Democrats would go on to gain 6 seats from Republicans. 5 (Missouri, Ohio, Pennsylvania, Rhode Island and Virginia) of those 6 were in either likely, lean, or toss-up column at this point. Only Montana proved to be a surprise. In 2008, Cook had 5 Democratic seats and 6 Republican seats in three aforementioned columns. Of the 8 seats Democrats eventually gained, only Colorado, Minnesota, New Hampshire, and North Carolina were listed in one of the three possibly competitive categories. 2010, similarly, broke differently than early analysis pegged. Cook put 8 Democratic seats in the three competitive categories, while putting 9 (including later party switcher Arlen Specter and 2010 loser) Republican seats. Republicans would take 6 seats in the 2010 elections. 3 (North Dakota, Illinois, and Wisconsin) of these were listed as competitive at this point in 2010, while 2 (Arkansas and Indiana) were solidly Democratic.
What does this mean for 2012?
First, as the Cook Political Report figures suggest, there are going to be some seats that may not be viewed as competitive at this point that could become competitive and vice-versa. Some of these newly competitive seats may be due to retirement (see Indiana 2010), while others may just be campaigns evolving over time (see Oregon 2008).
Second, very early polling and analysis under-forecasted the waves that would occur in the following year. Intrade (and my own preliminary Senate model) indicate that Republicans are the ones who will benefit from any 2012 wave. Right now, Cook has an amazing 7 Democratic seats in the toss-up column (more so than any of the three prior elections at this point). 13 Democratic seats overall are in the competitive columns, while only 5 Republican seats are. If Republicans do any better than they are currently projected to in either polling or expert opinion, then it could be a long Election Night for Democrats.
That said, the general lack of correlation between early expert analysis and final results and 2008 polling errors should read "caution Will Roger". Nothing in the Senate is set in stone. Still, I would put early money on a good Republican night.