Margin of Error

New site

2012-01-02T21:29:00.000-08:00

I'm now blogging at http://marginoferror.org/

PPP, Selzer, Cell phones, and Iowa Polls

2011-12-28T00:41:00.000-08:00

With the Iowa Republican caucus only 1 week away, pollsters will poll the state like nobody's business. Some polls will have some candidates higher than others, and some polls will ultimately be more accurate than others. So how do we know which polls we should trust?

Most long-time Iowa watchers will tell you to watch the well known Selzer and Co. Des Moines Register poll. The poll's most recent triumphs include correctly nailing an easy victory for both Barack Obama and Mike Huckabee in 2008. In fact, according to some Washington insiders, it's the only poll to watch.

I disagree. Selzer's polling deserves its praise, but nobody's perfect.

Just look back to the 2010 Iowa Republican gubernatorial primary. Midterm primaries are obviously different than presidential caucuses (though remember Iowa Republicans use a straw-poll procedure that is much closer to a primary than what most think of a caucus). Still, this was a very recent matchup involving mostly Republican voters in Iowa.

In this contest, Selzer's final poll had Terry Branstad winning by 28%, while automated and frequent Iowa pollster Public Policy Polling (PPP) had Branstad winning by 15%. Branstand's final margin was only 9.4%.

PPP was clearly the more accurate pollster in predicting the final margin.

Now, this isn't to say that PPP will end being more accurate in the 2012 Iowa Republican caucus. I am merely saying that no pollster is the one and only.

While there is such a thing as really bad pollsters (Zogby Interactive, I'm looking at you), we really cannot rank most pollsters from best to worst. Mark Blumenthal first demonstrated this point in 2010, Dick Bennett has continuously showed it, and my own work looking back at the 2010 midterm elections confirms Blumenthal and Bennett's finding.

What about the argument that PPP and other automated pollsters may be missing cell-phone only voters? There is some evidence that there is a difference in the political opinions of cell-phone only users vs. landline users, and Iowa has a relatively high percentage of adults only reachable by cell-phone.

The real question we should be asking though is how will this affect polls for the 2012 Republican Iowa Caucus.

In 2010, automated polls were on average as accurate as live interview polls. In 2011, we saw automated pollsters accurately predict difficult to poll House special elections in both New York's 9th and 26th districts. And let's not forget, PPP did better than live-interviewer Sezler and Co. in the 2010 Iowa Republican primary.

The evidence from recent history seems rather thin that automated polls will miss something that live-interviewers will capture just because automated polls are automated.

But will 2012 be different?

Earlier this month, Marist polled Iowa and found the following:

You can look at this data in a few ways. First, Ron Paul did better in this polling sample thanks to cell-phones being included*. Second, cell-phones did not greatly impact the poll's findings. In fact, the variation between candidates' percentages in landline only (as automated phones have) and overall voting population was statistically insignificant.

But let's take a look at some more data.

During the 11/27-12/6 window in which the Marist poll was taken, a lot of pollsters were in the field. The race seemed pretty static during this period with Newt Gingrich with a solid lead. We had a good mix of both live-interviewers who called cell-phones and automated pollsters who did not. Did a similar Ron Paul effect happen within this group as in the Marist poll?

Somewhat to my surprise, it did! On average live-interviewers who called cell-phones found Ron Paul's support to be at 17.2%, while automated polls found it to be at only 14%. This is the same exact 3% gap Marist found.

Here's the kicker stat though. PPP was the only automated pollster during this period that did not exhibit the Ron Paul cell-phone gap. At 18%, Ron Paul's PPP percentage of the vote was tied for the highest any pollster recorded Ron Paul at during this period.

In recent weeks, PPP has continued to show Ron Paul flying high. Their polls have shown Paul at a greater percentage than most other pollsters (e.g. automated pollsters Rasmussen and We Ask America).

I think PPP may have something. The lack of cell-phone voters seems to be affecting most automated samples, but PPP somehow seems immune.

When we begin to see more live-interview polls goes into the field later this week, it's probable most of them will find the same Ron Paul lead in Iowa that PPP has. The question of whether that lead holds and which pollster is ultimately the most accurate is still one to be answered, but I think we will have a pretty good idea of the final result going into the caucus.

*Something I know comes to no surprise to his supporters who CONSTANTLY argue this point.

******CORRECTION: CNN's state polls are landline, but do not include cell-phones. Only their national polls include cell-phone only respondents. The recalculated average for the third chart is 17.3% for Paul support among landline with cell-phone polls. In other words, the conclusion is the same.

Expecting an Iowa Polling Shock? Don't Be Prepared to Drop Your Jaw

2011-12-27T17:32:00.000-08:00

It has become popular to say that the Iowa caucus is where pollsters' dreams go to die. Experts from Mark Blumenthal to Nate Silver have noted that Iowa caucus polling accuracy is less than ideal (and certainly less than in other contests)... But I dare say that our inability to predict the caucuses is somewhat overblown?

First, let's remember that the Republican Iowa caucus is not actually a caucus. It's a straw poll. The difference? In a caucus (as the Democrats have in Iowa), voters have to gather for hours on end openly declaring their support for a candidate.

In a straw poll, voters merely have to listen to a few speeches and then cast their votes on secret ballots. This makes the act of voting far easier and invites less peer pressure to vote for a specific candidate (i.e. having a voter change their mind to fit what their neighbors are doing).

Another distinction is that in the Democratic caucus voting thresholds exist. Candidates who do not meet this threshold (15%) have their voters reallocate their votes to other candidates. On the Republican side, voters vote once and that's it. No thresholds.

Both of these facts make polling a Republican contest easier in theory than a Democratic contest.

What about the rather convincing charts indicating relatively high pollster error for past Republican contests in Iowa?

The truth is that we are dealing with a lot of old data. Note that with the exception of 2008, all the Iowa Republican caucus polling data is from 2000 and before. Why is that important?

It turns that polling data has gotten incredibly more accurate since 2000. For instance, you may remember that the final polls in 2000 incorrectly indicated that George W. Bush would win a small but solid victory in the popular vote over Al Gore. What you may not remember is that despite being relatively inaccurate, polling in 2000 was better than in all but two presidential elections since 1960.

Polling in 2004 and 2008, on the other hand, came with decimal points of correctly projecting the presidential popular vote. Polling for state contests has similarly become consistently more accurate in the 2000's.

Despite all the problems with new untested pollsters with supposedly unreliable techniques, polling in 2010 was on-par with other midterm elections since 2002. In 2011, we had automated pollsters (who are missing the cellphone population and have terrible response rates) correctly forecasting upsets in special Congressional elections (see New York 9 and 26).

Fitting with this pattern, the 2008 Republican Iowa caucus was more accurately polled than any before it. The average difference between the final polling average and candidates' actual percentage was only 2.7%. The biggest difference in percentage, 5%, was far less than the biggest difference in previous years.

If the trends of accuracy continue, polling in 2012 in both the Iowa caucus and other contests is more likely to be like 2008 than any other year.

None of this is to say that we will not see a "surprise" in Iowa. Considering how tightly the candidates are clustered, a 5% error between the polls and actual margin could make all the difference in the final outcome and, more importantly, media spin. That said, the polls are more likely to get a lot more right than wrong.

Romney helped by a split field? Not nationally.

2011-11-29T16:05:00.000-08:00

Almost all pundits believe that the Republican nomination will come down to Mitt Romney and a conservative alternative. Once the field winnows after the Iowa Caucus will conservatives coalesce around this "alternative"? Is Romney's 25% ceiling in national primary polls a sign that he will never garner a majority of Republican primary voters?

To answer this question, a few national pollsters have tested Romney against a sole conservative alternative.

They do this by first testing all the viable Republican candidates (Bachmann, Cain, Gingrich, Huntsman, Paul, Perry, Romney, and Santorum), and then following up this full-slate question by polling Romney vs. one other candidate (Cain, Gingrich, or Romney).

If non-Romney voters are as a group solely looking for a Romney alternative, we would expect Romney's support relative to a given conservative alternative to drop dramatically from the full matchup to the followup one-on-one question.

Mathematically we can test this through the following equation:

Romney split field advantage = Romney support / (Romney support + conservative alternative support) * 100 in the one-on-one test - Romney support / (Romney support + conservative alternative support) * 100 in the full trial heat.

This equation gives us Romney's percentage of the vote vs. the conservative out of 100% with undecideds allocated.

A negative value means that voters who do not vote for Romney or the conservative alternative in the full candidate lineup are mostly part of the "anyone but Romney" crowd. A positive means that Romney would not necessarily be hurt once the field comes down to one other candidate and him.

As the table below shows, Romney support relative to the conservative alternative has remained about the same in the full and one-on-one matchups.

In the 8 times the Romney one-on-one matchup has been polled, Romney has done on median only 2% worse in the one-on-one than in the full trial heat. The biggest difference was an early November NBC poll that had Romney's support dropping from 74% to 63% in a one-on-one against the now weak Rick Perry.

While this number may be close to being statistically significant, it is not substantially significant. If non-Romney voters in the full trial heat were absolutely against him, we would expect him to do far worse than 2% in the one-on-one.

Does this mean that Romney is going to win in a one-on-one against one conservative? No.

For one thing, we are talking about support relative to the full trial heat. When Romney trails the conservative in the full matchup, he trails them in the one-on-one. In the most recent Quinnipiac poll matchup, he held 46% of the vote relative to Gingrich in the question including all the candidates and 44% in the one-on-one.

It is only when Romney leads in a poll of all the candidates should we expect him to lead in a one-on-one poll. An example of this phenomenon is an early November NBC poll when Romney held 51% of the vote in both the full trial heat and one-on-one against Herman Cain.

I should also point out that any one candidate (e.g. Herman Cain) might have his/her support go overwhelmingly to one candidate. A Selzer and Co. Bloomberg Iowa poll shows Herman Cain's voters going to Newt Gingrich and Rick Perry, not Mitt Romney*.

Still, the evidence from the national polls suggests that Romney is not greatly benefitting greatly from a split field. If he is leading when there are eight candidates in the race, he will still be leading when there are only two.

*Note that Selzer and Co's New Hampshire poll has Cain supporters splitting far more evenly between Gingrich and Romney. Also a national Pew poll found Cain supporters splitting exactly evenly between Gingrich and Romney.

Republicans Winning The Senate... That Is Where The ---> Points

2011-11-19T23:47:00.000-08:00

It has been a few months since I checked in on the 2012 Senate scene, and, unfortunately for Democrats, very little has changed.

Democrats are playing defense in every region of the country with 23 Democratic seats up for reelection and are on offense in only a few states with only 10 Republican seats up for reelection.

To win back the Senate outright, Republicans need to have a net gain of 4 seats from Democrats. Polling indicates that they are well on their way.

I count 8* currently held Democratic seats where in a conceivable (i.e. both candidates in the match-up have declared they are running and have a legitimate shot of winning their party's nomination) general election match-ups the Republican candidate has led in at least one poll.

This includes Connecticut, Florida, Hawaii, Missouri, Montana, Nebraska, Virginia, and Wisconsin.

How good is eight for Republicans? By this point in 2010**, Republicans only led in 6 states where a "conceivable" matchup was polled. Remember that Republicans gained six seats in 2010, which if duplicated would give the Republicans a majority with room to spare in 2012***.

Democrats have led in at least one poll in only Massachusetts and Nevada. This number matches 2010 at this point when Democrats led in at least one poll in a "conceivable" matchup in Missouri and Ohio. Democrats would lose both races by double-digits.

What about expert ratings from the Cook Political Report? Experts, at least at this early point, might have information on the races that goes beyond polling data.

Cook list seats from the very competitive tossup category, to the somewhat less competitive lean category, to the possibly competitive likely category, and the least competitive solid category.

2012 Republicans are competitive in more seats at this point than in any year since 2004****.

In fact, next to the 15 Democratic held seats up not solidly held, the next highest total seats in play is 10 Democratic seats in 2004. There are also more Democratic seats in the tossup category than either party has had in any prior year, and there is already one Democratic seat leaning towards Republican control.

But how accurate are these early predictions? As the table below illustrates, they tend to be pretty accurate. Solid seats are won by the favored party 96% of the time, likely seats 79%, lean seats 80%, and tossup seats 50%.

Thus, we might expect Democrat Seats Won in 2012 = Solid D * 0.9610 + Likely D * 0.7895 + Lean D * 0.8 + Tossup D * 0.5 + Tossup R * 0.5 + Lean R * 0.2+ Likely R * 0.2005 + Solid R * 0.0390.

If these distributions held [as Nate Silver applied (but found was inaccurate) in his earlier analysis] for 2012, Republicans would be expected to pickup 4.1 seats. The issue is that in none of the past four elections did a normal distribution hold.

That is if we applied these percentages to the prior elections, the party that had more seats at risk (in likely, lean, or tossup) went onto lose more seats than a normal distribution would forecast.

In 2004, Democrats lost 4, not 1.8 seats; in 2006, Republicans lost 6, not 2.5 seats; in 2008***** Republicans 8, not 4 seats; and in 2010******, Democrats lost 6 not 0.9 seats.

If these trends hold for 2012, Democrats would not lose 4 seats, but somewhere between 6 and possibly 9.

With the 8 Democratic tossup seats or leaning Republican plus the currently leaning Democratic Florida where Connie Mack actually leads in one recent poll and close match-ups in Michigan and Connecticut (if Chris Shays wins the Republican nomination), 9 Republican pickups is not mathematically crazy.

I mean I am certainly not calling for 9 Democratic seats to fall into Republican hands, but the mere fact that it is possible says something. More than that, it fits in with the polling data and my own historic economic aggregate Senate model. They are all signaling for large Republican gains this far out from the election.

But the fact that we are a year out should lead us to caution. The datasets with which we are dealing contain a small amount of observations. Some yet unknown Senate incumbents may retire, and there may be some unforeseen primary match-ups. Indeed, it is always possible that one election cycle will break the mold (think Tim Tebow's recent performances).

The problem for Democrats is that, at least at this point, 2012 does not seem to be one of those years.

*Both Indiana and North Dakota saw Democratic Senators retire, and both were won by the Republicans by large margins.

**The currently Democratic held North Dakota Senate seat has yet to be polled, but it likely that Republican Rick Berg is in the lead.

***See my prior piece for 2008 polling. Though the piece refers to polls taken earlier in the 2008 cycle, the number of Republican seats that were predicted to be won by Democrats, 3, held through this point in the 2008 cycle. Democrats actually won 8.

****Earliest year in which the Cook Political Report lists their Senate predictions on their website.

*****Note that the Mississippi Special Election is not included in the link or in my standings because it was not yet scheduled.

******Note that the Massachusetts Special Election is included in the link, but not in my standings because it was held in January 2010. The West Virginia Senate Special is not included in the link or in my standings because it was yet to be scheduled.

Tea Party Costing Republican Senate Seats in 2010?

2011-11-19T23:18:00.000-08:00

With so much attention being paid to House redistricting and the ever evolving Republican presidential field, it is easy to lose sight of the 2012 Senate elections. Thankfully, The NY Times recently picked up the pace with an article titled "Feuding Hurts G.O.P.’s Hopes to Win Senate".

The article points out that in 2010 Republicans nominated Tea Party backed candidates in a number of states (Colorado, Delaware, and Nevada) when they were supposedly weaker general election candidates than their primary opponents. But is this hypothesis correct?

The answer is sorta. Yes, Delaware Republican voters threw away a near-perfect opportunity to pickup a Senate seat by nominating Tea Partier Christine O'Donnell over moderate Rep. Mike Castle.

It does not seem, however, that the nomination of Ken Buck in Colorado and Sharron Angle in Nevada cost Republican seats.

Colorado Republican voters passed over the more moderate Jane Norton. Yet, Norton actually was in a weaker standing in general election polls against Democrat Michael Bennet by the time Republican voters chose Buck.

She was also losing ground in the general election polls at the time of the primary.

Buck, on the other hand, lead in the vast majority of general election polls throughout 2010 (including polls taken during the final weeks of the campaign).

It is possible that Norton might have been a stronger general election candidate, but there is no sign that she would have been.

Sharron Angle in Nevada is the more interesting story. Yes, we can agree that she was such a weak candidate that she had to be hidden from the press by her handlers.

But who would have been a stronger general election candidate? The more moderate Sue Lowden? The same Sue Lowden who spoke about chicken exchanges in the healthcare business?

Like Norton, Lowden was in worse shape in general election polls than her Tea Party primary opponent, Sharron Angle, and was losing ground by the end of the primary season.

Angle lead in most general election polls leading up to the November election, only to have Democrat Harry Reid's turnout operation shock the political world.

I am certainly not claiming that primaries do not matter. Candidates can make a difference (though exactly how much is up for debate), but the idea that Republican primary voters cost themselves three Senate seats in 2010 just is not supported by the facts.

The Ron Paul Iowa "Boomlet"... and What It Really Means

2011-11-17T15:26:00.001-08:00

Most pundits have written on how a Mitt Romney victory in Iowa would score him a knockout blow in the Republican presidential primary contest. The basic logic being that Romney has New Hampshire all but locked up, and a victory there and in Iowa would propel him to victories in South Carolina, Florida, and beyond.

A Romney loss in Iowa would give the caucus winner the "conservative" standard bearer badge to Romney's more moderate candidacy.

If current conservative flavor of the month Newt Gingrich won for example, he could easily win South Carolina in his native south. Combine this Gingrich win with the likely Romney win in New Hampshire, and we would be in for an extended nomination process.

Indeed, pretty much of the punditry on the early states revolves around a conservative candidate (whether it be Gingrich, Cain, Perry, etc.) emerging to challenge Romney. The only way Romney can avoid such a scenario is to win Iowa. But a different more interesting solution for Romney may be emerging.

There are signs (see the great Selzer & Co.'s Bloomberg poll) in some Iowa polling that Ron Paul is capable of garnering 20% support of Iowa caucus goers. In a field that includes six other viable candidates (Bachmann, Cain, Gingrich, Perry, Romney, and Santorum), 20% could conceivably win the caucus. Check out a possible outcome below.

If Romney were then to win in New Hampshire, the two early contests would be split between Ron Paul and Mitt Romney.

Considering the momentum effect of Iowa and New Hampshire, it would seem likely that conservatives in later states would line up between one of these two candidates. That is, there would be no "traditional conservative" (as described by the mainstream media) to challenge Romney.

While Paul supporters would argue that he is the "true conservative" in the Romney-Paul pair, most Republican primary voters would disagree.

Paul's positions on foreign policy, national security, drugs, and a host of other issues do not line up with what most Republican primary voters believe. Although Mitt Romney does not exactly have a long track record of "true conservative" stances, his current policy positions on most issues are closer to the core of the Republican party.

That is the reason why Romney not only has a higher favorability among Republicans nationwide, but also has Republicans more enthusiastic about his candidacy. One would think that the great majority of conservative voters in later states would quickly line up behind Romney as the only viable conservative and propel him to the nomination.

In other words, a Paul victory in Iowa is about as good for Romney as Romney winning the state himself.

Keep in mind that the chances of a Paul victory in Iowa are not great, and I certainly do not expect it. But for a primary season that has had more ups-and-downs than an episode of General Hospital, a Paul victory and its repercussions are something we should all keep in the back of our minds.

Presidential Models Have Very Different Results for 2012

2011-11-16T23:20:00.000-08:00

Eight months ago I began blogging extensively on economic statistical models predicting presidential election results. I was not the first to do so, and the recent back-and-forth between pundits, commentators, political scientists, and everyone in-between on which model is the best and whether campaigns matter demonstrates that I will not be the last to talk about them.

My purpose here is not to declare one side right or wrong (though my work indicates that I think Douglas Hibbs' real disposable income growth economic indicator is an imperfect best of the bunch). Rather, it is to point out the most important fact for 2012: the major difference between the well-known Abramowitz and Hibbs model.

Last week, Alan Abramowitz wrote that Nate Silver's model was too pessimistic about Barack Obama's chances for re-election. He cited his own model, which showed Obama garnering 52% under projected economic conditions and being a favorite in almost any economic scenario.

The interesting fact about this critique was Silver's model, which shows probable Republican nominee Mitt Romney to be a favorite at this point, is that it is actually not anywhere close to painting the worst outlook for the President. The Hibbs model forecasts, under most economic conditions, Obama losing by a wide margin (getting only about 44%).

Hibbs model, which has been noted as one of the better of many presidential prediction models (see the great Andrew Gelman), differs greatly form Abramowitz in a number of ways.

Abramowitz's model utilizes GDP growth in 2nd quarter of the presidential election year, presidential approval in the same 2nd quarter, and a dummy variable for whether the incumbent White House party has been in power for more than 1 term.

Hibbs' uses a weighted average (closer to the election matters more) of real disposable income per capita (i.e. how much money each person makes after taxes and inflation) and deaths in unprovoked military conflicts throughout the president's term.

Note the key differences for our sake between Hibbs and Abramowitz are the use of the "term of the party in power" and measurement of the economy. The term variable essentially gives a bonus to the president if his party is newly elected. One might imagine Obama receiving extra votes because some voters still blame Bush for the state of the economy.

My own amendment to Hibbs' model indicates that this "term" variable helps Obama gain about an extra 2% of the vote* given current economic conditions. But under these same conditions, an extra 2% does not even get Obama to 46.5% of the two-party vote. This is not anywhere close to the 52% of the two-party vote Abramowitz's model currently projects for the President.

Why the major variation in prediction? The most obvious answer is that these models are imperfect. They carry large in-dataset errors and even larger out-of-forecast (i.e. future data that model clearly cannot know) errors. Still, 52% vs. 46.5% (or near 44% in the case of Hibbs' original model) is a very large difference.

The main reason for the disparities between these models is that their economic measures are different. Abramowitz's GDP growth was 2.5% last quarter and is expected to be that in 2012. Hibbs' disposable income growth variable for the presidential term is likely not be all that greater than 0% and could easily be negative (as it is right now).

Put concretely, weighted disposable income growth says this economy is by far the 2nd worst for a presidential term since 1952, while GDP of the 2nd quarter of a presidential year says it is only 5th worst (just behind 2004). The charts below says it all.

If you believe GDP, the economy is far better than the one confronted by mind 2008 voters. If you believe disposable income growth, it is about as bad if not worse than the economy for 2008 voters.

So which is the correct economic measurement? Again, I lean towards disposable income (as it is the measurement that is felt by most voters), but I really am not sure there is a right answer one way or the other.

To me, this is the beauty of all of this quantification. In political prediction, there is always going to be two or more ways to measure something.

Many times, as was the case in 2008 and the economy, the measurements will point towards a similar conclusion. Sometimes, however, the measurements will point you towards diverging conclusions (see Nyhan and Montgomery for an intriguing way to combine forecasts).

In fact, I think this is a perfect example of when a campaign could matter. President Obama could use GDP to say the "economy is getting better", while the Republican could use disposable income to say "the economy is getting worse".

I say we let the campaigns make their case. 2012 should be fun.

*Abramowitz states that a party in control of the White House for more than 8 years would lose 4.4% of the vote vs. a party in for only 4 years. I find nearly the same effect for the term variable, if it is included in the formula. My 2% figure comes from whether the "term" variable is included at all, not that whether we include the variable and then code it differently (i.e. first vs second or more terms) as Abramowitz has.

The Democrats' New Hampshire Problem

2011-11-16T16:27:00.000-08:00

Few states matter more in presidential politics than New Hampshire. Not only is it home to the first primary in the nation, but it also a classic swing state. In 2012, it is also supposedly home to a tight House race.

That's why the latest Selzer and Co. poll from the Granite State showing probable Republican nominee Mitt Romney with a 50%-40% lead over President Obama is major trouble for Democrats. If the poll is correct, New Hampshire*, as Sean Trende notes, is "sitting about five points to the right of the [national] electorate."

While it is true that Mitt Romney has somewhat of a home field advantage in New Hampshire because he governed next door Massachusetts and has been campaigning in the state for 5 years running, I propose a far more pro-Republican hypothesis for this New Hampshire vs. nation gap.

The voters of New Hampshire are subject to the political campaign earlier than our fellow countrymen. With non-stop ads running ahead of the presidential primary and candidate visits a dime-a-dozen, New Hampshirites might be moving towards the voting choice that historical presidential models of the economy indicate they will eventually choose.

That is, hard against a second-term for Barack Obama. Once voters from other states start receiving the campaign onslaught as much as Granite Staters, they could be making similar choices.

Worse for Democrats, a big Romney victory in New Hampshire could also have major implications for down-ballot races.

In 2010, Republican Charlie Bass won a very close race against Democrat Ann Kuster in New Hampshire's 2nd congressional district. With a rematch on the way, polling earlier this year had indicated that the slightly Democratic (with a +3 D PVI) district was headed for another photo finish. A 10% Romney win in the state would most likely carry Bass to victory.

Of the Republican seats rated as at least somewhat competitive by the Cook Political Report, NH-2 is tied for being 7th most Democratic on the national level.

A Democratic loss here would eliminate a big seat the Democrats could use on their march towards the 25 Republican seats needed to take back the House. Not only that, the inability to win such a Democratic seat would probably mean the Democrats lack the nationwide wave to win other seats in order to take back the House.

The good news for Democrats is that it is only one poll and it is still early. Maybe the Romney home-state advantage will be even bigger than fellow Massachusetts' resident John Kerry's, who did about 2% better in New Hampshire than he did nationwide.

Still, this poll is anything but pretty for Democrats.

*I should point out that my old friends with the NBC Political Unit, who have access to a lot of insider information (including state polling), have had New Hampshire in their lean GOP column for a few weeks.

Quinnipiac believable?

2011-11-02T21:20:00.000-07:00

Barack Obama and Congressional Democrats have increased their standing among the American public according to a new Quinnipiac poll. But a closer look at the poll reminds us of a divisive question amongst pollsters: should results be weighted by party identification?

Perhaps not surprisingly, not weighting can lead to a situation where results do not change among respondents who identify as a member of a given party, but does change in the overall result because the party i.d. makeup of the poll changes.

Today's Quinnipiac poll, which was not weighted by party i.d., is an example of such a change. As first pointed out by the National Journal's Steve Shepard, the latest Quinnipiac poll's respondents were 4% more Democratic and 6% less Republican than the previous Quinnipiac poll taken in early October.

In the Congressional ballot question, the "generic" Democrat was up 3% (from 39% to 42%), while the "generic" Republican was down 5% (from 39% to 34%) since October. On the question of approving of President Obama's job performance, respondents approved of his job by 6% more (from 41% to 47%) than they did October. The Democratic improvement on these two questions seem to be tied to the increase of respondents who identified themselves as Democrats.

When we break down these two questions' results by party i.d., a similar picture emerges.

Republican support for their party's Congressional candidate dropped by 3% (from 86% to 83%), but Democratic support for their party's Congressional candidate also dropped from 85% to 83%. Independent support of Republican candidate decreased from 35% to 33%. These changes within each party are certainly not enough to create this large of a difference in the overall result.

Approval for Obama is up across all parties: 7% to 10% among Republicans, 77% to 81% among Democrats, and 38 to 39% among Independents. If the sample party i.d. had been constant, Obama would have a higher approval, but not as high as 6%.

Does this lack of change within party merit concern that poll's results are off? It depends on whom you ask.

Most pollsters do not weight by party. This includes CBS, Democracy Corps, Gallup, Pew, and Public Policy Polling (PPP) among others. They only weight by demographic data such as age, education, region, and sex. Quinnipiac reports that these demographics did not change greatly from the October poll.

The reason not to weight by party i.d. varies amongst pollsters, but David Nir of Daily Kos Elections believes that "people can pick and choose their party i.d. If you decide in advance what proportion of the electorate 'should' be Democrats, then, as Tom [Jensen of PPP] says, you're going to miss out on these softer supporters who are apt to change their minds". The demographics (see above) these polls are weighted to do not change.

The argument for weighting by party i.d., practiced most famously by Rasmussen, is pretty simple. The number one predictor of vote choice (and approving/disapproving) is party id, and sometimes the makeup of party i.d. varies significantly from poll-to-poll. The change in Democrats and Republicans in the Quinnipiac poll is right on the edge of what would be deemed statistically significant.

Because both sides make a valid point, I believe the best choice of action is to look at data from other pollsters.

The only other two pollsters that I know who ask the generic ballot question in a manner similar* to Quinnipiac are Rasmussen and Democracy Corps. Rasmussen polls have fluctuated randomly between giving the Republican candidate a lead from 2% to 8% in its 9 samples since September. Democracy Corps mid-October poll showed a tie, little different from its early August poll that gave Republicans a 1% edge.

While the results between Rasmussen and Democracy Corps differ, neither pollster has seen a shift to the Democratic party. That is, the results within each pollster is the same.

Likewise, as this Pollster.com graph of approval ratings show, approval ratings for the President have remained consistent over the last month.

It would appear that Quinnipiac's numbers are a bit of an outlier. It is certainly possible that other polling will begin to show an improvement for Congressional Democrats and Obama. For now, Quinnipiac's poll is a great example of why a pollster's numbers should not be taken at face value.

*Asking Democratic vs. Republican candidate as opposed to asking whether respondents want a Congress controlled by the Democratic or Republican party.

** See Mark Blumenthal for the entire rundown on weighting by party identification.

How Conservative is Michele Bachmann?

2011-06-20T16:58:00.000-07:00

"Michele Bachmann is nuts!" is a line you can expect many Democrats to offer if Bachmann wins the Republican nomination for President. Bachmann does have the dubious distinction of perhaps suggesting a new round of McCarthy-like trials to find anti-Americans in government...

But is Michelle Bachmann really as far-right as some portray her to be?

I think the best way to determine a politician's political ideology is to look at his/her record. That is, we look look beyond the rhetoric to look at a candidate's actions, not words.

To do so, I suggest we look at Bachmann's voting record in Congress as represented by her DW-Nominate score. Last year, I completed a similar exercise and found now Sen. Pat Toomey (R-PA) to be far more conservative than I originally thought.

DW-Nominate scores have the distinct advantage of being objective, not subjective. They can also be compared across Congresses. The real disadvantage is that we cannot compare among governors.

As I noted in my Toomey piece, "DW-Nominate scores classify House and Senate members as liberal or conservative based on all their roll call votes than can be identified as liberal or conservative. These scores allow one to compare how rightward or leftward legislators are on a single dimension -1 to 1 scale with higher positive scores indicating a more conservative record."

So where does Bachmann rank?

Among the Representatives and Senators who have served since 1995, Bachmann's record is more conservative than 92% of them with a DW-Nominate score of 0.577. If Bachmann ever become the Republican nominee, Democrats could make a honest case that Bachmann is far to the right of the median legislator.

In a Republican primary, the case for Bachman's far-right conservatism is less clear. Bachmann's rating is about 85% more conservative than her fellow Republicans. That puts her in the conservative part of the Republican caucus, but not too far right.

Consider that the architect of the GOP budget and medicare plan Paul Ryan (R-WI) has a nearly equivalent score of 0.562.

In a race where support for Ryan medicare plan has become a litmus test, Bachman seems in a good position to win over many very conservative and plenty of moderately conservative Republicans too. Her record is also less conservative than Senators Jim DeMint (R-SC), Tom Coburn (R-OK), and Pat Toomey (R-PA).

How about among past Republican nominees for President since 1996? Among this relatively small field, Bachmann finds herself to the right, at right around the 85th percentile.

Since 1996, there have been 17 Representatives and Senators who held Congressional office post-1995 and ran for President. Of those, Bachmann is the third most conservative. Only Ron Paul and Tom Tancredo have a more conservative voting record. Her closest match is Phil Gramm, who was a pretty serious contender in 1996.

Bachmann is not so conservative as to be "extreme" in a Republican primary, but conservative enough to be outside of the mainstream. Of course, primaries are more than just about how one matches up ideologically within one's own primary.

Michele Bachmann could probably win an election among Republican primary voters. But with "electability" in a general election becoming more important to Republicans in recent months, primary voters are more likely to deny her the nomination because she of how her voting record would be viewed among a general election electorate.

Notes:

"*For those interested, you can read a more in-depth non-technical explanation of DW-Nominate scores here and a more technical discussion here."
"**The reason I use 1995 as the cutoff is because prior to the 1980's a legislator's liberal-conservative record was also highly correlated with a second dimension of DW-Nominate scores.
Since the 1980's, however, the scores I use correlate highly with a legislator's overall record vote. Also, many conservative Democrats, who left the Congress after 1994, made Congress less polarized. In an effort to correctly contextualize each legislator's record discussed here, I decided to use 1995 as my starting point for scores."
***This exercise looks only at the first dimension of DW-Nominate scores. Prior to the late 1990's, a second dimension was necessary to pick up on social issues (e.g. civil rights). This second dimension still picks up on some social issues (e.g. abortion), but these issues are "increasingly picked up by" the scores used in this piece.

The Hibbs Model DOES Work at Predicting and Explaining Elections

2011-06-17T22:56:00.000-07:00

Presidential elections are all about the economy, or are they? Those who have followed my writing know that, at least at this point in the election cycle (far away from the actual 2012 general election), I am big fan of utilizing fundamental models. I stand by this belief.

One of my favorite models is the Douglas Hibbs' fundamental Presidential economic model.

By using just two simple variables (growth in real disposable income per capita over the 16 quarters of a presidential term and military fatailities in unprovoked foreign conflicts), the model explains 87% of the difference incumbent party's share of the two-party Presidential vote from 1952-2008.

Recently, the model came under attack.

The New York Times' Nate Silver ran through a bunch of exercises that seemingly proved that the model was not nearly as accurate at predicting out-of-dataset results for the 1992-2008 elections as its in-dataset (i.e. those data that help build the model) confidence interval would lead you to believe. The model was far worst when fitted to elections prior to 1952.

I appreciated Nate's post because I think we should always be re-evaluating forecasters' claims and theories. I, myself, had previously noted that Hibbs' model would do a poor job at predicting the 1948 election. There can also be little doubt that in-dataset confidence intervals almost always undersell actual uncertainty in estimates.

Yet, I must admit that I think Nate's post misses on a number of points.

Let's first cover the out-of-dataset forecasts for those elections that Hibbs' model has actually been used to predict (i.e. 1992 and onward).

My own amended Hibbs' model (with terms added for congressional control and term of the party in the White House) performs better than the original Hibbs' model with an absolute mean error for out-of-dataset forecasts of 2.09% instead of 2.6% of the original.

With the (big) exception of the 2000 outlier, all of the 92 and onward out-of-forecast errors fell within each year's in-dataset 95% confidence interval (which should be, as noted, too confident).

Personally, I think that is pretty good. We were able to use only a few data-points in each year and no polling data to get a pretty gosh darn good idea of what would occur in these 5 elections.

Of course, in forecasting, we judge a prognostication not only on its ability to predict the final outcome, but on how well it completes this task relative to other predictions. In that spirit, I have decided to dredge up final weekend polling from 1992-2008.

Final weekend polling should be fantastically accurate. Almost all the campaigning is done; almost all the voters have made up their minds; and, all of the polling is of likely, not registered, voters.

Amazingly, in these five elections, the final polling mean is only on average 0.98% more accurate in predicting the final outcome than my amended Hibbs' model. In fact, the amended Hibbs estimate was actually more accurate in two elections (1992 and 1996), and, unlike the polling average, always correctly projected the popular vote winner.

I should point out that polling is far less accurate the further we recede from the election to the point of having no predictive value 300 days out. (We are currently more 500 days away from the 2012 election).

The Hibbs' models, on the other hand, have the potential to produce solid estimates on who will win far from the election. These days we can suppliment known economic data with economic forecasts to produce reasonable estimations of the vote even, I believe, at this point.

But what about Nate's point that the Hibbs' model widely misses on elections prior to 1952?

There are a number reasons not to input data prior to 1952 into the model. Some of those (including how voters viewed the role of government differently prior this party era) are outlined by Seth Masket.

One other point that I have not heard brought up is simply that prior to 1952 elections lacked any true television campaigning. It's difficult to underestimate the effect of how television nationalized elections.

I think these factors that apply to 1952 and afterward make these elections fundamentally different to those prior. It's not surprising that 5 out of the 7 out-of-dataset errors Nate found prior to 1952 are approximately 2 times or more greater than any of those from 1992-2008.

Indeed, political scientists have known elections prior to WWII and afterward seemed to respond differently to the fundamentals.

For example, esteemed political scientist James Campbell noted that there was a difference between presidential coattails pre and post-WWII and actually mapped out different equations for pre and post-WWII.

Some scientists have tried to fit pre and post-1952 data into one model. During the 2010 midterm election, for instance, we saw many fundamental models (i.e. no polling data) of this design, but not a single one correctly projected the Republicans winning control*.

In fact, the only fundamental that saw the House turnover was Hibbs' midterm House model, which uses real disposable income growth data post-1948 like the Presidential model.

So, these combined models may take into account more years, but more years does not necessarily mean a better forecasting model.

If Hibbs' Presidential model initially included Nate's estimates for real disposable personal income growth prior to 1952 (i.e. 1924-1948) and was refitted to predict the 1992-2008 elections out-of-dataset, Hibbs' forecasts would have been worse.

Both the 1992 and 2008 elections would have been incorrectly forecasted to be won by the Republican party. Instead of the 2.6% mean error of the original Hibbs' model, Hibbs' estimates would have been off by about 3.91% with '24-'48 data added.

This difference in 1992-2008 out-of-forecast errors between the models that include and do not include pre-1952 data is so large that it's actually greater than the difference in error between my amended Hibbs' model (2.09%) and the final weekend polls (1.11%) during the same 1992-2008 period.

Personally, I want the model that best predicts the future, not one that makes us feel good inside for post-projecting elections prior to 1952.

But let's say that you are interested in knowing how well the economy explains pre-1952 vote. Nate would like you to believe that these high pre-1952 errors prove that "it’s the economy, stupid. And everything else too" when it comes to explaining Presidential elections. And in one way, I agree with him.

See, the number 1 predictor of how any voter casts her/his ballot is not the economy, but party identification. It's the main reason why we never see Presidential candidates winning much more than 60% of the vote, or the losing candidate earning less than ~40%.

In other words, in a race between a Republican and Democratic candidate, each candidate starts with a base of about 40% of the vote with only about 20% of the vote up for grabs.

This means that even if there was 15% growth (or decline) in real disposable personal income, the effect of this growth on the vote would run out as most of the population makes their vote choice based upon the candidates' party affiliation.

You can see this concept graphically for the incumbent party's, which typically wins a greater, vote share. Note the flat tails at each end as growth becomes incredibly positive or negative.

This high growth/decline factor is not an issue in the 1952-onward data, but it is for the election years 1932, 1936, and 1944.

The question is how do we control for this issue. One adequate and simple (but not perfect) way is to simply cap the economic growth variable at -5 and +5. That is, we simply recode the economic variable in 1932 as -5%, in 1936 as +5%, and in 1944 as +5%.

Let's also add an interaction variable between term of the party in the White House (a fundamental dummy variable that has been found to be quite significant in other models) to this recoded real disposable personal income growth variable and the military fatalities' variable.

Doing so, we can now explain not 59% of election results between 1924-2008, but 77.4%! Three variables** accounting for 77.4% of the difference in results among 22 elections spanning two party systems, a World War, and the move into the television age is fantastic as far as I'm concerned.

But I actually believe that we can explain more than 77.4%. When I first built my amended Hibbs' model, I too had searched the Census archives, but could only find yearly data on disposable income from 1929-onward.

I must admit that I was surprised that Nate found the growth for the 1928 election to be only about 0.30% (the amount needed for Republican Herbert Hoover to receive 47% of the two-party vote). After all, the 1920's had a supposedly booming economy.

In search of data, I plowed through Google and was able to locate disposable personal income data from this (gated) 1946 economic paper.

The data seems pretty accurate***, and I calculate growth rates**** in real disposable income per capita of 2.00% for 1925, 1.35% for 1926, 1.37% for 1927, and 3.39% for 1928.

Remember, however, that Hibbs' model is based off quarterly income growth in which later quarters are weighted more heavily than earlier ones. Exactly how one converts yearly estimates to quarterly will determine the exact weighted quarterly growth rate.

There is no perfect method, but my own imperfect estimation is 1.80%.

I have no idea whether Nate's or my finding for income growth heading into the 1928 election is correct, but I feel my calculated rate better jibes with the overall consensus the economy was doing quite well during the 20's (despite a few bumps along the way).

If we assume my growth finding is right, then 82%, not 77.4%, of the differences among election results between 1924-2008 are accounted for by the model. This means that this model has only about 5% less explanatory value than the 87% of the original Hibbs' model for 1952-2008.

I find it amazing that three non-polling variables can explain 82% of difference in results among 22 elections spanning two party systems, a World War, and the move into the television age.

I still stress, however, that I don't find this pre-1952 prediction exercise all that important for predicting results in this modern age.

We should not incorporate more data into a model just to boost the number of data-points. As I discovered, our ability to forecast elections with economic modeling in 2012 and beyond is impeded, not helped, by adding data prior to 1952.

Overall, these exercises should give us confidence in economic modeling to give us a good early read on the political environment in 2012. Certainly the state of the economy conveys upon us a better idea of the ultimate outcome than determining which candidates smile more or are more charismatic or even the early horse race numbers.

I am willing to say right now that, unless something totally unexpected occurs, the 2012 presidential election will be about the economy just like pretty much every election since 1924. We should be looking growth in real disposable personal income per capita to lead the way.

Notes:

*Ray Fair's House model did not estimate seat share. However, almost all political scientists would agree that to win back the House, House Republicans, due to the Democrat's incumbent advantage, needed to take the popular House vote by more than the 1.6% Fair's model projected.

**Four if you include the original "term" variable, although this makes no difference in explanatory power.

***Note, I found slightly different estimates of real disposable income in a few other papers, but all led to the same conclusion I make.

****Yearly income data is given as the average for a given year. I assume that this average is equal to the real disposable income at the mid-point of a given year. To determine what the income is at the beginning of a calendar year, I average the income data for a given year with that of the prior year. It's not perfect, but I tested it on known beginning year data points and found it to be more than acceptable.

Troubling Economic Signs for Obama's Re-Election

2011-06-02T00:02:00.000-07:00

President Obama is currently flying high. Off the killing of Osama Bin Laden, Obama's approvals have risen into the low to mid 50's, and he leads all the major Republican challengers in the 2012 Presidential race. Unfortunately for the President, recent economic news suggests that this boost is nothing but a facade for what is shaping up to be a tough re-election fight.

The Bureau of Economic Analysis (BEA) released new numbers on the state of the economy this past Friday, and they were much worse than expected.

Many analysts will point to the lack of upward revision in Gross Domestic Product (GDP) and poor Unemployment as bad, but of far greater importance to President Obama's re-election hopes is the tumble in real disposable personal income per capita (RDPI).

According to the BEA's prior April report, RDPI grew at 1.8% in the fourth quarter of 2010 over the preceding quarter and 2.9% in the first quarter of 2011. Last Friday, the BEA re-adjusted those numbers to 1.1% and 0.8%.

Why should the President be worried?

It turns out that weighted* quarterly growth in RDPI (along with three other already known variables* for the 2012 election) over the President's term can account for 92.9% of the variation in the incumbent party's percentage of the two-party vote in the 1952-2008 Presidential Elections.

To put this 92.8% of explanation into context, Chris Wlezien has shared with me Bob Erikson's and Wlezien's finding (in a must read upcoming book) that polls during the final week of the campaign during this same 1952-2008 stretch account for 93% of the variation in the incumbent party's share of the two-party vote.

In other words, both RDPI and the last minute polls will likely do a good job at predicting the final outcome in 2012.

Polls 300 days out from the election from this same time period, however, explain less than 5% of the outcome and will probably do a very poor job at predicting the final outcome 2012.

Thus, I believe it is more important to pay attention to RDPI prior and projected growth than polls at this point (far more than 300 days away from Election Day 2012) in the campaign.

We can estimate the RDPI growth the country will look back upon in the 2012 Presidential election by combining BEA reports from 2009 to the present with RDPI projections through 2012 from Wells Fargo.

With the new BEA numbers from last Friday, President Obama's term is predicted to be 12th out of 16 for weighted RDPI growth for Presidential terms beginning with Harry S. Truman's first full term.

According to my model, which has an in-dataset margin of error of 3.3% at 95% confidence, Obama is forecasted to win only 51.1% of the two-party vote. While Obama leads, his lead is definitely within any margin of error.

Predicting where the economy will turn is not an easy task. Otherwise, we probably would be able to avoid many economic crises. While it is possible that future RDPI growth will do better than Wells Fargo believes, I believe it is more likely to get worse.

Beyond the dire Drudge Report headlines, Wells Fargo's economic forecasts (which have tended to be far more pessimistic on other economic measures like GDP compared to the Philadelphia Fed's) have been overestimating RDPI growth for 2011.

Their January projection for first quarter of 2011 growth was 4.0% (against a 0.8% reality). Their January 2nd quarter of 2011 projection of growth was 3.5% and is now 1.0%; their 3rd quarter projection was 3.3% and is now 2.1%; and, their fourth quarter projection was 3.0% and is now 2.5%.

In other words, the 2011 economy is already doing worse that prognosticated. Any further dip in the economy, and President Obama might find those comparisons between the effect of capturing of Osama on his approval ratings and the effect of the Gulf War on H.W's Bush's hitting a little too close to home.

NOTES:

*Each successive quarter's economic growth is weighted more than the prior, so that final quarter's growth of a President's term is far more important than the first. See Douglas Hibbs for more details.

*The three variables are fatalities in aggressive foreign wars started or continued for more than one term by the President's party, term of the Party of the President in the White House, and whether the President's party controls Congress. They are explained at length in a prior post.

My model is based off Douglas Hibbs' Bread and Piece model, which accounts for RDPI and fatalities. Plugging in the same forecasted RDPI growth into the original Hibbs' model finds President Obama garnering only 49.6% of the two-party vote!

The reason for this difference between models is my model's accounting for term of the Presidential party. When a party is in its first term in the White House, it has generally received a bonus.

Institutional Factors Keep Obama The Favorite

2011-05-22T19:07:00.000-07:00

President Obama seems like a near lock for re-election... Or so it would seem given that potential Republican candidates are dropping out of the race quicker than they can say "Reagan".

An often cited reason for Obama's great chances is that first term Presidents for a given party's term (so George H.W. Bush was actually the third term of a Republican administration) almost always win re-election. But is this true? Is another factor helping him?

The best way to understand whether or not Obama is aided by being a first-term President is to model past election results.

Alan Abramowitz's "Time for Change" model attempts to achieve this task. Utilizing second-quarter GDP growth in the Presidential election year, Presidential approval in the second quarter of the election year, and a dummy variable (1 or 0) for whether the party in the White House has completed 1 or more terms, he finds that the term variable is very important.

In fact, President Obama is estimated to garner 4.4% more of the popular vote than he would if his party were in its second term in the White House for a projected percentage between 53-54%.

Abramowitz's model, however, as good as it is at predicting, is not a straight fundamental model. The Presidential approval term can encompass many variables (wartime fatalities for instance) and does not explain the "fundamental" cause of disapproval of the President's job.

To test whether Abramowitz's results on term of the party in the White House hold true in a totally fundamental model, I turn to Douglas Hibbs' well-respected fundamental model based off of 15 elections worth of weighted real disposable income growth per capita (DPI) and fatalities in aggressive foreign wars* over the Presidential term.

We find that the addition of the variable for term of the party controlling the Presidency (0 for 1st, 1 for 2nd or later) does have a statistically significant impact (at the 90% confidence interval) on Hibbs' model. Given the current and projected state of DPI for a weighted value of 1.18 (see this post for more on economic projections), Obama is estimated to receive 51.6% of the two-party vote.

If 2012 was the end of the second or later term of a Democratic administration, Obama's estimated vote percentage would be 49.3%. The effect of the term variable seems less than the Abramowitz model, but enough to turn Obama from an underdog to a favorite.

One other factor that some argue is helping Obama's re-election chances is the Republican House. Regardless of the ideology of this Republican House, the center of the American electorate seems to like divided government.

Plugging an ordinal variable (0 for no, 1 for half, 2 for yes) into our reformed Hibbs' model (with term of the President taken into account), we can reestimate our model to test whether or not Congressional control by the President's party adds to our ability to explain past Presidential election results.

It turns out that when President's party controls Congress, it hurts the incumbent's party chance of re-election to the White House. This effect is statistically significant at the 95% confidence level and makes the term variable have a greater affect and is more significant. This new model has an in-dataset margin of error at 95% confidence of 3.3% and explains 92.9% of the past 15 Presidential elections.

Our re-estimated model controlling for Congress projects Obama to win 51.8% of the two-party vote (little different from 51.6% for the model without a variable for Congressional control).

If, however, the Democrats controlled all of Congress (2 for yes) instead of half of it (1), Obama's predicted percentage would drop to a calculated 50.2%. As the table below illustrates, his projected vote would drop to 46.1% if this control of Congress were combined with a second (or later) White House term.

As it is, Obama looks to be a favorite, but his election is not assured even given his term and split control of Congress. The few Republican challengers left in the race have a decent shot of winning and even better chance if the economy does worse than forecasted.

*The variable takes on a value of 0 in 2012 because the war must be started by the current President's party.

Predicting Elections is like the NFL... We have parity (and may not have a major contest until 2012)

2011-05-09T00:54:00.000-07:00

As some of us have tried the first attempts at predicting the 2012 elections, we are all wondering which prediction will be most accurate at the end of the day. Can we skip most of the "expert" predictions and concentrate on only one website. The answer it would seem is yes... but only because we're likely to get the same answer wherever we go. Why?

Pretty much all expert projections are privy to and rely on the same polling data, and most pollsters are as accurate as one another. Nate Silver's pollster rankings suggest that this is not the case, but there is considerable debate within the academic/polling community over whether his rankings are statistically significant in most cases or hold any predictive value for future campaigns.

Without reliving old arguments over statistical significance (see this Mark Blumenthal piece on anti-significance and Silver's response), I think the predictive value is far more important.

Following the 2010 midterm elections, Nate released a preliminary 2010 general election pollster scorecard* for eight pollsters. As you can see, their errors ranged from 3.3-5.8% with Rasmussen registering as the least accurate. How did their relative rankings compare with past performance?

There was actually a slightly insignificant relationship. The highest ranking pollster (Quinnipiac) was actually ranked lowest in the previous incarnation of the pollster rankings. YouGov, which ranked 7th, came in at 3rd. Disturbingly, the prior rankings had punished YouGov because it conducts polls over the Internet. In other words, an effort to add to prior pollster performance to create a more accurate forecast of future accuracy did not help.

Based on this evidence as well as the fact that a different rating system by American Research Group's Dick Bennett that included data from the primary actually found Quinnipiac towards the bottom of the pack (due a poor primary performance), I do not believe that, for the most part, past pollster accuracy foretells future performance.

Not surprisingly then, poll aggregation techniques are as accurate as each other. Chris Bowers (a pioneer of simple polling averages) found that the difference in accuracy between the final predictions of Pollster.com, FiveThirtyEight.com, and a simple 25-day polling average for the 52 closest Presidential, Senatorial, and Gubernatorial contests in 2008 and early 2010 (before the general) was only 0.27% (with the simple average coming out statistically insignificantly ahead).

In the 2010 general election, Bowers' calculated that in closest 45 campaigns the difference in mean error between Pollster.com, FiveThirtyEight.com, Real Clear Politics, and a simple 25-day polling average was only 0.31% (with FiveThirtyEight coming out statistically insignificantly ahead). Dick Bennett's review of the aggregation methods found similar insignificance in error.

What about in the House of Representatives, where neither Bowers nor Bennett have roamed?

Few websites, I know of, actually attempt to predict the results (not just the winner) in each House race. To do so, you need not only polling data, but also past district voting history (on both the Congressional and Presidential level) among other variables.

The only two that did so in 2010 were FiveThirtyEight.com and Stochastic Democracy (which I have previously contributed to). Looking at all the House races that had both a Republican and Democratic candidate (406 in total), I found that average error (on the two-way) was 6.27% for FiveThirtyEight.com and 6.72% for Stochastic Democracy. This difference is not statistically significant at any mathematically accepted level.

What happens when we just look at who better predicted the winner? FiveThirtyEight.com had 19 missed calls, while Stochastic Democracy had 18. Again, not statistically significant. Another prominent site, Larry Sabato's Crystal Ball (to whom I have recently contributed) also had 18 missed calls. Most prominent websites had similar track records.

Where does this leave us? Any prognosticator/pollster who claims that they are more accurate than the other guy probably is not (at least for more than one cycle). The fact that a simple 25-day and simple Real Clear Politics average does as well as some of the more complicated methods in statewide contests indicates that those at home can try their own hand at beating the pros (and on any given day have a decent chance of doing so). Most importantly, I feel secure knowing that readers are getting good information no matter where they go.

*I have not seen an updated scorecard, but would be more than happy to update the post based on a new one.

Note: If you are interested in any part(s) of the 2010 House dataset, feel free to email.

Minnesotan Marriage Ban Is Early Favorite

2011-05-05T00:11:00.000-07:00

It now seems likely that Minnesotan voters will vote in 2012 on a constitutional amendment to ban same-sex marriage. If you have been following the national polls and punditry, you might be led to believe that the amendment will probably fail. At this point, however, I would put the money on it to pass. Why?

1. The latest polling (I could find) discovered that more Minnesotans are against same-sex marriage than are for it. The poll from September 2010 found that among likely voters 49% opposed same-sex marriage, while 41% were for legalizing same-sex marriage. Reallocating undecideds based on decided voters (as there is no undecided when it comes time to vote), 54% of Minnesotans are against legalizing same-sex marriage.

Some might point to a 2006 poll, which indicated that even though most Minnesotans were against same-sex marriage, they would vote against a constitutional amendment to ban it. The fact is that polling before the California's infamous Prop. 8, a constitutional ban against gay marriage, polling suggested a similar split*: a number of Californians who were against same-sex marriage would vote against Prop. 8. In the end, most voters against same-sex marriage, but also against the amendment, voted for Prop. 8. I would expect a similar trend in Minnesota.

2. Minnesotan demographics indicate that the amendment is likely to pass. Take a modification of Nate Silver's same-sex marriage model that controls for a state's religiosity, a state's median voter's level of conservatism on social issues (with -2 being very liberal to 2 being very conservative), the year of the election, whether the election was held during an off-year or non-Presidential primary, and whether the ballot measure sought to ban same-sex marriage and civil unions or just same-sex marriage.

64% of Minnesotans consider religion to be an important part of their lives; Minnesotans' tend to be quite moderate on social issues (with a score of -.08); 2012 is 15 years after the first gay marriage amendment nationwide; 2012 is not an off-year; and, the measure seeks only to ban same-sex marriage. Given these variable values, the model projects the marriage amendment to pass with a little over 56% of the vote (quite close to the 54% polling number above).

Ah, but only if it were so easy to predict. Points 1 and 2 come with some caveats that deserve explanation.

1. Like the rest of the nation, support for same-sex marriage legalization seems to be increasing. In the aforementioned 2006 poll, only 29% of Minnesotan voters** supported gay marriage, while 54% opposed. That means that in 4 years, support climbed 12%, while opposition dropped by 5%. If that trend continued over the next 2 years, we'd be looking at an electorate that evenly split on the marriage question come 2012.

2. The demographic model has a within dataset margin of error at 95% of about +/- 8.3%. The model is telling us that it is not unreasonable (even if unlikely) that the Minnesotan same-sex marriage ban fails with 49%. For Maine's 2009 same-sex marriage referendum, the model out-of-dataset forecasted the ban to fail with a little greater than 47% of the vote, but it actually passed with a little less than 53% of the vote (an error of about 5.5%).

Keeping these qualifications in mind, I should drive home the point that past history does not look too kindly upon the pro-same-sex marriage side.

As I had previously found and Patrick Egan has expanded upon, same-sex marriage ballot questions tend, if anything, to do worse on election day than pre-election polls predict. In fact, Egan found the bans ("yes" side) picked up, on average, 7% support from the final polls, while the "no" side picked up no appreciable support. In Minnesota, therefore, we might expect the ban to pass in the high 50's, instead of the 54% projected above.

Further, Egan demonstrated that campaigns have little effect on the outcome. The polls six months before the election occur were about as accurate as those right before the election. While October 2010 is far more than six months out from 2012, I would imagine that the swing in favor of same-sex marriage (and against the ban) will probably not advance to the point that two sides reach equality in vote share as spoken about above.

Given all this information, the ban side seems to have the edge. The election will not take place until 2012 and many things may change. Minnesota may defy the trend, but I would not count on it.

Notes:

* The two situations are not exactly similar, however. Without the constitutional amendment, California would have continued to allow same-sex marriages. Even if the marriage amendment fails in Minnesota, state law bans same-sex marriage. Perhaps, more of the "no to marriage, but no ban" folks will vote against the amendment in Minnesota.

** The sample population for the two polls are not the same. The 2010 poll employed a likely voter sample, while the 2006 sample utilized a registered voter model. 2010 was a very Republican year, so it is probable that the electorate was more liberal in 2006 and is more liberal in 2012. That makes the gains in support of same-sex marriage between 2006 and 2010 more impressive and means that new polling for 2012 might reveal a closer election than the 2010 poll indicated.

Britons should vote NO on AV

2011-05-03T22:13:00.000-07:00

On Thursday, Britons will vote on the voting system to be used in future parliamentary elections. This blog does not endorse candidates or parties, but it does endorse methods of voting and vote counting. The current British method, plurality, is elegantly simple: each voter selects one candidate, and the candidate with the most votes wins the election. It is not perfect, but the substitute method up for a vote, alternative vote (or instant runoff voting), is far too flawed for British voters to approve.

Alternative vote (AV) is a type of preferential voting in which voters are asked to rank the candidates from first to last. The basic idea is that if no candidate is the first choice of 50% + 1 voters, then the candidate who received the fewest first place votes is eliminated. This candidate's voters then have their votes reallocated to the candidate they ranked second. This reallocation process continues until one candidate achieves 50% + 1 votes (more on this later). A majority is achieved (or so we think)!

AV supposedly gives voters more freedom of expression to vote for the candidates they want. I will allow AV supporters to develop this argument further.

What I want to do first is give you three reasons why AV is an unacceptable system to me.

1. No-Show Paradox. The number 1 rule I hold for any voting system is that when you vote for a candidate you should be helping her/him. Conversely, when you stay home and do not cast a ballot, you should hurt the candidate you want to win. In plurality voting, these simple (and very logical) rules hold true. In AV, these rules do NOT hold.

Consider, the follow 21 voter and three candidate example borrowed from Warren Smith's Range Voting website (a great site for a more in-depth look at the problem described below).

In this example, the Liberal Democrat candidate has the most votes (8), but it does not hold a majority. The Conservative candidate has the second most first places votes (7). The Labour candidate has the fewest first place votes (6), and its voters have their votes reallocated to their second choice.

With the votes reallocated from those who ranked Labour first, the Conservative candidate has a majority with 13 votes and wins. All seems well... but is it?

What happens if 3 of the voters who ranked the Liberal Democrat first got sick on Election Day and could not make it to the polls? The modified electorate from above would look like this

Now, the Conservative candidate has the most first place votes (7), while the Labour candidate has the second most first place votes (6). The Liberal Democrat is eliminated, and its voters have their votes reallocated to their second choice.

All of a sudden, it is the Labour candidate (with 11 votes) who wins the election. Nothing may seem wrong with that at first glance, but a further examination reveals something is quite wrong here. Three voters who preferred the Labour candidate to the Conservative were only able to get the Labour candidate elected by NOT VOTING! Put another way, the three voters who were sick would have gotten their least favorite candidate elected by voting!

The mere thought of such an event occurring is a deal breaker for me. Another scary statistic is that when the AV and plurality winner differ, the chance of a no-show paradox occurring is about 50%!

2. Non-monotonicity. Related to the no-show paradox are violations of monotonicity. Ranking a candidate (e.g. Liberal Democrat) higher should help, not hurt, her/his chances of winning, while ranking them lower should hurt, not help, their chances of winning. Seems pretty obvious, and we know in plurality voting that voting for a candidate helps them, while not voting for them hurts them. Yet, AV fails this simple test.

Consider, this one example, again from Warren Smith's Range Voting website, with 3 candidates and 17 candidates demonstrating how IRV show this terrible characteristic.

Here, the Labour candidate has the most votes (8), but just misses a majority. The Conservative advances to the second round with 5 votes, and the Liberal Democrat is eliminated with 4 votes. With the votes from those who placed the Liberal Democrat first redistributed, the Conservative comes from behind and wins 9 votes to 8 over the Labour candidate. Seems good that a majority was formed... right?

But look at what happens when two of the voters who ranked the Labour candidate first and Liberal Democrat second decided at the last moment that they preferred the Liberal Democrat to the Labour candidate...

Now, the Conservative is eliminated with only 5 first place votes, while the Liberal Democrat and Labour candidate advance with 6 votes. With the Conservatives votes re-distributed, the Labour candidate has won the election over the Liberal Democrat 11 to 6.

Did you just see what happened? 2 voters who had initially preferred the Labour candidate to the Liberal Democrat were able to secure the election of a Labour candidate by ranking the Liberal Democrat over the Labour candidate!

And yes, it is possible (see Smith's website) to ensure the defeat of a candidate by ranking so-said candidate higher (as opposed to ranking a candidate lower to ensure victory as the above example illustrates). In combination, these two monotonic problems occur no less than in 5% of the time in 3 candidate elections. The percentage is even higher for elections with more candidates.

3. AV violates one-man/one-vote. Certain electoral systems (e.g. cumulative voting) allow voters to vote for more than one candidate, but they give each voter the right (even if they choose not to execute it) to theoretically cast and have the same amount of votes counted as one another. The current British system is one-man/one-vote. AV, on the other hand, gives certain voters more votes than others.

Take a gander at any of the examples above. You'll note that the voters who have their first choice eliminated in the first round have their second-choice votes counted in a second round. In other words, they have 2 votes counted. The voters whose top choices make it into the final 2 only have 1 vote counted. If there were more candidates in this election, there is the obvious possibility of some people having infinite votes counted, while others still having only 1.

Now, for those whose first choice is the winner, I have little sympathy. But re-examine our first example,

As shown above, the Conservative wins, while the Liberal Democrat (who would win a plurality election) advances to the second round and loses. The voters who ranked the Liberal Democrat first only have 1 vote counted. This characteristic of AV is even more ridiculous when you realize that the Liberal Democrat voters in this election would have preferred any candidates besides the Conservative. In fact, they could have formed a majority with first-place Labour voters to allow the Labour candidate (Liberal Democrat voters second choice) to win the seat.

An unfair outcome has clearly occurred.

Finally, I want to make two points with regards to the idea that AV allows a majority to be formed.

1. You might be lead to believe that AV guarantees that the Condorcet winner (the candidate who would win one-on-one match-ups against each other candidate), if it exists, wins the election. The truth is that it does not.

Look directly above to the prior example, and you'll quickly realize that if there were one-on-one match-ups, the Labour candidate would win every single one. 14 voters prefer the Labour candidate to the Conservative, and 13 voters prefer the Labour candidate to the Liberal Democrat. Yet, in AV, the Labour candidate is the first one eliminated. AV does not even do something that many would argue it should.

2. Recognize that the candidate who eventually wins the majority does not necessarily have the vote of 50% + 1 of the voters who cast a ballot. When a ballot is filled with many candidates, many people might not rank all of the candidates (e.g. for lack of time).

I can speak from real experience. In the 2010 Dartmouth Student Assembly Vice-Presidential election, which utilized AV, Brandon Aiono won by 7 votes over Will Hix (1,075 to 1,068). It turns out, however, that 2,246 ballots were cast in this race. That means that Aiono actually only won with a little less than 47.9% of the people who voted in this race.

If AV actually encourages more third-party voting (which is debatable) as some AV supporters suggest, then it really is possible than many Britons will not rank every candidate. If they do not rank every candidate, then we could get a winner who does not have support from 50% + 1 of the voters.

This last example is typical of AV's failure to deliver. AV gives voters a false sense of security that they are in greater control of the electoral process. In reality, they are in less control and cannot truly be sure what their vote truly means.

A Generic Lead Doesn't Meant What You Think

2011-04-19T14:39:00.000-07:00

Like most people, I like to get to get a lot of mileage on one tank of gas. When it comes to predicting Congressional elections, this means I try to get as much as I can out the national generic ballot question. Yet, it is useful to know the limits of this tool.

1. Like any poll, be aware of the sample population. In the past few weeks, Rasmussen (likely voters) has shown Republican leads of 3-6%, while Public Policy Polling (registered voters) shows the same lead for the Democrats. While the difference between the two pollsters may or may not fully explained by sample universe, it definitely is part of the difference. Not surprisingly, all other pollsters employing a likely voter model have also found that over the past few months the Republicans lead on the generic ballot by anywhere from 2-7%.

2. Democrats are going to need to do better than a tie in the national vote to earn a tie in the seat count. Due to incumbency advantage (among other factors), the party in control of Congress (the Republicans) can actually lose the national House vote and still maintain a majority. In 1996, for example, Democrats slightly bested Republicans in the national House vote, but were only able to gain 2 seats off their 1994 performance.

3. Re-districting efforts make converting vote to seats harder. If you look around the web trying to find a vote to seats curve you'll quickly notice that many of them leave out re-districting years (e.g. what 2012 is). Any estimate of vote to seats will be difficult in 2012 without a re-adjustement for re-districting, and we still do not know how many of the state Congressional maps will look.

4. Despite difficulties in estimating a vote-seat curve, it seems reasonable to believe that the 2010's re-districting will be not be pretty for Democrats. Republicans hold control of re-districting efforts in more states than usual. Therefore, Republicans could potentially do better than any prior vote-curve suggests.

5. A model that a friend of mine (who has specifically asked for no credit) helped develop illustrates how favorable re-distrcting in the 2000's (not anywhere as favorable for Republicans as the 2010's) definitely helped Republicans.

Utilizing just prior seat percentage in Congress controlled by Republicans, national vote, and a dummy for the 2000's and 1950's, we get a rough guide (95% confidence interval of +/- 14 seats) for how many seats Republicans gained in Congressional elections (including re-districting years) since 1942. Note, the model would have out-of-dataset predicted Republicans controlling 240 seats after the 2010 election.

6. If we assume that re-districting efforts of the 2010 census yield a situation no worse for Democrats than the 2000's re-districting, they need about 51% of the two-party vote for Congress to have a 50/50 chance of gaining control. If Republicans win the national vote by 2% (as suggested by their smallest lead on the likely voter models), they win about 232 seats. Note Republicans will likely do slightly better than these conversions suggest.

So what does all of this stuff mean? Be careful of reading too much into the generic ballot this year, but also that Republicans will likely do better in seat count than the national vote suggests.

Senate Analysis This Early?

2011-04-09T00:42:00.000-07:00

As the polls and expert analysis for Senate races in 2012 begin to come in fast and furious, one question we all should be asking is "do the polls and analysis actually mean anything?"

To help answer this question, let us first look at polling data from early on in the last Presidential year Senate campaign (2008). As we will see, one should proceed with caution when examining Senate polling data this early in the cycle.

In 2008, Democrats gained 8 Senate seats from Republicans. By the end of 2007, only 12 eventual match-ups had been polled (according to Pollster.com). I have taken an average of all the polls for 2007 in these races. I have converted the Democratic and Republican percentages to two-way (i.e. allocated undecided and third party votes to the two major candidates proportional to their initial unaltered percentage).

Of the 12 match-ups polled, the average error between the early polls and actual margin was 10.9%, while the median error was 8.3%. That is about double the average pollster error seen in Senate polls in the final 21 days of the campaign from 1998-2008.

More interesting is the direction of the error. In the Democratic year that 2008 became, Democratic candidates over-performed their early projected total by an average of a little over 4.1%.

Of the 12 races polled in 2007, Democrats over-performed in 9 of them. Of the 9 over-performances, Democrats actually won 3 (Minnesota, North Carolina, and Oregon) races they were polled to lose. The early polling correctly projected that they would pick up 4 (Colorado, New Hampshire, New Mexico, and Virginia) other seats. The 8th seat pick-up, Alaska, was not polled this early in the cycle.

What about non-partisan analysis? Non-partisan analysis this early in the cycle in 2006 and 2008 (2010) under-predicted Democratic (Republican) gains.

At this point in 2006, the invaluable Cook Political Report placed 9 previously held Democratic seats in the likely, lean, or toss-up column, while only placing 6 Republican seats in these designations. Democrats would go on to gain 6 seats from Republicans. 5 (Missouri, Ohio, Pennsylvania, Rhode Island and Virginia) of those 6 were in either likely, lean, or toss-up column at this point. Only Montana proved to be a surprise.

In 2008, Cook had 5 Democratic seats and 6 Republican seats in three aforementioned columns. Of the 8 seats Democrats eventually gained, only Colorado, Minnesota, New Hampshire, and North Carolina were listed in one of the three possibly competitive categories.

2010, similarly, broke differently than early analysis pegged. Cook put 8 Democratic seats in the three competitive categories, while putting 9 (including later party switcher Arlen Specter and 2010 loser) Republican seats. Republicans would take 6 seats in the 2010 elections. 3 (North Dakota, Illinois, and Wisconsin) of these were listed as competitive at this point in 2010, while 2 (Arkansas and Indiana) were solidly Democratic.

What does this mean for 2012?

First, as the Cook Political Report figures suggest, there are going to be some seats that may not be viewed as competitive at this point that could become competitive and vice-versa. Some of these newly competitive seats may be due to retirement (see Indiana 2010), while others may just be campaigns evolving over time (see Oregon 2008).

Second, very early polling and analysis under-forecasted the waves that would occur in the following year. Intrade (and my own preliminary Senate model) indicate that Republicans are the ones who will benefit from any 2012 wave.

Right now, Cook has an amazing 7 Democratic seats in the toss-up column (more so than any of the three prior elections at this point). 13 Democratic seats overall are in the competitive columns, while only 5 Republican seats are. If Republicans do any better than they are currently projected to in either polling or expert opinion, then it could be a long Election Night for Democrats.

That said, the general lack of correlation between early expert analysis and final results and 2008 polling errors should read "caution Will Roger". Nothing in the Senate is set in stone. Still, I would put early money on a good Republican night.

Give me Obama's stats, not anecdotes

2011-03-27T01:42:00.001-07:00

President Obama may or may not be re-elected in 2012 (models indicate a close vote is in store), but I have to take some issue with a recent piece by Republican pollster David Hill. He claims that "five items point to Obama loss", and I pretty much disagree with each point.

1. Hill posits that Obama is not likable. While I feel that most elections are not decided on the likability factor, I think the American public likes Obama. For one thing, 83% of the public found Obama "likable" in a January GfK poll. Hill brings up some facts about Obama's strict diet (among other things) that may or may not be true, but I think the actual polls speak for themselves.

Obama's favorables (which Hill probably believes is not the best measure of "likability") run ahead of his approval ratings. In fact, his aggregate favorable rating has never dropped below his aggregate unfavorable rating, while the same cannot be said for his approval/disapproval.

There is a good 5% (+/- a few % depending on the poll) who hold a favorable

view of Obama, but do not think he is doing a good job. That type of spread between favorability and approval is pretty much equal to what "likable" President Bush saw throughout most of his Presidency.

2. Hill submits that Obama does not have support from his base. While I could tell you anecdotally that my roommate Derek (a diehard Obama supporter) still loves Obama, you're better informed by knowing Obama's approval among Democrats is actually above what either of the last two Democratic Presidents (Carter and Clinton) had in the middle of their first term.

His current ~80% is a stunning ~10% above Clinton's and ~30% above Carter's. How is that not getting support from your base? If anything Obama's problem is that he is too polarizing to those outside of his base.

3. Hill puts forth the belief that Obama has had no "noteworthy" accomplishments. This type of question is subjective (and one I am not in a position to answer). What I can say is that I am not sure it even matters. If these big "accomplishments" matter in the end, then why can economic models do such a fine job of explaining past Presidential election results? The only error the famous Hibbs model had was in a year (2000) where the incumbent could not run for re-election. If the economy does improve (see point 5), then I am sure pollsters will begin to hear "the economic recovery" as the number 1 accomplishment for Obama.

4. Hill argues that voters do not feel "personally connected" with Obama. I have not seen any polling to suggest that voters actually feel this way, nor that it actually matters. Hill may have some insider information that he could share with us, but fact is that the aforementioned GfK also found that 61% of Americans said that Obama "in touch with ordinary Americans". That seems, to me anyway, like Americans feel they have a personal connection with the President.

5. Hill points to the famous "are you better off than you are four years" question that Carter failed and says Obama is failing it too. I actually think Hill and I agree on this point. Obama would lose based on economic performance through this point in his Presidency. Luckily for Obama, the economy should improve (see these forecasts from the Philadelphia Fed and Wells Fargo) on a number of measures (GDP, Real Disposable Income, and Unemployment) to the point that Obama is at least a 50/50 proposition to be re-elected.

To reiterate, Hill may very well end being correct: Obama may lose in 2012. However, none of the evidence he presents is very convincing.

Is Your House (model) On Order?

2011-03-24T01:54:00.000-07:00

Earlier this evening, Nate Silver posted a response to my House model. In a sentence, I agree with his basic finding (like many of his). In fact, I made it a week ago in my comment section. My model simply will not work for 1948. It has a difficult time for any year prior to 1948.

Issues such as those raised in Nate's piece about coding often plague political scientists. When does a new political era begin? What constitutes an aggressive war? I do not pretend to know the answers. I only hope to know them.

I have made my judgement about 1952 beginning a new political era that continues to this day. Nate clearly disagrees with that belief. My argument is pretty simple. World War II along with the Great Depression forever changed the political landscape of our country. Some might argue that then I should include 1948 in my model, as it is post World War II. My postulation is that World War II played a role in the 1948 election in ways we cannot model.

Truman won re-election despite having negative yearly growth in real disposable personal income per capita in the first three years of his term (before a recovery in the fourth year). While quarterly disposable personal income data was not published for the first half of Truman's term, my guess is that Douglas Hibbs' quarterly growth real disposable personal income per capita based Presidential model (which Nate and I agree is solid) would have difficulty handling the Truman election.

We know for instance that Ray Fair (who has his own economic Presidential model) had to make ad-hoc adjustments to the all the elections in which the President's term (including 1948) encompassed any part of America's involvement in World War I or World War II. My model is not immune to this World War problem, but considering we have not since had a World War and will probably not have one for a long while, I am not particularly worried for future forecasting.

Another issue that Nate finds fault with is my use of the war variable, which my dataset codes as true for 1976 and 2008. As I noted in my piece, this variable is merely a dummification (not a word, I know) of Douglas Hibbs' Fatality variable. I apply the dummy only to years in which the Majority party in the House differs from the party that controls the Presidency. To answer Nate's question, Libya will count as a war in my book if Douglas Hibbs counts it in his.

A fair question to ask is why does this variable not apply to years in which the party in the White House is the same as the Majority party in the House. Note that including it does not affect my findings. Why? The answer is there is another variable that encompasses the war variable in years in which the President's party is the same of the Majority party.

As Hibbs' model illustrates, Presidential vote (which is my model's main variable for predicting House seats in years in which the Majority party is the same as the President's) is mostly a function of the quarterly real disposable personal income per capita growth and military fatalities. Thus, it would make sense (to me anyway) that the model warps out any effect my war variable would have when the Presidential vote variable is in effect. The question that should be asked is why does economic growth not matter in House elections in which the Majority party differs from that of the President.

Of course, coding of wars is always interesting to deliberate. Hibbs does not count any of the fatalities under Nixon's first term (and Obama's first term for Afghanistan and Iraq) against him because he "inherited" the war from Johnson. I happen to agree with Hibbs, but I know some who do not. The point is that a "right" answer in coding is not always apparent.

I also want to address a few other points.

First, Nate and I have "known" each other (at least in the blogosphere) for about a year now. During that time, I have felt (and continue to feel) that we have had a respectful and amicable relationship. I have "critiqued" some of his work (see here), and he has returned the favor. This exchange is healthy for me (as I hope it is for Nate), and more than that, it is important for the reader. Without this back and forth, anyone could post a poor model at a whim, and we would be none-the-wiser.

Second, part of the reason that it was so easy for Nate (or anyone) to check my work is because I post my datasets for everything I do online. If I do not, I tell readers (and I really do mean it) that they can email me. I wish more online political writers would post their datasets. It is the right thing to do.

Third, margins of error can be tricky for any out-of-data forecast. I do not pretend to know that my +/- 10 will definitely turn out to be right (e.g. it might actually be closer to +/-12), but it is based on something concrete (the existing dataset). I would rather make my margin of error that way then create some large arbitrary error to cover my behind. Some can point to this simple Gallup house model as an example where the margin of error (+/- 11 seats) of the in-dataset did not quite cover the spread in 2010 because it ended up being off by 13 seats. Unlike that dataset however, my model is not reliant on some measure (a Gallup poll) that is subject to additional error.

Finally, a wise man once said, "to be clear, accuracy ought to be the paradigm here. We're not trying to prove or disprove anything to an academic certain degree of certitude; we're trying to make a forecast... [My] model may be wrong, but I'd rather fail by being too ambitious than too stubborn."

I agree with him.

EDIT: One more point on degrees of freedom and number of cases. If you take the full model, you have 6 variables on 15 cases. If you take the not full model (i.e. only having the interaction variables without the original), you have 4 variables on 15 cases. I'm in nowhere comparing model to Douglas Hibbs' midterm House model, but there you had 3 variables on 15 cases. The Gallup model discussed above also had 3 on 15.

If you were to breakdown my model to the years in which the Majority party was different the President's party (which is basically my argument... that we have two types of House election is Presidential years since 1952), you would get the same result with 10 observations but only 2 variables (war and previous seat count). I have seen some suggestions that a solid model should have at least 10 to 20 times as many observations as variables. Most academic Congressional models and mine fail this test.

The real question, however, is whether statistically the model is too "tight" a fit. I have made the point that 1952 began a new political era. This belief has worked through 2008. In fact, as you can see in the comment section of my original post, the model did just fine in every year since 1952. If 2012 ends up being different from the '52-'08 dataset, the model will meet Reaper, Grim.

However, I really do not believe 12 will mean the model's death. Good aggregate based modeling (such as Hibbs and even the Gallup model that Nate does not like) did just fine in 2010. Most of them were inside their margin of error, and even the Gallup model missed the margin of error by just a few seats (or less than three standard errors). I will be making a post that the generic ballot (at least at this point) also supports my model.

The true test will be in a year and half. If it is clear my model is going down the drain before then, I will jump off the bus faster than fans of the Yankees when they are heading for last. And as those who know me can attest, I will come back here and admit my fault.

Three models say: It's a Tossup in 2012 Prez Race

2011-03-21T07:20:00.000-07:00

In a few weeks, I'll be re-debuting my 2012 fundamentals based Presidential vote prediction model. It is a re-specification of Douglas Hibbs' bread and peace model that if nothing else will make you think. Until that time (and even after it), I want to throw out some caution on any long term Presidential fundamental (i.e. non-poll based) forecast (including my own).

Currently, three solid models* are out and about in the blogosphere (with many more on a Google Scholar near you).

-The aforementioned Douglas Hibbs Bread and Peace model that utilizes quarter-to-quarter growth rates in real disposable personal income per capita (dpipc) expressed annually and fatalities in "unprovoked, hostile deployment of American armed forces in foreign conflict" over the term**.

-Ray Fair's model that employs real per capita GDP quarter-to-quarter growth rates expressed annually in the 13-15th quarter of a term, GDP deflator quarter-to-quarter growth rates expressed annually over the 1st-15th quarter of the term, and number of quarters in the first 15 quarters of the Presidential term where the real per capita GDP growth expressed annually rate is above 3.2.

-Alan Abramowitz's model that uses 2nd quarter election year real GDP growth rate expressed annually, election year's June Presidential net approval (as measured by Gallup), and a dummy variable for number of terms the incumbent party has been in the White House.

Capitalizing on available economic forecasts from Wells Fargo (as well as from Ray Fair for his own prognostication), we can get an idea of what each of these models would call for if the election were today.

Hibbs' has the election teetering on the edge with Obama winning 50.05%-49.95% in the two-major-party vote; Fair's has Obama taking 52.5% of the two-major-party vote (down from 55.9% in his November prediction); while, Abramowitz's sees Obama winning between 53-54%*** of the two-major-party vote.

All three estimates for Obama's vote share are pretty much within each model's (at least +/- 3.5%) margin of error, so the spread between them is more than understandable. Therein lies part of the problem, however.

These models (and my own) are not built for close elections. Yes, sometimes they will do very well in close (53.5% or less for the winner) races (such as in 2004), but sometimes they will have large errors (2000 for Hibbs and Abramowitz and 1992 for Fair) where their winner for the popular vote received at least 3% less of the vote than forecasted. Considering all the models are giving Obama a small majority of the vote, the words "margin of error" cannot be repeated enough.

What about difficulties specific to each model?

Solid as a rock for post-election explanation of election results, the Hibbs model (as with mine) is sensitive to very small changes in real dpipc.

For example, the Hibbs' model utilizing an economic forecast from two months ago would have had Obama winning 50.3%-49.7%. A small change to be sure, but the only difference in Wells Fargo's growth rate of real dpipc forecast was that it was 0.1%-0.2% higher in some quarters (e.g. 2011's quarter 1 forecast of 4.0% to 3.9% now).

Still not worried? September 2010's to November 2010's quarter 1 of 2011 Wells Fargo's forecast for real dpipc changed by 0.6%****.

Thus, it is more than possible that slight (to not so-slight) revisions, especially in the long-term, will be made to the real dpipc forecasts. These possible changes make any Presidential forecast based of this measure of economic strength difficult.

The Fair model is less sensitive to slight shifts to two of its main economic variables (GDP deflator and real per capita GDP). Its greatest issue (besides ad-hoc adjustments) is the use of an interval variable to express the number of quarters during the first 15 quarters of a Presidential term in which real per capita GDP growth at an annual rate is greater than 3.2%.

Only one such quarter has occurred during Obama's Presidency, but Fair is predicting that there will be 3 more. My issue with this estimation is that I simply do not believe it will happen.

When we adjust Wells Fargo's real GDP growth rate to real per capita***** GDP (which basically takes about 1% off growth), no quarter-to-quarter growth at an annual rate for the rest of Obama's term climbs higher than 2.2%. Even the more optimistic Philadelphia Fed consensus estimate (when adjusted to per capita rates) gets only to about 2.7% in the best quarter.

And if only one quarter during Obama's term has real per capita growth rate above 3.2%, Fair's model actually shows the Republican candidate winning the election with 50.5% of the two-major-party vote!

The Abramowitz model, on the other hand, actually lends itself the most to a long range forecast, despite its use of Presidential approval in June of election year. Why?

Presidential approval is only a small part of the model with a coefficient for the variable of only 0.107; holding all other variable values constant, we should only expect about a 2% difference in Obama's projected vote percentage if his net approval is -10% vs. +10% (a 20% swing).

The model's utilization of real GDP is also an asset as the forecast for 2012's second quarter real GDP rate expressed annually has held steady at right around 3%, according to Wells Fargo. Even if the actual growth rate ended up being 2%, Obama would only lose about 0.5% off his estimated vote percentage.

The possible problem with the Abramowitz model is that (as Abramowitz, himself, points out) in each of the last four Presidential elections the model has over-predicted the vote of the incumbent candidate by at least 1.85%.

If Obama's net June approval ends up at -5% (very possible), and real GDP growth expressed annually for the second quarter is at 2.9% (per Wells Fargo), a 1.85% error in the negative direction pushes Obama under 51%. Even so, I would say that Abramowitz's model still indicates that Obama is the slight frontrunner.

So what can we take away from this analysis?

Two models based entirely off the fundamentals have the election shaping up to be a 50/50 affair where stump speeches in suburbs of Denver could make the difference. The other model (with a poll variable) contends that Obama is the slight favorite.

I think we can say that the 2012 election is going to be very tight.

Notes:

*Please visit each writer's link to get full details on their models. They do better job of explaining them than I ever could. If you're still confused, just ask me a question.

**Note that a President gets a one term grace period for wars that his party did not start, so this value is currently 0 for 2012.

***Given Obama's net approval is between -3% and +6%.

****Both of these forecasts were before the extension of the tax breaks to those in the upper income brackets. Evidence suggests this extension may have negatively impacted real dpipc growth and thus had a negative effect Obama's changes of re-election.

***** Assuming the population grows by 670,000 each quarter. This number was achieved using the average quarter-to-quarter growth during the four quarters of 2010.

Republicans to maintain control of the House in 2012

2011-03-16T19:36:00.000-07:00

Unless a historic event occurs, Republicans will still be in control of the House of Representatives after the 2012 election. How can I be so confident even when House re-districting is still occurring?

It turns out that the difference between House election results during Presidential election years are very well accounted for by fundamentals variables*. What I mean by "fundamentals" is simply variables that we know (or can reasonably predict) before the election and do not include polling data. In the 15 Presidential year House elections since 1952 (a common post-war cutoff for political science), these variables** include

-Percentage of seats won by the majority party in the last election. The majority party in the House wins more seats when it previously held more seats. In 2012, this variable is 55.6 because the Republicans won 242 seats out of 435 in the 2010 House elections.

-A dummy (1 or 0) for whether the majority party in the House is the same as in control of the Presidency. In 2012, this variable is 0 because the House is controlled by a different party than the White House.

-The percentage of the vote the party in control of the White House wins in the Presidential election during years in which the majority party in the House is the same as in control of the Presidency. Not surprisingly, when the party controlling both is the same, the majority party in the House gains more seats when its candidate for President wins a higher percentage of the vote. Surprisingly, Presidential vote has little relation to the House election in years when the House and Presidency are controlled by different parties. In 2012, this variable is 0 because the House is controlled by a different party than the Presidency.

-A dummy (1 or 0) that is 1 when the party in control of the White House is different from the majority party in the House, and the President has started or maintained for more than one term an "unprovoked, hostile deployment of American armed forces in foreign conflict" (as defined by Douglas Hibbs) which has resulted in at least 1 fatality during the past term. Interestingly, as if to penalize the party in the White House, the majority party in the House wins more seats when this variable is true (like in 2008). In 2012, this variable is 0 (false) because the Iraq War was started by a Republican President (Bush), and Democratic President (Obama) has not continued it for more than one term.

In simple linear regression equation form, the model reads for 2012,

Percentage of seats won by the current Majority Party (54.8%) = Coefficient for Previous Seat Share (.67) * Previous Seat Share of the Previous Vote share (55.6) + Constant (17.3).

The model is therefore predicting that the Republicans will win 238 seats, more than enough for them to maintain their majority.

We must ask how accurate is this model in explaining past results?

The answer is very accurate. The model is able to account for 95.9% of the difference in the results of the 15 Presidential year House elections since 1952.

What about the chance of error? The root-mean-squared-error (a statistic that measures errors in estimate and penalizes for larger errors) is 1.1% of seat share, which given our small sample size (15) corresponds to a margin of error at 95% confidence of +/- 2.3% seats in Congress. The largest error (and the only with an error of greater than 1%) in our set is the 1988 election, which the model misfit by 2.5% (or 11 seats). For 2012, our margin of error indicates that Republicans winning as many as 248 seats and as few as 228 is a reasonable expectation.

Of course, none of these findings are useful for 2012, unless we know how well the model not only explains but also would have predicted past House elections. To do so, we take out a given election from the model and re-run the regression. The results give us confidence that the 2012 estimate should be a good one. In 2008, the model called for the Democrats to win 252 seats, when they ended up winning 257 seats. In 2004 (thus we take out 2004 and 2008 from our dataset), the model would have forecasted Republicans to win 236 seats, when they won 232.

And what about the last time a Presidential election took place directly after re-districting (1992)? Eliminating 1992 from the dataset and re-estimating the model, we find that the model would have projected the majority party (Democrats) to win 254 seats, when they took 258. Thus, while this year's re-districting may cause the model problems, past history does not indicate that it will.

Given this past accuracy and current Republican majority, it looks like it will still be Speaker Boehner come January 2013.

Notes:

**As always, email if you are interested in the dataset or have questions.

*To those statistic masters out there, yes it would be proper to include both "original" independent variables that make-up the interaction terms (war and Presidential vote). They were left out to preserve a reasonably low number of variables given the low number of observations as well as for simplicity of explanation. There is no statistically significant difference in the accuracy of the "full" model to explain past results.

Britain's 2011 AV referendum showdown: Should Britons worry about hung parliaments?

2010-12-04T19:58:00.000-08:00

Can Mexican exit polls detect fraud when American exit polls cannot? A study of the different roles exit polls play in these two countries

2010-12-04T15:34:00.000-08:00

Why do British and American exit polls differ?

2010-12-04T14:42:00.000-08:00

Welcome

2010-10-21T22:33:00.000-07:00

I see Dartmouth was kind enough to link to this blog (again). I am currently in the midst of preparing for the Midterm Election conference and have been caught a little unprepared for linkage.

In case you're wondering what I am currently projecting for the midterm elections, I have the following projection in the House based solely off projections of the national house vote: Republicans will gain 57.5 seats +/- 7.5. If Republicans pick up exactly 57.5 (57 or 58) seats, they will hold 235.5 (235 or 236) seats in the 112th Congress. Democrats would control 199.5 (199 or 200) seats and be forced back into minority status. This projection has not changed since my first estimate all the way back in February.

Of course, the House of Representatives is actually made up 435 single member districts. Specific seat estimates (e.g. will Democrat Anne Kuster or Republican Charlie Bass be the representative from the seat that represents Dartmouth in Congress) will be available by mid-afternoon. To forecast the outcome in every district, I've been working with the folks over at Stochastic Democracy. Developed in July, our in-depth model utilizes not only the projection of the national house vote, but district-by-district polls, Cook Political Report Ratings (visit their site as my friend Dave Wasserman has forgotten more about the House than I will ever know), and Cook's Partisan Voter Index to predict races.

Projections for Senatorial and Gubernatorial races will also come this afternoon. As a preview, I can tell you that Democrats should hold onto the Senate, but lose a number of seats. Republicans should pick up at least 2 (and probably 4 or more) Gubernatorial mansions, enough to control a major of Governorships nationwide.

Campaigns have NOT made a difference in 2010

2010-10-10T03:33:00.000-07:00

"GOP Romps to Victory; Captures Majority in both House & Senate" is a possible November 3rd headline that is starting to look like a real possibility. In the case of the House, a majority looks more like a probably. The Democratic argument against such an outcome goes something like "our campaigns are just kicking into high gear, and we will bring the argument to the people." If the headline holds true, Republicans are likely to echo this platitude after Election Day "we ran good campaigns, people heard our arguments, and we won!" The truth is that both of these generic statements are mostly false. While some campaigns have made a difference (e.g. Dick Blumenthal's large military record exaggerations, and the emergence of Christine O'Donnell as the Republican nominee in Delaware), the overall nature of the upcoming Republican romp was determined long ago.

In February, I wrote an article entitled "Republican Blizzard on the Generic Ballot". Using a regression based on past midterm elections from a paper by Joseph Bafumi, Bob Erikson, and Chris Wlezien. I wrote that the Republican position on the generic ballot was likely to improve throughout the year. That belief has not borne out.

In fact, the generic ballot has remained amazingly consistent when controlling for pollster and sample population (likely voters to likely voters and registered voters to registered voters). While certain pollsters (e.g. Gallup and Democracy Corp.) have shown movement towards the Republicans, other pollsters [ABC / Washington Post (TNS) and PPP] have actually seen Democrats gain a couple of points. In fact, I believe any difference between these pollsters February/March numbers and September/October numbers can be attributed to the margin of error. What has happened is that pollsters [e.g. ABC / Washington Post and CNN (Opinion Research)] have switched their sample populations from registered voters to likely voters. The Republican surge talk about by some is merely the change by pollsters from registered voters to likely voters, which takes into high voter enthusiasm among Republicans.

What about district-by-district polling? There certainly has been a movement towards Republicans in a number of races. Polls in Republican leaning districts like Alabama 2nd and Kentucky 6th have recently shown Republican challengers gain their first leads of the year. The Cook Political Report and Rothenberg Political Report has also seen many seats shift towards the Republicans. While some of these movements can be attributed to well run campaigns, many of the Republican advances simply have to do with the fact that challengers are becoming as well known as incumbents. In both AL-2 and KY-6, the polls as well as expert ratings are moving towards the generic ballot in these districts. In other words, district-by-district indicators are lagging behind national indicators, but are beginning to catch-up to them.

Not surprisingly, many districts that feature rematches of prior match-ups have not surprisingly remained relatively stable. In Maryland's 1st, incumbent Democrat Frank Kratovil and Republican Andy Harris have remained within a few points of each other this entire year. In Pennsylvania's 12th, incumbent Mark Critz’s numbers in Democratic and Republican polls have matched his pre-special election numbers against Republican Tim Burns. In Florida's 22nd, incumbent Democrat Ron Klein has consistently lead in the Democratic sponsored polls, while Republican polls have shown Republican Andy Harris slightly ahead. Although this is just a small smattering, similar trends have occurred in other districts.

Similar to these House districts with two very well known candidates, US Senate races have shown amazing consistency. Back in February, I wrote

I am by no means saying that the Republicans will take back the Senate; however, the polling in conjunction with past results indicate that it [is] not that long of a shot that they do. Democratic candidates seem to be consistently weak over the last six months, and the Republicans seem to be moving into a stronger position in the last two months.

Looking at Senate races with at least one poll in February / March that the Cook Political Report currently rates in at least a somewhat competitive category (i.e. not solid), Republicans have maintained their edge. In these 17 states, only one state, Washington, has the leader in the polls at the beginning year (Dino Rossi) lose her/his lead. Rossi's deficit was small then, as his lead is now, and there are signs Rossi is moving back in the lead.

Most races have only moved slightly. The average absolute change between February / March and now is only 3.92%, which is largely driven by Connecticut where the once invisible Dick Blumenthal has seen his lead cut by two-thirds. The median (which somewhat controls for outliers like Connecticut) absolute change is only 2.13%. That means that despite all the advertising and baby kissing, most Senate campaigns like those in the House have made only the smallest of dents among the electorate.

As to the question of whether there has been a national swing to either the Democrats or Republicans, the answer is also no. The average change in Republican margin between February / March and now polls is a tiny 1.74%, which again is driven mostly by Connecticut. The median (not absolute) difference is a miniscule 0.34%. While some Democrats and Republicans have gained ground, neither Republicans nor the Democrats have gained ground in the aggregate. There has been no national tide.

Does all of this mean that campaigns did not make a difference in either House or Senate elections this year? Not necessarily. For one thing, we still have a little over three weeks until the election. It is possible that races swing dramatically between now and Election Day, though I wouldn't bet on it. More to the point, one never knows when a race will turn into a Connecticut. Candidates make mistakes, and sometimes strange things happen. In addition, this sort of analysis cannot possibly get down to the nitty gritty 1% or 2% differences that sometimes make or break elections.

Still, at least in 2010, most outcomes have seemingly been predetermined. Campaigns have not made a difference, and voters made up their minds a long time ago.

Edit: I want to make it clear that I am not saying candidates should not campaign. If one candidate campaigned and her/his opponent did not, it would obviously make a difference. An inherent assumption here is that both candidates have at least a bare boned campaign and have the necessary funds to at least minimally compete. Pretty much all House and Senate races meet this criteria.

Notes

All polls were taken from Pollster.com.
Senate races in Alaska, Delaware, and West Virginia were not included in the numerical analysis because no polls of the current match-ups were taken in the February to March period.
In Florida, I test the difference between Marco Rubio (R) and his nearest competitor Charlie Crist (NPA).
No February to March polls tested the Senate match-up between Rob Johnson (R) and Sen. Russ Feingold (D) in Wisconsin. I have substituted the Tommy Thompson (R) vs. Sen. Russ Feingold (D) match-up, as I believe Thompson and Johnson are similarly well-funded and non-extreme Republican candidates. Not including Wisconsin in my numerical analysis does not make a significant difference in my ultimate findings.

Why the Democrats will lose the House

2010-10-06T22:36:00.000-07:00

Back in February, I used a regression equation from Joseph Bafumi, Bob Erikson, and Chris Wlezien to show the Democrats were likely to lose the House. Although, the seasons have changed, the news remains the same. Here's why:

1. The Republicans only need to be leading by about 3% in the national popular vote to regain the House. The current Pollster.com aggregate of the national House ballot has Republicans teetering on the edge with a lead of about 3.5%. That lead would lead to a cliffhanger on Election Night (as Republicans need 40 seats for the majority), but it does not tell the whole story.

2. Regardless of anything else, a refined Bafumi et al. regression for the current time period for most polls (60-31 days before the Election) suggests that the party of the President is destined to better in the House vote that the September polls indicate. This makes sense as voters take into account information including the wish to bring an "ideological balance" to Washington. Voters want to check the power of the President. This wish for a check translates a Republican lead of 3.5% in September to a little greater than 5% lead in October. Now, there is an error associated with this estimate such that about 20% of the time Republicans would actually do worse than the 3.5% lead they hold currently. Still, a look at the chart below tells us that those errors happen when the non-Presidential party already holds a large House majority (1982) and the need for the aforementioned balancing is minimized; or, when the President sports a high approval rating in the mid 60's (1986 and 1998). In 2010, the party not in the White House is in the minority, and the President's approval rating is mired in the mid 40's. Thus, neither type of error seems likely to occur in this election.

3. In addition to the natural inclination towards the party out of power, Gallup finds Republicans are currently much more enthusiastic about voting than Democrats. As I noted in October, this current edge in enthusiasm is pretty much unprecedented. Why is this important? It is true that plenty of un-enthusiastic voters will cast ballots on Election Day, but a relationship between enthusiasm and likelihood to vote does exist. Since 1994, years in which Republicans held a large lead in enthusiasm (like this one) meant that they did 7-11% better among likely voters than among the larger population of registered voters. That's why the current Gallup national Congressional vote estimate of a 3% Republican lead among registered voters is actually a double digit lead (13 or 18% depending on the turnout model). The problem for Democrats is that over the past month the best have managed on Gallup's national congressional ballot is a 1% lead. Even in the best case scenario, Republicans should win the national vote by 6% and win about 228 seats in Congress.

4. What about district-by-district? The vaunted Cook Political Report finds that Democrats hold 81 seats that would be considered vulnerable, while Republicans only hold 7. An additional 5 Democratic held seats are pretty much guaranteed to go Republican. This 79 seat edge in possible seat turnover is significantly larger than in 1994 when it was only 54 seats in the final Cook ratings (which equaled the Republican seat gain). As James Campbell of the University of Buffalo found, the party who wins in a "wave election" such as 2010 is turning out to be wins at least 66% of the seats rated toss-up and even those rated leaning towards the other party. In addition, the winning party pretty much win all of the seats leaning in its direction. Considering that 73 seats lean Democratic or are tossup, Republicans should win 49 of these seats. Add that to the 10 Democratic seats that are leaning Republican or likely Republican, and Republicans could gain up to 59 seats. I should point out that Campbell's accuracy measure is from Labor Day, while I'm Cook ratings from early October. Even so, Campbell found that Republicans would gain about 51 seats using Cook's 2010 Labor Day statistics. My higher seat gain makes sense considering the Republican edge in competitive edge is now 10 seats higher than on Labor Day.

Finally, I should explain why I do not look at district polls. The fact of the matter is that they generally are not that accurate. Whether it be RT-Stategies vailient multi-district effort in 2006 or a tabulation of other publicly made available House polls from 1998-2008, house polls have a high degree of error associated with them. In effort to account for this inaccuracy (especially this far from an election), one must considerably expand the cone of uncertainty [like the forecasts at Stochastic Democracy (to which I contribute) or FiveThirtyEight]. The result is very wide confidence intervals (or large margins of error). In my mind, these intervals are far too wide in the aggregate. Yes, any one seat change has a high degree of uncertainty, but the aggregate amount of seat change is not (see points 2-4 above). Finally, both of these rely upon what is called a random walk to estimate a forecast for the generic ballot (which helps aide district-by-district estimates because of poor past polling results). FiveThirtyEight also uses a likely voter adjustment (though I believe it underestimates the eventual Republican edge) to account for point 3. The problem with these adjustments is that a random walk does not take into account point 2. We shouldn't expect a random walk, but rather a walk that aides Republicans in the national House vote.

Thus to me, it is clear that the Republicans on their way to gaining back a majority. The exact seat gain will remain uncertain until right before the election. The present Cook Political Report ratings and generic ballot suggests that Republicans will gain 50+ seats, which matches my prior February estimate. The one thing I am certain of is that something quite unusual would have to happen for Democrats to hold onto the House after November 2nd.

It's not a push poll: version 5,024

2010-08-26T23:46:00.001-07:00

The words "public" and "house poll" are usually not put together in a sentence. Often, those trying to dissect individual house races are forced to rely upon polls done by partisan interest groups, political parties, or candidates. One possible problem with partisan polls is that the sponsors may try to ask questions and present the results in a manner most benefitting the client. In Kentucky's third district, Republican Todd Lally earlier this summer accused his opponent's (Rep. John Yarmuth) pollster Cooper & Secrest of "cultivat[ing] a reputation as Democratic push-pollster in a number of districts across the country. Most recently, Coopers & Secrest was accused of attacking Gov. Bobby Jindal in 2007 for being Catholic. Such allegations have surfaced in other areas as well. In Ohio, Coopers & Secrest alleged in their polling that Republican Martin Hoke was a member of a cult." Lally "call[ed] on John Yarmuth to immediately make all polling scripts from his campaign available to the public." While Lally has reason to attack a poll that showed him trailing by 26%, I wondered whether Cooper & Secrest really had previously engaged in push polling.

A search of the Google archives reveals no news source actually reporting a push poll by Cooper & Secrest in the 2007 Louisiana Gubernatorial Election. The only article mentioning Cooper & Secrest and push poll in that election is a campaign memo released by the Jindal campaign. The push poll question allegedly asked by Cooper & Secrest was "Bobby Jindal wrote articles for a Catholic magazine against Protestants...Would this lead you to believe that he, as governor, would prevent Protestants from practicing their religion?" While that may seem like a "smear" (and I most definitely do not condone spreading even half-truths), it turned out that the substance of the question was at least legitimate enough for two major Democratic candidates for governor to not call for the removal of an advertisement telling voters about Jindal's articles. In addition, a 2007 Associated Press article did not say the ad was untrue, only that Jindal's position was more "nuanced than the ad suggests". But beyond whether or not Cooper & Secrest was spreading a misrepresentation, no one was able to determine how many voters Cooper & Secrest called with its question. Why is that important?

As Mark Blumenthal has pointed out, push polls are meant to reach tens or even hundreds of thousands of voters. Stu Rothenberg notes "hiring a firm to do 500 or 800 advocacy calls in a Congressional district or state — where thousands or tens of thousands or even millions of voters are going to go to the polls — would be idiotic, since it wouldn’t accomplish what advocacy calls are trying to do, which is change opinion." The problem with the charge that Cooper & Secrest conducted a push poll is that they use live interviewers, who would take forever to reach 10,000 people in a Congressional district. Something especially difficult considering that Cooper & Secrest asked numerous questions, according to the Jindal memo. The question seemed to be designed to test an attack for future use in campaign messaging, which would make a lot of sense considering that questioning Jindal's position on faith was used in campaign ads. In other words, there's little proof that this inquiry meets any definition of a "push poll".

The case of Martin Hoke is a little more interesting, but far more clear. For those that do not remember, Hoke was a Republican Congressman in Ohio in the mid 90's who in his youth practiced Sikhism. Cooper & Secrest ran a poll in which one question mentioned that Hoke was "part of religious cult where he wore a turban, a beard, and had an assumed name." No sane person would call Sikhism a cult, so the use of that word was untruthful. The issue with calling this a push poll is that the Washington Monthly article in which the story of the Hoke (as well as few other instances of possible Cooper & Secrest) "push poll(s)" appeared reveals that only 400 to 500 received the poll. As mentioned, no person wanting to push poll calls so few people. Even if Cooper & Secrest was asking "highly misleading" questions, they were not trying to spread misinformation. Rather, it would seem that they were testing messages for a campaign. And while the questions may seem unfair, politics is many times dirty and half truths are spread like wild fire.

The fact of the matter is that there is no proof that Cooper & Secrest has done anything out of the ordinary for a campaign pollster. And in the case of Kentucky's 3rd district, not only was Cooper & Secrest's poll done only among 566 voters, but a public Braun Research poll arrived at the same conclusion as Cooper & Secrest's 20%+ Yarmuth lead.

We could have seen Alaska from our houses

2010-08-26T18:16:00.000-07:00

As Lisa Murkowski fights for her political life in Alaska, the question being thrown around the blogosphere is "could we have seen it coming"? I'll be honest in saying that I made a mistake a political analyst should never make; I took the common political wisdom at its word. Had I (or anyone) investigated the issue at any length, it would have been VERY clear that Joe Miller was closing faster than a Ferrari on a Buick Century. Why?

1. Despite what most in the media are throwing around, there were polls showing Miller gaining ground. This past Tuesday The Wall Street Journals' John Fund (who deserves a heck of a lot of credit for staying on the story) noted "Internal polls taken for her campaign show Fairbanks attorney Joe Miller closing the gap". A publicly released RT Nielson poll conducted a week before the primary among 300 likely voters on behalf of the Tea Party Express showed Murkowski's 37 point edge a month before the primary (in a Dittman Research poll) dropping to only 12 points (47-35). Most in the press seemed to ignore the poll because of the poll's sponsor and a lack of disclosure about the poll's methodology, but it now appears as if the poll was correct.

2. Murkowski's campaign's actions seemed more like a campaign in trouble than a campaign gliding to victory. A day before the primary an article by the Alaska Dispatch's Amanda Coyne noted "things seem to be changing... there seems to be something in the way that [Murkowski's] responding [to Miller's advertisements] that bespeaks nervousness." The campaign was releasing advertisement on top of advertisement, as if nothing they threw at Miller was sticking. As Coyne pointed out "instead of using her air time to take on Miller for wanting to privatize social security and Medicare, say, or questioning how his fiscal hawkishness will jive with Alaska's taste for pork, or even talking about the things she has done in D.C., she's spending it defending herself..."

3. More than a few Alaskan insiders believed that Murkowski was on the ropes. The aforementioned Republican Pollster Dave Dittman warned the day before the primary that he would not be "shocked" by a Miller victory. Yet, many in the national press only decided to cite Dittman's final poll, not his final words.

Now, I am not suggesting that Miller victory was evident (as it still is not with votes still being counted), but I would say that if analysts (including myself) had actually done some research we would have thrown out a lot more qualifiers when speaking about an "impending Murkowski victory".

Rasmussen's Senate House Effect

2010-07-12T11:05:00.000-07:00

Whenever I meet a Democrat, and they find out I write for Pollster.com, the first question almost always asked is "what are your thoughts on Rasmussen? They are always biased against the Democrats". On the liberal site Daily Kos, Steve Singiser's nightly political wrap-up packs all Rasmussen polls at the bottom of his posts and mockingly refers to the polls as from the "House of Ras". I also got in on the criticism last month in a post where I pointed out that Rasmussen had higher pro-Republican house effects during important news cycles in 2008.

But does Rasmussen have a large pro-Republican house effect in 2010? Looking at the generic ballot, the answer seems to be absolutely. Using only Rasmussen polling, the Pollster.com aggregate gives the House Republicans a 7.3% advantage as of this writing. Using all other pollsters except for Rasmussen, House Republicans hold only a 0.1% lead. Some of this may be their use of a likely voter model, although it is unlikely that it accounts for all of Rasmussen's difference.

Of course, the generic ballot is only one of the many contests that Rasmussen polls. Rasmussen has accounted for a little over 50% of the legitimate polls conducted for United States Senate races in 2010. Being that Rasmussen has flooded the zone, we must ask whether their Senate polls have had the same sort of bias as their generic ballot. Many liberals would like to believe so, but no one to my knowledge has affirmed or disproved it... until now.

David Shor, a visiting graduate student collaborator at Princeton University, has estimated Rasmussen's house effects in all possible Senate races (his methods are outlined here). Using data supplied and collected by Rasmus Pianowski and me, he found that the difference in house effects between Senate races to be mostly insignificant (or "eerily consistent") . However, the difference between Senate races and the generic ballot was highly significant.

Instead of a 5% pro-Republican bias as seen on the two way generic ballot, the pooled Senate house effect is only 2%. For example, if you encounter a 44% to 36% Republican lead on Rasmussen's generic ballot, it is probably a tie: 44% for the Republicans / (44% for the Republicans + 36% for the Democrats) - 5% House Effect = 50% of the decided vote for the Republicans. On the other hand, a 44% to 36% Republican lead in a Senate race means the Republican is leading: 44% for the Republican / (44% for the Republican + 36% for the Democrat) - 2% House Effect = 53% for the Republican of the decided vote.

Does this variation in generic vs. Senate house effects make a difference in terms of how many seats Republicans would pickup if the election were held today? Shor, who, along with his Stochastic Democracy blog team, Princeton Election Consortium, and me launched a preliminary 2010 Election Projection System, has shown that differing house effects do make a difference in our projections. Instead of the Republicans picking up 3 seats, they are predicted to pick up 5 seats with Missouri and Pennsylvania falling into the Republican column.

It should be pointed out that Rasmussen, like any other pollster, will have outlier polls that will not fit perfectly in with any assigned house effect. Still, when viewing Rasmussen's Senate polls, one should realize that these polls tend to be more pro-Republican than other polls, but not as pro-Republican as Rasmussen's generic ballot polls.

Pat Toomey: Conservative Hero

2010-05-10T09:22:00.000-07:00

Is Pat Toomey too conservative for Pennsylvania?

This weekend in response to a post I made about possible Pennsylvania Senate match-ups Alan Reifman made that assertion "I think Toomey is too far to the right for Pennsylvania." When I saw Reifman's post, I was going to respond with something along the lines of "but Pennsylvanians elected Rick Santorum... twice." But before I did, I decided to contrast Santorum's and Toomey's DW-Nominate scores. These scores allow one to compare how conservative and how liberal a legislator on a -1 to 1 scale with higher positive scores indicating a more conservative record. What I found surprised me.

Using joint House and Senate scaling (to compare across chambers), we find that Pat Toomey (.718) had a considerably more conservative voting record than Rick Santorum (.349). To put that number into context, Lincoln Chafee (the ultimate liberal Republican and now independent) had a DW-Nominate score of .002. The ideological gap between Pat Toomey and Rick Santorum is slightly larger than the gap between Santorum and Chafee. You could argue that Toomey is to Santorum as Santorum was to Chafee.

Still, I wanted to get a better idea of how conservative Toomey voting record was. To do so, I pulled the DW-Nominate score of every United States legislator (House and Senate) since 1981 (see below for why I chose this cutoff). Toomey is further to the right than the GOP caucus as a whole as seen in the percentage histogram below. Indeed, of the 1,520 legislators to receive a DW-Nominate score for their career since 1981, Toomey ranked as the 24th most conservative.

Toomey ranked more conservative than 98.4% of all United States legislators since 1981. He had a more conservative voting record than J.D Hayworth, Jim DeMint, and even Jesse Helms. Toomey did have a more "liberal" voting record than Tom Coburn and Tom Tancredo.

To put it into prospective, Pat Toomey would most likely be the second most conservative Republican in the United States Senate, which would be quite an accomplishment considering Pennsylvania is a moderate state.

Note: The reason I use 1981 as the cutoff is because prior to the 1980's a legislator's liberal-conservative record was also highly correlated with a second dimension of DW-Nominate scores. Since 1981, however, the scores I use cover roughly both economic and social votes.

PA: Sestak Is Stronger in the General Election

2010-05-07T22:00:00.000-07:00

Is Arlen Specter really a stronger general candidate Joe Sestak? It's an argument we've heard from Specter supporters. Unfortunately for Specter fans, I see limited evidence to support their claim.

A quick glance at the Pollster.com aggregates in a Specter vs. Republican Pat Toomey and Sestak vs. Toomey matchup shows that both candidates lose to Pat Toomey. Yes, it is true that Specter only trails by 6.9% (while Sestak trails by 9.1%), but the Sestak vs. Toomey matchup also has a larger pool of undecided voters. Thus, Sestak would have a greater opportunity to pick up voters and close any gap between Toomey and himself. Now you might say, but Specter could potentially pick up the undecided voters in a Specter vs. Toomey matchup. This scenario does not seem likely when we look at the favorability ratings of Specter among the general electorate.

In the last three polls (conducted in early April), more voters rated Specter unfavorably than favorably. In two of the polls, a majority of voters had an unfavorable view of Specter. It is very unlikely that Specter would be able to win under these conditions. On the other hand, Sestak had a higher net favorable than Specter in all polls and in none of the polls did a majority of voters grade Sestak unfavorably. To me, this illustrates that Sestak definitely has a better chance of picking up undecided voters than Specter to have a chance at beating Toomey in November.

Of course, Toomey is the favorite whomever he faces in the general election. He leads in both potential match-ups and has higher favorables than Specter and Sestak in all recent polls.

How IRV screws it up

2010-05-07T20:50:00.000-07:00

To answer a question Patrick Ruffini has, here is how ranking a candidate lower in IRV can help them.

Let's say we have three candidates Reagan, Carter, and Anderson, and we have 17 voters.

Let's say, Patrick Ruffini prefers Reagan to Anderson to Carter. 7 other people also have this preference (which we'll call RAC).

Five people prefer Carter to Reagan to Anderson (which we'll call CRA).

Four people prefer Anderson to Carter to Reagan (which we'll call ACR).

To me these seem like possible legitimate breakdowns based on the dynamics of a given race (economic vs. social conservatives, economic vs. social liberals, etc.)

Under IRV, Mr. Anderson is eliminated in the first round because he has the least amount of first place votes. The four voters who ranked him first now transfer their votes to their second choice, Mr. Carter. Mr. Carter now has 9 votes and a majority. He wins 9 to 8 over Mr. Reagan.

But let's say, Mr. Ruffini sees what's going on... See, Ms. Soltis conducted a poll before the election and informed Mr. Ruffini that if they both switched from RAC to ACR (or ARC... doesn't matter), Mr. Anderson would have six first place votes.

Now, Reagan has six first round votes as does Anderson. Mr. Carter is now the candidate eliminated with only five first round votes. With Mr. Carter eliminated, his voters (who ranked CRA) votes are now distributed to their second choice candidate... Mr. Reagan.

Mr. Reagan now has 11 votes and a clear majority. He wins the election thanks to Mr. Ruffini and Ms. Soltis switching their votes.

Note, this happened in the 1990 Ireland election.

Specter/Sestak: Do the I(VR)s Have It?

2010-05-03T15:01:00.000-07:00

Is Joe Sestak closing the gap between Arlen Specter and himself through television ads, or was the race always close?

About a month ago three polls conducted for the 2010 Pennsylvania Democratic Senatorial primary were released within a week of each other, and they all showed vastly different results. Quinnipiac had Specter leading Sestak by 21%, Susquehanna by 14%, and Rasmussen by 2%. This past Saturday I tweeted "IVR polling was much more accurate in 04 Rep primary with Specter v. Toomey... Does this mean Ras[mussen] is right?" The tweet spoke to something I noticed two weeks ago when referencing back to polling on the 2004 Republican Senatorial Primary in Pennsylvania. In that contest, then Republican Arlen Specter just escaped a strong challenge from now Republican nominee Pat Toomey. In the final month of that contest, automated phone polls conducted by SurveyUSA showed at times a much closer race than live interviewer polls done by Quinnipiac and Franklin and Marshall.

Three weeks to a month before the 2004 primary, Quinnipiac had Specter up by 15% and Franklin and Marshall by a nearly identical 13%, but SurveyUSA had him up by only 6%. Then 10 days before the primary, all the pollsters agreed that Toomey was only 5-6% behind. On the eve of the primary, Quinnipiac gave Specter a 6% lead, but SurveyUSA saw a 48-48% tie. On election day, Specter won by only 1.5%.

Today, as we stand two weeks before the 2010 Democratic Senatorial Primary, a new live-interviewer poll from Muhlenberg College paints a much closer race than previous live interviewer polls. Like the live interviewer polls taken around this point in 2004, Specter has a 6% lead over his opponent. This result is close to the Rasmussen poll released about three weeks ago.

Why was the automated phone polling right in 2004 and looks to be onto something in 2010? It could have to do with the tighter likely voter screen automated phone polling usually uses. Base voters (who would have favored a challenge from the right by Toomey in 2004 and Sestak in 2010) are the voters who are most likely to actually vote. It is the reason that both the Keystone and SurveyUSA polls found Toomey performing 5+% better 2004 when applying stricter voter screens. It should also be noted that base voters are also more firmly committed to their candidate, which even the Quinnipiac poll, that gave Specter a 21% lead over Sestak, showed.

The bottom line is I think this election is probably going to be a close one, and Specter better hope for a healthy turnout.

Arkansas Democratic Primary Runoff?

2010-05-02T15:12:00.000-07:00

It is no secret that I have believed that Bill Halter would be a strong challenger to Blanche Lincoln. Lincoln's weakness in general election match-ups combined with aggravating of the liberal base have left her vulnerable on two key fronts.

Now with the primary election only two weeks away and early voting starting in earnest, one question is will Lincoln survive? I do not know the answer to that question. The other query is will Lincoln receive the necessary 50%+1 to escape a runoff with Halter. I think the answer is increasingly no. Why?

Well, let's take a look at the trend seen in the only pollster (DKos/Research 2000) to poll the race consistently.

The overall trend shows that Lincoln has been pretty much stuck in the middle 40s and Halter slowly rising. However, the more interesting part of the graph is what happened with the other category. Right around the time conservative Democrat D.C. Morrison filed to run, the other category all of a sudden started polling in the mid to high single digits.

Normally, the rise of an otherwise unknown third candidate might be scoffed at as nothing more than poll respondents disgusted with negative attacks of the two major candidates going for the third name on the questionnaire. In the 2009 New Jersey gubernatorial election, Independent Chris Daggett did much better in the final aggregate of polling than on election day due most likely to such a phenomenon. The key difference here is that, if the poll is conducted correctly by the interviewers, Morrision's name was not even mentioned by the them ("If the Democratic Primary for U.S. Senate were held today, for whom would you vote for if the choices were between Bill Halter and Blanche Lincoln?"). It was the interviewee who without mentioning by the interviewer said some form of "other". That is people knew there was another candidate, and they were actually going to vote for him/her (not just threatening to vote for them). To me, the support for Morrison seems to be real.

Interestingly (but not surprising considering considering he's a conservative), Morrison's support seems to be among whites only.

When we break down the Kos polls by race, we also see that Halter's support among blacks is significantly higher than among the electorate at large. In fact, black voters have preferred Halter to Lincoln in the last few Kos polls. This lead has gotten larger with each successive poll. Why is this important? Blacks are pegged to make up less than than 20% of the Arkansas electorate, but they are also much more undecided at 30%. Of course, with a little less than 70 respondents in the Kos surveys, we should keep in mind that the black subgroup has a much larger margin of error. That said, because black support for Halter has risen in every poll, it seems to be indicative of a real trend. Why else would Barack Obama (who is NOT beloved in Arkansas, but has a 91% favorable rating among Arkansan blacks) and Bill Clinton (loved by Arkansan blacks) be cutting ads for Lincoln?

In the olden days, the fact that Lincoln is stuck in the mid 40's would mean she was not going to rise any further. Thus, Halter's rise and Lincoln fall would continue, and Halter would pick up the majority of remaining undecideds. Caution prevails though as such a trend did not occur in the Illinois Democratic Gubernatorial primary (and I was BURNED by it). Unfortunately for Lincoln, even if undecideds did not break as previous undecideds and instead voted as the rest of the electorate as a whole, Lincoln will just escape the runoff with about 50.5% of the vote.

Still, with Lincoln's support falling and being right on the edge of a runoff, I'm thinking a runoff is probable.

Note: Thanks to the Mark Blumenthal for polling advice.

The No-Party Affiliation Primary aka Crist Fail

2010-05-01T21:39:00.001-07:00

What primary is the Governor speaking of? The one he skipped out on? Of course as a dear friend said maybe Crist wants people to change their party registration for the "No Party Affiliation Primary"....

Coming up: Arkansas

2010-04-30T11:28:00.000-07:00

I'll be on later today with a post on why I believe Blanche Lincoln will have to face a runoff...

Charlie, the Dark Horse, Crist

2010-04-28T08:32:00.000-07:00

Charlie Crist looks like he'll be declaring his independent candidacy tomorrow. I think his chances of winning are quite low. As I have pointed out, Crist will be not be the new Joe Lieberman. Crist will not have the dough and more importantly Kendrick Meek is no Alan "Gold" Schlesinger, the sacrificial Republican lamb in '06 Connecticut. But even if somehow Crist got some money, the math just does not seem there.

Looking at the four most recent polls and using the Florida electorate calculator created by my boss Mark Blumenthal, it is very difficult for me to see the path for Crist to win... even in his best case scenario.

Issue 1:
As Nate Silver has noted, Marco Rubio is well on his way to earning 70-80% of the Republican vote in a general election. Rubio's net favorable split among Republicans was an astronomical 66/8 in the latest Quinnipiac poll. Only one of last four polls has registered Crist's support among Republicans above 30%, while the average is in the upper 20's. Expect that number to fall or stay level once Crist declares an Independent candidacy. Does anyone think undecided Republicans will flock to the candidate who deserted the party (Crist) and not the Republican they like (Rubio)?

Issue 2:
Since 1998, no Democratic candidate for major statewide office has earned less than 78% of the Democratic vote. Even in the Jeb Bush romp of Buddy MacKay in 1998, Bush only took 21% of the Democratic vote. Crist only got 14% of Democrats in 2006. Now you may say Crist running as an independent candidate will be different. Maybe. Remember Meek, an African-American, will almost certainly win all the approximately 20% of African-American Democrats. But for arguments sake, we will say that does Crist holds the nearly 30% of Democrats polls claim he will get come November (and those polls will change as Meek brings up his very low name recognition with advertising).

Issue 3:
Independents. Crist is in the mid to upper 30's in all the polls among his "base". So said number may rise on declaration of an independent run, but they probably will not. Why? Most independents usually lean strongly to one party or the other. It is the reason that Lieberman's numbers stayed steady among independents even after he declared an independent run in 2006.

So where does that leave us? Using the 2006 midterm electorate as a model (it worked quite well in Virginia and New Jersey in 2009 and Massachusetts in 2010) and giving Crist his most favorable electoral breakdown (30-R, 30-D, 50-I), he only gets 35% of the vote. That means, he would have to pray that Rubio and Meek literally split the rest of the vote down the middle.

Just to show you how tenuous this prayer is give Meek 2% of Republican support and understand Meek will probably take at least that much. All of a sudden, Crist takes only 34.2% of the overall vote.

It is literally a straw house that the big bad wolf is ready to blow over.

Dem Doom on Generic Ballot Still

2010-04-26T13:05:00.001-07:00

This will be part of a series of Generic Ballot posts this week. I am currently in consultation with David Shor (a much smarter person than I) of Stochastic Democracy on an improvement to the "Bafumi" model. This post will be updated as warranted. Stay tuned.

When I first forecast a doom and gloom House of Representatives forecast for Democrats in February, I got a major response from across the spectrum. Since that time, others from my boss Mark Blumenthal to Alan Abramowitz to Nate Silver have chimed in on the conservation.

For those that do not remember, I plugged in the current Pollster.com generic vote estimate into a midterm House national vote model brought to us by Joseph Bafumi, Robert Erikson, and Chris Wlezien. Two months ago, the model predicted the Republicans would win the National House vote by around 8% and almost certainly win control of the House. What I love about the model is that it is the only one I know of that is not based off of the notion of "if the election were held today". Rather, the model takes into account how far out you are from a midterm election. This time adjustment gives the model an extra degree of accuracy because it turns out that the generic ballot tends to have a Democratic as well as current party of the president bias this far out from an election. That is, the Democratic party and the party of the president almost always do better on the generic ballot this far out than they do in November.

As we are now "only" 189 days until the election, I thought it was time to re-examine my initial post and see where we stand according to the model. The purpose here is not to not predict the exact percentage of the National House vote the Republicans will get OR the exact amount of House seats they will win. If I could do that, I'd be heading to Vegas. What I want to show is the chance Republicans win at least a majority of the national vote and seats in the House of Representatives.

Before we begin, let us lay down some ground rules. First, since this model was based of what I believe was entirely live interview polls (as opposed to those conducted using a computer or automated voice), I am only going to use those. That means I will not be using Public Policy Polling, Rasmussen, or YouGov/Polimetrix. This exclusion does not mean these pollsters are inaccurate (and lord knows I have defended automated voice polls to the high heavens), but these polls may have a bias on the generic ballot that this model does not take into account. Second, I will also be taking out all Republican firm polls because I want to illustrate that the result the model outputs is not due to some Republican bias in the polling data.

With that out of the way, let us begin with a projection for the National House vote. As you can see, Republicans currently hold a 45%-42% lead in the Pollster.com aggregate. If you allocate undecideds based on how those who already registered a preference say they are going to vote [Republican % of vote / (Republican % of vote + Democratic % of vote)], it's about a 3.4 point Republican advantage. As the chart below illustrates and as it was in February, Republicans have not since the generic ballot was first implemented in 1946 held such a large advantage this far out from an election with a Democrat in the White House. Because of the unprecedented nature of this lead, the model projects Republicans will win the national popular vote by about 8.6%. Put another way, Democrats garner about 45.7% of the two way National House vote.

The question that should be running through your mind is how accurate is this forecast? The root mean square error is 1.82% for the Democratic share of the two way National vote. If we double the root mean square error, we will essentially (not perfectly) have your standard 95% confidence interval. So if we add 2*1.82 to our models estimate of 45.7, we end up with an upper threshold for the Democratic share of the two-way National House vote of a little less than 49.4%. Of course, 2.5% of the region outside of our 95% confidence is on the lower end (meaning actually below our estimate for the Democratic two way vote). Thus, there is only a 2.5% chance that the actual Democratic vote ends up being greater 49.4%. The model would have to be really, really off the mark for the Democrats to win the National House vote.

What about actual House seats? Each House election has a different popular vote to seats curve, which makes projecting control of the House more difficult than forecasting the National House vote. Later this year when it becomes clear how many incumbents are running for re-election and the quality of their challengers, we will be to use more precise votes to seat measurements as Bafumi et al. did in 2006 and Jonathan Kastellec, Andrew Gelman, and Jamie Chandler did in 2006 and 2008.

For now, we can use a less precise generic vote to seats estimate that Bafumi et al. supply (see page 9). Taking the current two way generic vote and allocating undecideds as we did above, we find that the model predicts that Democrats will win only 44% (or 191) of the House seats. Now personally I do not believe a 66 seat lost for the Democrats is likely. I think it will likely be lower. Still, the standard error of the estimate of this projection is +/-3.70%. That is, we can be 68% confident that the ultimate percentage of seats controlled by the Democrats in the House will be +/-3.70 of 44%. Remember half of the 32% (100 - 68) of the uncertainty can be relegated to Democrats garnering lower than what is within our confidence interval. Only 16% of the time would we expect Democrats to do better than winning 47.70% of the House seats.

Now does mean the Democrats are dead in the water? Not automatically. Keep in mind this data is based off of only 15 elections (though the inclusion of the 2006 midterm makes no substantive difference in the model's accuracy, see page 41). For all we know, something that really shakes up the political atmosphere may happen. But to ignore what are some pretty clear historical signs would be crazy.

As I said two months ago, I have a very hard time believing the Democrats maintain control of the House.

Note:

For the nerds out there who will actually look at the Bafumi, Erikson, and Wlezien paper, you will know that I did something slightly different than what they did. If I did use just a general average (weighting by sample size) as Bafumi et al. did and take into account not weighting by likely voters (see page 18), I would still be projecting with the same certainty that the Democrats lose the national House vote and control of the House.

Generic Vote: Update

2010-04-25T20:59:00.000-07:00

I will definitely be doing a post on this tomorrow... But I will be projecting a Republican controlled House of Representatives with 85% confidence... and the Republican party winning the National House Vote with 98% confidence...

And yes from this far out.

Test: Campaign Finance Calculator

2010-04-22T20:07:00.000-07:00

This will hopefully be part of a larger and more organized post in the next few days...

Whenever the latest quarterly reports from candidates are reported by the Federal Election Committee (FEC), I really have no clue what the heck is going on. How can raising $2 million in a campaign for Senate in New Hampshire be comparable to $5 million in a campaign for Senate in New York.

So here is what I did. I compiled the finance reports of all Senate races that were rated competitive by the Cook Political Report as of the 1st quarter of 2010. Most of these data are available from OpenSecrets, but I had to fill in some gaps using news reports from the National Journal among others. I collected both cash-on-hand (the amount campaigns have left to spend) and the total amount raised in the campaign for the most likely candidate to win the Democratic and Republican nominations.

But how do you compare between states? I found out the total amount of voters in each of these states in the last midterm election (2006) from a nicely compiled list by Michael McDonald at George Mason University.

I then took the amount of money raised and cash-on-hand and divided each of these by the amount of voters in 06. Therefore, we know how much each candidate has raised per voter in a state. The amount of voters in a state will correlate pretty highly with the amount needed for advertising, direct mail etc. This allows us to know if $5 million raised by a candidate in a large state is really that much compared to $1 million by a candidate in a small state.

I've also included where the race currently stands according to the Pollster.com aggregate or RealClearPolitics, if a Pollster.com aggregate is not available.

Here's a preliminary table... be on the look out for a more substantive post in the future. What you see is pretty simple. Harry Reid is down by a lot, but he'll have a lot of cash to try and make up that deficit. Barbara Boxer, on the other hand, may have over $8.7 million, but that really isn't that much considering how many voters are in California. Luckily for her, Tom Campbell (the likely Republican nominee) can only spend around 12 cents per voter right now (that number should go up if he wraps up the nomination).

Either way, it is just another way to look at fundraising. On the left side is the regular fundraising just toggle to the right to see money per voters. Also feel free to stretch out the columns to read their full names. This post is still preliminary, but I like getting things out there.

Florida Calculator test

2010-04-21T16:51:00.000-07:00

Post in Progress: Is Crist the new Lieberman

2010-04-20T16:04:00.000-07:00

This post is a rough outline for a post, but it is placed online as a sort of copyright. It compares Joe Lieberman's successful senate run as an independent and Charlie Crist's possible run in Florida.

Comparing a poll three weeks before Lieberman became an independent in 2006 and the last Quinnipiac poll from Florida (three weeks before deadline).

http://www.quinnipiac.edu/x1297.xml?ReleaseID=1445 and http://www.quinnipiac.edu/x1296.xml?ReleaseID=940

1. Lieberman had a 20% net approval (Crist's is only +10). His approval was +3 among
Dems (only +15 for Crist), +48 among Reps (+7 for Crist), and +16 among Indies (only
+13 for Crist). In other words, Lieberman doing significantly better in his crossover
appeal and among indies. Lieberman's favorable among Reps was +43 among Reps (+7 for
Crist), +11 among Dems (+20 among Dems), and +19 among Indies (+15 among for Crist).

2. Schlesinger, like Meek, is unknown, but Schelsinger was even more unknown. Only 9%
were able to form an opinion. 24% can form an opinion about Meek, and unlike
Schlesinger, Meek's response has been overwhelmingly positive.

3. Lieberman led in the last pre-primary three poll by 24% over Lamont and was over
50%. Crist only leads by 2% over Rubio and is only at 32%. As you said, an indie's best
day is before he declares.

4. Money. MEEK HAS ALREADY RAISED $4.7 MILLION. Schelsinger raised $221,000 for the entire election.

Crist's only raised $1.1 million in the second quarter. Rubio raised $3.6 million. Although Crist currently holds a 4 million dollar edge in fundraising (and cash-on-hand), Rubio should easily be able to make that up and probably exceed Crist with the netroots and party infrastructure behind him.

Crist won't be able to boot Meek from the race, and he won't be able to outspend Rubio.

Exit poll make up in 2006. Not that many indies in Florida who are Crist's strongest group.
38 Democrats, 26 Republican, 36 independent - Connecticut
36 Democrats, 39 Republican, 25 independents - Florida

To believe or not, Rasmussen on PA's Dem Sen Primary

2010-04-13T19:43:00.000-07:00

Is the latest Rasmussen poll for the Democratic Senate Primary in Pennsylvania showing Joe Sestak trailing by only 2% to Arlen Specter an "outlier"? By pretty much any person's definition, it is.

Take a look at the Pollster.com aggregate prior to the Rasmussen poll being added:

Specter leads by over 22%, and the trend lines are pretty flat (indicating a stable race). Do any other recent polls have Sestak even close to Specter?

A Quinnipiac poll released just last week showed Specter up by 21%. Indeed as the table above shows, Rasmussen's last poll also showed a much closer race (11% Specter lead) than other polls taken around that same time (all showing a 20%+ Specter lead).

Now does that mean Rasmussen is wrong? That is not my place to say; however, I would not dismiss Rasmussen out of hand. A little dip into the polling archive reveals that a Quinnipiac poll taken around this time for the 2004 Republican Senate Primary had then Republican Specter leading Pat Toomey by 15%. A Quinnipiac poll taken only two weeks later had Specter up by only 5%. He would go on to win by only 2%. So maybe it will be the other pollsters playing catchup to Rasmussen... Only time will tell.

Rasmussen: Generic Ballot Right or Wrong?

2010-04-13T10:22:00.000-07:00

Are Democrats going to lose 60+ seats? A lot has been written about the generic ballot in the past week. One thing Alan Abramowitz '02 (vs. a separate model that incorporates presidential approval) and Nate Silver's findings agree upon is that if the Republicans win the generic vote by 3.0% or greater on election day, they will probably take control of the House of Representatives. A loss of 7 points or more would result in a 1994 or worse scenario for Democrats.

The latest Rasmussen poll has Republicans winning on the generic ballot by 9 points (47%-38%). Is Rasmussen right? Could they be even under-doing a Republican romp as Nate Silver suggests they may?

The answer is maybe.

Since 2002, when Rasmussen first asked the generic ballot question, they underestimated, nailed, and overestimated the Republican vs. Democratic margin. As the table above illustrates, Rasmussen's final pre-election poll low-balled the Republican margin in 2004 by 8.2%, but did the exact opposite in 2008, overshooting it by over 4%.

It should be noted that Rasmussen adopted its currently dynamic weighting process in 2006, which helped them perfectly predict the spread between the two parties in the national house vote in 2006. Yet, this same dynamic weighting led to Rasmussen under-predicting the Democratic victory in 2008.

Interestingly, the spread between the two parties on the generic ballot barely budged from April to Election Day from 2004-2008 . That is, Rasmussen's polling was very stable, which can likely be contributed to Rasmussen's weighting by party. Of course, in only 2006 could the stability be seen as a sign of accuracy.

So what does all of this information tell us about Rasmussen's generic ballot polling 2010?

Basically, it may be accurate or it may not be, but it will most likely be consistent.

Senate Update: Republicans right on the edge

2010-04-09T06:17:00.000-07:00

About a month ago I made a post on how the Republicans really had a pretty darn decent chance of taking back the United States Senate based on historical polling data since 2006. I thought it might wise to update that analysis right now.

For those that do not remember, no governor or senator has been re-elected when trailing by more than 1.5 points in a January to June polling average. Needless to say, Democratic incumbents are still in major trouble.

Here's a race by race rundown of the January-now average with likely Democratic incumbents and Republican opponents.

Starting with the incumbents:

1. Arkansas- Leaving aside Bill Halter, who if he wins the primary all bets are off, we see that Blanche Lincoln has a better chance of becoming a New York Yankee than winning re-election. Against John Boozman, Lincoln trails by 14.6%. Against Gilbert Baker, it is not much better at 11.1%.

2. Nevada- Harry Reid better hope the supposed Tea Party candidate has a get out of jail key card because the numbers look like the average day in Syracuse, NY (downright dark). He trails Sue Lowden by 9.4% and Danny Tarkanian by 7.9%.

3. Colorado- Sticking with the incumbent Michael Bennet (trust me the numbers are NOT brighter for Andrew Romanoff), the polls are not looking all that bright. He trails Jane Norton by 7.4%, and he is down 1.7% to less likely nominee Ken Buck. Bennet better pray on facing the latter.

4. Pennsylvania- Arlen Specter might have finally run out of luck. He looks like he'll make it passed Joe Sestak, but history is against him in the general. He trails Pat Toomey by 4.8% in the average. Indeed, Specter seemed to get an umph in March, but the latest polling shows him falling right back behind.

5. New York- About the best news for Democrats. Kirsten Gillibrand still trails George Pataki by an average of 3.4%, but Pataki is more likely to run for President (and lose). I guess Gillibrand has the luck of Dartmouth behind her.

6. Washington- I'm going to wager that Dino Rossi does run. Right now, the average has Patty Murray up by 1%. If Rossi jumps in, expect a lot of ratings on this race to change in a hurry.

7. California- This race should be a lot higher on people's list. Barbara Boxer only leads Tom Campbell by 3.25% in the average. If Carly Fiorina and her farm animals win the nomination, she trails by 5.9% on the average. Boxer better pray for sheep.

8. Wisconsin- As of my last rankings, I did not believe Tommy Thompson would enter, but he is now about 50-50. He has got a 4% lead over Russ Feingold. If Thompson does not enter... well say goodnight to a Republican pickup opportunity.

I count at least 4 greater than 50-50 pickups in there and maybe more depending on whether Thompson or Pataki runs.

What about the non-incumbents?

1. North Dakota- John Hoeven (R) leads by an average of 47.5% over Tracy Potter (D). It is over (barring a scandal of some sort).

2. Delaware- Mike Castle (R) leads by an average of 22.7% over Chris Coons (D). Coons will need to pull a rabbit out of his hat.

3. Indiana- If Dan Coats (R) is the nominee, he beats Brad Ellsworth (D) by 10%. If John Hostettler (R) is the nominee, he defeats Brad Ellsworth by 14.3% . Either way, it looks difficult for Ellsworth at this point.

4. Illinois- Well, I must say Alexi Giannoulias (D) is in deep stuff against Mark Kirk (R). Family banks are messy business, and he now trails by 0.5% (including a few internal polls found in the last post). Expect it to get worse for Giannoulias, if the banking stories get worse.

So what does that leave us with? Republicans need 10 seats for control. History and polling dictate they are right on the edge. They would probably take 9 seats, if the election were held today.

You heard it here first

2010-04-09T05:44:00.000-07:00

http://www.fivethirtyeight.com/2010/04/generic-ballot-points-toward-possible.html

http://poughies.blogspot.com/2010/02/republican-blizzard-on-generic-ballot.html

Right. Well, let me state it once from the mountains for y'all to hear. There isn't a SINGLE CONGRESSIONAL MEASUREMENT INDICATING THE REPUBLICANS DON'T DO VERY WELL THIS NOVEMBER.

Presidential approval, which has been linked in the past to Congressional loses, is about the only "good" measurement right now. Though if we look back to last big Republican wave (1994) that so-called advantage died for Democrats around July.

Same-sex marriage referenda from within... You only need three

2010-03-22T21:53:00.000-07:00

Any modern student of political science has read Andrew Gelman's et al. magnum opus Red State, Blue State, Rich State, Poor State. In the book, Gelman et al. illustrate one of the more interesting facets of American political life: rich states vote Democratic, while poor states vote Republican. It is a bit counter-intuitive (as Democrats are supposedly the party of the workingman), but it turns out that it all makes sense in the end. WITHIN each state, rich voters are more Republican, while poor voters are more Democratic. You might be wondering what this has to do with Gay marriage bans. The answer is pretty simple: what works to explain electoral phenomenon between states does not necessarily work to explain what happens within them.

For those that follow this blog, predicting (and explaining) support for same-sex marriage referenda has been a favorite topic. Before I discovered that polling data worked so well to predict same-sex marriage referenda, I tried a buffet of other explanatory variables. With the ever-expanding research in this subsection of LGBTQ studies, I mostly concentrated on the findings of Patrick Egan and Ken Sherrill in their 2008 paper "California Proposition 8: What Happened and What Does the Future Hold". The paper was published after Californians voted in favor of Proposition 8 to dispel the myth that African-Americans were responsible for the passage of Proposition 8. Egan and Sherril found that race had very little predictive or explanatory power in understanding a person's vote on Proposition 8. Instead, people's votes were much more likely to breakdown along party identification and level of religiosity (tendency to attend church). One variable they did not (and could not) check (because their dataset did not have a question on it) was level of education. Yet, in the less discussed, but equally as close Florida Amendment 2 to ban same-sex marriage (and civil unions), education was found to be the main factor in people's vote. In fact, Daniel Smith found that in the Florida vote, "education level was five times more important than race in determining how people voted. The more educated people were, the more likely they were to oppose the amendment". Thus, we had three variables that predicted how people voted in California and Florida: education, party identification, and religiosity.

Wanting to make a big splash (as I always want to do), I decided to plug these variables into a large model containing the votes of same-sex marriage ban referenda nationwide. Since 1998, there have been 33 such referenda. For each referendum, I have collected the percentage of people who consider religion to be an important part of their lives in each state, the percentage of people in each state who had a bachelor’s degree, and the partisan lean in each state measured by the Cook Partisan Voting Index (many states do not have voter registration by party identification).

In a linear regression model of these variables predicting support in the aforementioned 33 elections, we can explain about 65% of the differences between states' support. That is pretty decent, but it is not anywhere near the 95% of our polling model. More than that, only religiosity and the percentage of people with bachelor’s degrees really add anything to the model. That is, the partisan nature of each state does little to predict the changes in support for the referenda from state to state.

But this comparison is a bit unfair. Nate Silver's research tells us that earlier same-sex marriage elections as well as those that just ban same-sex marriages (not both same-sex marriages and civil unions) are more likely to garner higher percentages of the vote. What happens when we control for the year and whether an election is a ban on just same-sex marriage or a ban on same-sex marriage and civil unions?

Now, we see that the partisan nature of a state, religiosity, and education level of a state do have statistically significant prediction (explanatory) power with 90% confidence (the effect is not due to chance) of support for same-sex marriage referenda. As we would expect, Republican states, religious states, and states with less educated people tended to vote for same-sex marriage bans in higher percentage. In addition, this model explains 83% of the difference in vote between different elections. However, it should be noted that it is really religion, year, and the nature (just a marriage or marriage and union ban) of the election that have the greatest impact on understanding the vote in each state. To me, this model does not do much to add to our understanding or predictive value. Our polling model does a much better job.

Still, I remembered what Gelman et al. with concern to income and state. I asked myself if the differences between any two states' votes cannot be explain by a model because the dynamics between the elections played out differently. That is, a highly publicized vote in California is going to be different by nature than say one in South Dakota. At the same time, religion might be played up more in some elections than in others. It would be very difficult for a simple regression model playing off of mostly demographic data to be able to pick up on these differences. Fortunately, within any one election, the dynamics would be the same. Most people would read the same newspaper endorsements, watch the same television advertisements, and be delivered the same mailers.

To test my theory, I have decided to check the county-by-county differences in vote in three recent same-sex marriage elections. The massive volume of counties that have voted in same-sex marriage referenda elections is 1,000+, so that it is not within the scope of this study to look at all of them. Instead, I downloaded data from the Arizona 2006, California 2008, and Maine 2009's same-sex marriage elections. For each election, I collected the percentage of people with bachelor degrees in each county, percentage of people who are religious adherents, and percentage of the vote each county gave to Obama in 2008 (a measure of party identification). Within each state (5 of 15 in Arizona, 18 of 58 in California, and 1 of 16 in Maine), some counties do not have a percentage of bachelor degree measurement because of a lack of population. Still, each state has more than 2/3’s of the counties covered, and there is no reason (I can think of) that not having these counties would greatly affect our models. With these counties and the variables I had, I made three linear regressions for each state: percentage of county supporting a ban on same-sex marriage = percentage of people with a bachelor degree within each county + percentage of vote Obama earned in each county + percentage of people who are religious adherents in each county.

It turns out that this model does a very good job at explaining the differences in vote within each state. In each state, over 91% of the difference in vote between the counties is explained by these three variables alone. In all three states, all three variables were statistically significant with at least 90% confidence. Counties with more religious adherents, less Obama supporters, and less educated people vote were more likely to vote for the same-sex marriage bans. Perhaps it is intuitive, but due to the nature of regression we also know such things as more religious Democratic counties are more likely to vote for bans than less religious Democratic counties.

In the 2006 Arizona election, 94.7% of the vote differences between counties can be understood with these three variables. Obama vote and bachelor degree are statistically significant with 99% confidence, while religion is significant with 90% confidence.

In the 2008 California election, I confirm the finding of Egan and Sherrill and find that religion and partisanship are statistically significant in their prediction of voting yes on Proposition 8 with well over 99% confidence. Education, a variable that they did not check, is also significant with well over 99% confidence. Overall, 94.0% of the vote difference between counties can be explained by these three variables.

In the 2009 Maine election, 91.5% of the differences between the 15 counties I checked are explained by these three variables. Both religious adherence and Obama vote are statistically significant with near 99% confidence, while the percentage of people with bachelor degree is significant with greater than 99% confidence.

All of these results make a lot of sense. Democrats are likely to see same-sex marriage as a civil rights issue, and therefore support it. Education affects views on same-sex marriage because, as Daniel Smith put it, "Education is so important because it increases exposure to those who are different. Studies show very clearly that the more educated people are, the more tolerant they are of differences". Religious people are more likely to oppose same-sex marriage on "moral grounds".

Going forward, it will be interesting to see if these three states are merely aberrations or more indicative of a larger trend. One thing I do caution is extrapolating these results to the individual (person) level. The research indicates that education, political affiliation, and religiosity do impact people's stances on the issue of same sex marriage, but it may not hold in all these states (with the exception of California where Egan and Sherrill have found it to); however, I would guess that they do.

Notes-

1. Religiosity was measured in two ways. For the state-by-state model, religiosity is measured by the percentage of adults in a state who considered religion an important part of their daily lives in a 2008 Gallup study. I think this a very good measure, but it is not available by county. For the county data, I had to settle for the percentage of religion adherents per county. The estimate comes from the Association of Statisticians of American Religious Bodies (ASARB) The data has been “adjusted” by Roger Finke and Christopher P. Scheitle of Pennsylvania State University to account “for the historically African-American denominations and other religious groups not listed in the ASARB totals”.

2. Education is measured by the percentage of people with at least a bachelor's degree in a given county or state. Since this data only goes back to 2001 (and it would take a lot of time to gather the data by year of election for the state level... something I was not too interested in obtaining for this study), I used data from the 2008 American Community Survey. For the county data, I used the American Community Survey data from 2006 (the year of the election) for Arizona, 2008 (the year of the election) data for California, and an average of 2006-2008 (in order to obtain data from more counties) for Maine.

3. I prefer the Cook Partisan Voting Index as a measure of partisanship (as it combines data from the 2004 and 2008 Presidential Elections), but it is not published county-by-county. Therefore, I just use Obama vote share by county.

4. In case you are wondering, the addition of a race variable has little predictive power. In Arizona, it adds nothing to the model. In California, it explains an additional one percent of the vote. Counties that had less white people were more likely to vote for the ban. In Maine (not exactly a diverse state racially), counties with less white people were actually LESS likely to vote for the ban. Overall, race just did not seem to have that much of an effect.

What do Arizona (2006), California (2008), and Maine have in common?

2010-03-22T02:25:00.000-07:00

The percentage of religious adherents, people with bachelor degrees, and Obama vote explains over 90% of the variance between county votes on same-sex marriage within each state.

Kendrick Meek

2010-03-20T04:21:00.000-07:00

I'll have something to say about his Senate candidacy at some point this weekend or Monday, but needless to say I think he might end up being the Democrats best chance to win a Senate seat currently held by Republicans.

More later on that...