Friday, June 17, 2011

 

The Hibbs Model DOES Work at Predicting and Explaining Elections

Presidential elections are all about the economy, or are they? Those who have followed my writing know that, at least at this point in the election cycle (far away from the actual 2012 general election), I am big fan of utilizing fundamental models. I stand by this belief.

One of my favorite models is the Douglas Hibbs' fundamental Presidential economic model.

By using just two simple variables (growth in real disposable income per capita over the 16 quarters of a presidential term and military fatailities in unprovoked foreign conflicts), the model explains 87% of the difference incumbent party's share of the two-party Presidential vote from 1952-2008.

Recently, the model came under attack.

The New York Times' Nate Silver ran through a bunch of exercises that seemingly proved that the model was not nearly as accurate at predicting out-of-dataset results for the 1992-2008 elections as its in-dataset (i.e. those data that help build the model) confidence interval would lead you to believe. The model was far worst when fitted to elections prior to 1952.

I appreciated Nate's post because I think we should always be re-evaluating forecasters' claims and theories. I, myself, had previously noted that Hibbs' model would do a poor job at predicting the 1948 election. There can also be little doubt that in-dataset confidence intervals almost always undersell actual uncertainty in estimates.

Yet, I must admit that I think Nate's post misses on a number of points.

Let's first cover the out-of-dataset forecasts for those elections that Hibbs' model has actually been used to predict (i.e. 1992 and onward).

My own amended Hibbs' model (with terms added for congressional control and term of the party in the White House) performs better than the original Hibbs' model with an absolute mean error for out-of-dataset forecasts of 2.09% instead of 2.6% of the original.


With the (big) exception of the 2000 outlier, all of the 92 and onward out-of-forecast errors fell within each year's in-dataset 95% confidence interval (which should be, as noted, too confident).

Personally, I think that is pretty good. We were able to use only a few data-points in each year and no polling data to get a pretty gosh darn good idea of what would occur in these 5 elections.

Of course, in forecasting, we judge a prognostication not only on its ability to predict the final outcome, but on how well it completes this task relative to other predictions. In that spirit, I have decided to dredge up final weekend polling from 1992-2008.

Final weekend polling should be fantastically accurate. Almost all the campaigning is done; almost all the voters have made up their minds; and, all of the polling is of likely, not registered, voters.


Amazingly, in these five elections, the final polling mean is only on average 0.98% more accurate in predicting the final outcome than my amended Hibbs' model. In fact, the amended Hibbs estimate was actually more accurate in two elections (1992 and 1996), and, unlike the polling average, always correctly projected the popular vote winner.

I should point out that polling is far less accurate the further we recede from the election to the point of having no predictive value 300 days out. (We are currently more 500 days away from the 2012 election).

The Hibbs' models, on the other hand, have the potential to produce solid estimates on who will win far from the election. These days we can suppliment known economic data with economic forecasts to produce reasonable estimations of the vote even, I believe, at this point.

But what about Nate's point that the Hibbs' model widely misses on elections prior to 1952?

There are a number reasons not to input data prior to 1952 into the model. Some of those (including how voters viewed the role of government differently prior this party era) are outlined by Seth Masket.

One other point that I have not heard brought up is simply that prior to 1952 elections lacked any true television campaigning. It's difficult to underestimate the effect of how television nationalized elections.

I think these factors that apply to 1952 and afterward make these elections fundamentally different to those prior. It's not surprising that 5 out of the 7 out-of-dataset errors Nate found prior to 1952 are approximately 2 times or more greater than any of those from 1992-2008.

Indeed, political scientists have known elections prior to WWII and afterward seemed to respond differently to the fundamentals.

For example, esteemed political scientist James Campbell noted that there was a difference between presidential coattails pre and post-WWII and actually mapped out different equations for pre and post-WWII.

Some scientists have tried to fit pre and post-1952 data into one model. During the 2010 midterm election, for instance, we saw many fundamental models (i.e. no polling data) of this design, but not a single one correctly projected the Republicans winning control*.

In fact, the only fundamental that saw the House turnover was Hibbs' midterm House model, which uses real disposable income growth data post-1948 like the Presidential model.

So, these combined models may take into account more years, but more years does not necessarily mean a better forecasting model.

If Hibbs' Presidential model initially included Nate's estimates for real disposable personal income growth prior to 1952 (i.e. 1924-1948) and was refitted to predict the 1992-2008 elections out-of-dataset, Hibbs' forecasts would have been worse.

Both the 1992 and 2008 elections would have been incorrectly forecasted to be won by the Republican party. Instead of the 2.6% mean error of the original Hibbs' model, Hibbs' estimates would have been off by about 3.91% with '24-'48 data added.



This difference in 1992-2008 out-of-forecast errors between the models that include and do not include pre-1952 data is so large that it's actually greater than the difference in error between my amended Hibbs' model (2.09%) and the final weekend polls (1.11%) during the same 1992-2008 period.

Personally, I want the model that best predicts the future, not one that makes us feel good inside for post-projecting elections prior to 1952.

But let's say that you are interested in knowing how well the economy explains pre-1952 vote. Nate would like you to believe that these high pre-1952 errors prove that "it’s the economy, stupid. And everything else too" when it comes to explaining Presidential elections. And in one way, I agree with him.

See, the number 1 predictor of how any voter casts her/his ballot is not the economy, but party identification. It's the main reason why we never see Presidential candidates winning much more than 60% of the vote, or the losing candidate earning less than ~40%.

In other words, in a race between a Republican and Democratic candidate, each candidate starts with a base of about 40% of the vote with only about 20% of the vote up for grabs.

This means that even if there was 15% growth (or decline) in real disposable personal income, the effect of this growth on the vote would run out as most of the population makes their vote choice based upon the candidates' party affiliation.

You can see this concept graphically for the incumbent party's, which typically wins a greater, vote share. Note the flat tails at each end as growth becomes incredibly positive or negative.

The image “http://img840.imageshack.us/img840/16/screenshot20110618at429.png” cannot be displayed, because it contains errors.

This high growth/decline factor is not an issue in the 1952-onward data, but it is for the election years 1932, 1936, and 1944.
Link

The question is how do we control for this issue. One adequate and simple (but not perfect) way is to simply cap the economic growth variable at -5 and +5. That is, we simply recode the economic variable in 1932 as -5%, in 1936 as +5%, and in 1944 as +5%.

Let's also add an interaction variable between term of the party in the White House (a fundamental dummy variable that has been found to be quite significant in other models) to this recoded real disposable personal income growth variable and the military fatalities' variable.

Doing so, we can now explain not 59% of election results between 1924-2008, but 77.4%! Three variables** accounting for 77.4% of the difference in results among 22 elections spanning two party systems, a World War, and the move into the television age is fantastic as far as I'm concerned.

But I actually believe that we can explain more than 77.4%. When I first built my amended Hibbs' model, I too had searched the Census archives, but could only find yearly data on disposable income from 1929-onward.

I must admit that I was surprised that Nate found the growth for the 1928 election to be only about 0.30% (the amount needed for Republican Herbert Hoover to receive 47% of the two-party vote). After all, the 1920's had a supposedly booming economy.

In search of data, I plowed through Google and was able to locate disposable personal income data from this (gated) 1946 economic paper.

The data seems pretty accurate***, and I calculate growth rates**** in real disposable income per capita of 2.00% for 1925, 1.35% for 1926, 1.37% for 1927, and 3.39% for 1928.

Remember, however, that Hibbs' model is based off quarterly income growth in which later quarters are weighted more heavily than earlier ones. Exactly how one converts yearly estimates to quarterly will determine the exact weighted quarterly growth rate.

There is no perfect method, but my own imperfect estimation is 1.80%.

I have no idea whether Nate's or my finding for income growth heading into the 1928 election is correct, but I feel my calculated rate better jibes with the overall consensus the economy was doing quite well during the 20's (despite a few bumps along the way).

If we assume my growth finding is right, then 82%, not 77.4%, of the differences among election results between 1924-2008 are accounted for by the model. This means that this model has only about 5% less explanatory value than the 87% of the original Hibbs' model for 1952-2008.



I find it amazing that three non-polling variables can explain 82% of difference in results among 22 elections spanning two party systems, a World War, and the move into the television age.

I still stress, however, that I don't find this pre-1952 prediction exercise all that important for predicting results in this modern age.

We should not incorporate more data into a model just to boost the number of data-points. As I discovered, our ability to forecast elections with economic modeling in 2012 and beyond is impeded, not helped, by adding data prior to 1952.

Overall, these exercises should give us confidence in economic modeling to give us a good early read on the political environment in 2012. Certainly the state of the economy conveys upon us a better idea of the ultimate outcome than determining which candidates smile more or are more charismatic or even the early horse race numbers.

I am willing to say right now that, unless something totally unexpected occurs, the 2012 presidential election will be about the economy just like pretty much every election since 1924. We should be looking growth in real disposable personal income per capita to lead the way.

Notes:

*Ray Fair's House model did not estimate seat share. However, almost all political scientists would agree that to win back the House, House Republicans, due to the Democrat's incumbent advantage, needed to take the popular House vote by more than the 1.6% Fair's model projected.

**Four if you include the original "term" variable, although this makes no difference in explanatory power.

***Note, I found slightly different estimates of real disposable income in a few other papers, but all led to the same conclusion I make.

****Yearly income data is given as the average for a given year. I assume that this average is equal to the real disposable income at the mid-point of a given year. To determine what the income is at the beginning of a calendar year, I average the income data for a given year with that of the prior year. It's not perfect, but I tested it on known beginning year data points and found it to be more than acceptable.

Comments: Post a Comment

Subscribe to Post Comments [Atom]





<< Home

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]