Wednesday, November 02, 2011
Barack Obama and Congressional Democrats have increased their standing among the American public according to a new Quinnipiac poll. But a closer look at the poll reminds us of a divisive question amongst pollsters: should results be weighted by party identification?
Perhaps not surprisingly, not weighting can lead to a situation where results do not change among respondents who identify as a member of a given party, but does change in the overall result because the party i.d. makeup of the poll changes.
Today's Quinnipiac poll, which was not weighted by party i.d., is an example of such a change. As first pointed out by the National Journal's Steve Shepard, the latest Quinnipiac poll's respondents were 4% more Democratic and 6% less Republican than the previous Quinnipiac poll taken in early October.
In the Congressional ballot question, the "generic" Democrat was up 3% (from 39% to 42%), while the "generic" Republican was down 5% (from 39% to 34%) since October. On the question of approving of President Obama's job performance, respondents approved of his job by 6% more (from 41% to 47%) than they did October. The Democratic improvement on these two questions seem to be tied to the increase of respondents who identified themselves as Democrats.
When we break down these two questions' results by party i.d., a similar picture emerges.
Republican support for their party's Congressional candidate dropped by 3% (from 86% to 83%), but Democratic support for their party's Congressional candidate also dropped from 85% to 83%. Independent support of Republican candidate decreased from 35% to 33%. These changes within each party are certainly not enough to create this large of a difference in the overall result.
Approval for Obama is up across all parties: 7% to 10% among Republicans, 77% to 81% among Democrats, and 38 to 39% among Independents. If the sample party i.d. had been constant, Obama would have a higher approval, but not as high as 6%.
Does this lack of change within party merit concern that poll's results are off? It depends on whom you ask.
Most pollsters do not weight by party. This includes CBS, Democracy Corps, Gallup, Pew, and Public Policy Polling (PPP) among others. They only weight by demographic data such as age, education, region, and sex. Quinnipiac reports that these demographics did not change greatly from the October poll.
The reason not to weight by party i.d. varies amongst pollsters, but David Nir of Daily Kos Elections believes that "people can pick and choose their party i.d. If you decide in advance what proportion of the electorate 'should' be Democrats, then, as Tom [Jensen of PPP] says, you're going to miss out on these softer supporters who are apt to change their minds". The demographics (see above) these polls are weighted to do not change.
The argument for weighting by party i.d., practiced most famously by Rasmussen, is pretty simple. The number one predictor of vote choice (and approving/disapproving) is party id, and sometimes the makeup of party i.d. varies significantly from poll-to-poll. The change in Democrats and Republicans in the Quinnipiac poll is right on the edge of what would be deemed statistically significant.
Because both sides make a valid point, I believe the best choice of action is to look at data from other pollsters.
The only other two pollsters that I know who ask the generic ballot question in a manner similar* to Quinnipiac are Rasmussen and Democracy Corps. Rasmussen polls have fluctuated randomly between giving the Republican candidate a lead from 2% to 8% in its 9 samples since September. Democracy Corps mid-October poll showed a tie, little different from its early August poll that gave Republicans a 1% edge.
While the results between Rasmussen and Democracy Corps differ, neither pollster has seen a shift to the Democratic party. That is, the results within each pollster is the same.
Likewise, as this Pollster.com graph of approval ratings show, approval ratings for the President have remained consistent over the last month.
It would appear that Quinnipiac's numbers are a bit of an outlier. It is certainly possible that other polling will begin to show an improvement for Congressional Democrats and Obama. For now, Quinnipiac's poll is a great example of why a pollster's numbers should not be taken at face value.
*Asking Democratic vs. Republican candidate as opposed to asking whether respondents want a Congress controlled by the Democratic or Republican party.
** See Mark Blumenthal for the entire rundown on weighting by party identification.
Subscribe to Posts [Atom]