Thursday, September 27, 2012

The Recurring -- and Misleading -- Focus on Party Identification

The discussion of the party identification composition of poll samples comes up in every presidential election with which I've been involved.  Interested observers often opine that when a given poll shows that Candidate X is ahead, it cannot be correct because there is a higher percentage of voters who identify with Candidate X’s party in the sample than there should be, based on comparison to some previous standard.

There are several reasons why this is a faulty approach to evaluating a poll's results.

Party identification is basically an attitudinal variable, not a stable population parameter. It is designed to vary. This is distinct from demographic variables such as age, gender, race/ethnicity, and education, which are, generally speaking, stable indicators measured by the U.S. Census Bureau. The only issues relating to demographic variables are measurement concerns -- e.g., how the census, which creates the targets, measures ethnicity versus how individual pollsters measure it. But, generally speaking, these are fairly stable targets.

Party identification is not measured by the U.S. Census Bureau, nor are there any other official state or national standards for what party identification "should be" in terms of the percent per party as it relates to the general population.

Many people use the exit polls as a standard. But exit polls use a distinct question wording, a different methodology (in person interviews at the polling place as opposed to telephone interviews), a different environment (people are asked their party identification just after having voted, which could affect how they answer), and different sampling techniques to develop who it is that is asked the question. So party identification figures as measured by a specific poll aren't easily compared to party identification as measured by an exit poll because of these and other potential issues.

Party identification changes as political tides change. General shifts in the political environment can affect party identification just as they can affect presidential job approval and results of the “Who are you going to vote for?” question.

Here is how Gallup asks party identification: “In politics, as of today, do you consider yourself a Republican, a Democrat, or an independent?”

Note that this question does not ask, “What was your party identification in November 2008?” Nor does it ask, “Are you registered with one party or the other in your state?” Our question uses the words "as of today" and "consider."  It is designed to measure fluidity in political self-identification.  

We know that party identification moves over time -- sometimes in very short periods of time, just like other political variables.  Generally, if there is a political tide toward either of the two major parties, all questions we ask that are of a political nature will move in that direction. This includes the ballot, job approval, party identification, among others.

So, it would not be surprising to find that if Barack Obama is enjoying a surge in popularity in any given state, that surge will show up on the ballot question, on his job approval measure, and on the measure of party identification. So, data showing that Obama is ahead on the ballot in a specific state poll and that Democrats have a higher-than-expected representation on the party identification question, are basically just reflecting two measures of the same underlying phenomenon.

This doesn’t obviate the possibility that a sample is a “spurt” or a sample that happens to pick up higher than usual support for one candidate or the other for whatever reason. But, if it is a spurt, the cause is not “getting too many Democrats/Republicans in the sample.” It is instead a matter of  “Getting too many people who, in response to all political questions, answer in a more Democratic or Republican” way.

Basically, if an observer is concerned about a poll’s results, that observer should skip over the party identification question and just look at the ballot directly. In other words, cut to the chase. Don’t bother with party identification sample numbers.  Look directly at the ballot.

For example, we know that in Ohio:

  • Obama won by 5 points in 2008
  • Bush won by 2 points in 2004
  • Bush won by 3 points in 2000

Now if a given poll in Ohio in this election shows Obama with a 10-percentage-point lead, one should just ask, “How likely is it that Obama would be ahead by 10 points if he won by five points in 2008?” -- forgetting party identification, which we assume is going to be higher for the Democratic Party if Obama is ahead, anyway. The discussion of the ballot in the context of previous ballots is, in fact, a reasonable discussion. It may be unlikely that Obama will double his margin in 2012 from what occurred in Ohio in 2008. Or maybe not.  But the focus should be directly on the ballot, and discussions of reasons why it might be different than one expects should not involve an attempt to explain the results by focusing on changes in party identification  -- which is basically a tautological argument.

In Florida:

  • Obama won by 3 points in 2008
  • Bush won by 5 points in 2004
  • Bush won by [much] less than one point in 2000.

So, if one sees a poll saying that Obama is leading Romney by nine points in Florida, then one should ask how likely it is that Obama will exceed his 2008 margin by six points. That is a reasonable discussion. But one need not attempt to say that the nine-point lead in the poll is suspect because there were too many Democrats and not enough Republicans in the sample compared to 2008. The finding of differences in party identification is, instead, simply reflecting what one sees on the ballot.

Essentially, it is much more direct to just focus on the trends and comparisons of the ballot question than it is to introduce an extraneous look at trends in party identification.

I’ve been analyzing election surveys at Gallup since the 1992 presidential election, and I don’t personally put a great deal of stock in survey-to-survey variations in party identification. All of our weighting focus is on the effort to bring more solid demographic variables into alignment with census figures -- including in recent years cell phone and landline phone use. We don't find that party identification is stable enough to be of much use when it comes to comparing sample-to-sample variations, or sample to exit poll differences.

No comments:

Post a Comment

Comments are moderated by Gallup and may not appear on this blog until they have been reviewed and deemed appropriate for posting.