Guest post by Gallup Senior Editor Lydia Saad
Gallup’s inaugural general election trial heat -- showing Romney leading Obama by two percentage points -- sparked several observers to review the internals for evidence about what was behind the numbers. This was driven by the fact that a number of polls released around the same time showed the race anywhere from a nine-point lead for Obama (CNN), to a tie (CBS), to a two-point lead for Romney (Gallup, Fox News).
Any number of factors could cause polls by different firms to produce different results, including, but not limited to: the dates the surveys were conducted, the wording and placement of the ballot question within the questionnaires, the sampling and calling procedures used to obtain completed interviews (including reliance on cell phones), the “house effects” of the interviewing centers used to conduct the survey, and the weighting procedures used to ensure that the final samples match known parameters in the population. One cannot fairly evaluate the accuracy of any one poll against another without reviewing all of these variables.
However, some observers have latched onto the racial composition of Gallup’s tracking poll as a basis for evaluating the results. Here is what National Review’s Ronald Brownstein had to say about this based on a comparison of Gallup’s April 11-15 sample to past Election Day exit polls:
"But the Gallup track, which is conducted among registered voters, has a sample that looks much more like the electorate in 2010 than the voting population that is likely to turn out in 2012: only 22 percent of the Gallup survey was non-white, according to figures the organization provided to Emory University political scientist Alan Abramowitz. That was close to the non-white share of the vote in 2010 (23 percent), but in 2008, minorities comprised 26 percent of all voters, according to exit polls; the Obama campaign, and other analysts, project the minority share of the vote will increase to 28 percent in 2012. In its survey, Pew, for instance, puts the non-white share at 25 percent."
The problem with this analysis is that the racial composition of Gallup’s election samples have typically differed from Election Day exit polling, even in years when our presidential election forecast was spot on. Part of this stems from the different approaches used by Gallup and the network exit poll survey for measuring race. Gallup uses a multi-question approach to assess the race of the respondent, similar to Census Bureau protocols -- so precise comparability shouldn’t be expected. But as shown in the table below, the proportion of whites has been going down in both exit polling and Gallup’s final pre-election polls, and the 76% we show today among registered voters is lower than the 78% we had in our final 2008 likely voter sample.
Additionally, Gallup’s final pre-election samples typically have a slightly higher proportion of whites than does the exit poll. So in 2004, for instance, 77% of exit poll respondents were categorized non-Hispanic white. That compares with 82% of likely voters in Gallup’s final pre-election survey in 2004. There was a similar gap in 2008: 74% were categorized non-Hispanic white in exit polling vs. 78% in Gallup’s final likely voter sample. However, just because Gallup’s samples were more heavily white in both years, doesn’t mean the final results were skewed to Republicans. In fact, it’s the opposite. In 2004, Gallup’s final poll showed John Kerry doing slightly better than he actually did, and in 2008, Gallup’s predicted share of the vote for Barack Obama was two points higher than he received.
Regardless, the point is that to assess the trend in the racial composition of Gallup’s current trial heat samples, they need to be compared to Gallup election poll trends, not to the exit polls (which, it should be noted, are themselves polls with their own complex issues). On that basis, the 76% of registered voters in Gallup’s recent tracking poll can be compared with the 78% of likely voters in our final 2008 pre-election poll, and so far it looks like the white proportion of the electorate may be trending further down in 2012. However, we can’t make any firm conclusions about the trend since we aren’t yet screening for likely voters. How that looks will partly depend on relative turnout among whites and blacks, and it remains to be seen whether black turnout will be as high in 2012 as it was in 2008.
This year Gallup’s general election trial heats are being reported in near real time based on five-day rolling averages using large samples. The ballot question is the same Gallup has used in presidential elections, and it is the second question asked in the survey, after the registered voter question. Gallup conducts interviews via landline and cell phone, includes Spanish-language interviewing, and weights its samples to the latest census targets (2011 Current Population Survey) for gender, age, race, Hispanic ethnicity, education, region, number of adults in the household, and phone status. Further, Gallup’s unparalleled sample sizes -- reporting over 2,200 registered voters -- across five days of interviewing provides strong statistical reliability and protection from random nightly sample variation.
While the results of various recent national polls do vary, as is normal, most are within expected range of each other when taking into account margins of sampling error around each. Gallup’s tracking at this point indicates that the race is statistically even, with Romney owning a slight advantage, similar to what was shown by the Fox News and New York Times/CBS polls. Two others -- Quinnipiac and Reuters/Ipsos -- show Obama with a slight lead while NBC/Wall Street Journal and CNN give Obama larger leads.
As Brownstein noted, the larger takeaway from these early looks at the general election is that the race appears to be close. That’s the same conclusion one might draw from a variety of other polling and real-world indicators, including Obama’s sub-50% approval rating, depressed economic confidence and U.S. satisfaction, and sluggish improvement in the U.S. unemployment rate.