Bookmark and ShareShare
Wednesday, October 31, 2012

Status Update on Gallup Election Polling Following Superstorm Sandy

Here's an update on Gallup’s thinking when we suspended our national daily tracking of the presidential election campaign as of Monday, Oct. 29. Basically, we reached the conclusion that Superstorm Sandy had compromised the ability of a national survey to provide a nationally representative assessment of the nation’s voting population.

An unprecedented estimated 8 million households have lost power because of the storm and many others have had their lives disrupted. Cell phone service has also been compromised in many areas. Additionally, those in the storm-affected regions who can be reached by phone may be unwilling or unable to be interviewed. Ascertaining likelihood to vote may also be problematic when many voters are uncertain of their access to the polls and may not be able to say with a high degree of confidence whether they will vote.

These problems are obviously not uniformly distributed across the country but are concentrated in parts of the Middle Atlantic and Northeast and in specific areas within these regions. Thus, while we could have achieved the targeted number of completed interviews in these regions, those interviews would not have been representative of the overall population of the area. New York state is a prime example. While it would be possible to complete the proportionate number of interviews in New York state, those interviews would be skewed disproportionately upstate, not in New York City and Long Island.

All survey data are weighted to match the demographic characteristics of the population as a whole, including weights for telephone use now that interviewing involves both land lines and cell phones. But it is impossible to adequately weight to compensate for large segments of the population who cannot be reached at all in a survey, or in very low percentages, and whose opinions may have changed from previous, pre-storm measures.

Gallup is now tentatively planning on conducting interviewing over the last four days of this week, Thursday through Sunday, to provide a final pre-election estimate of the election race. The decisions we make on the validity of the sample and the analysis of the data that results will be carefully informed by the degree of recovery from the storm over the period of the survey.

Wednesday, October 10, 2012

Survey Methods, Complex and Ever Evolving

The methods we and other pollsters use to conduct surveys are very complex, but also very important -- and very interesting to people. Lots of correspondents, with varying degrees of understanding and different motivations, ask about how we conduct our surveys. As a former college professor and former president of our country’s largest professional polling association (AAPOR), I think it’s great to educate and explain what it is that we do.

Gallup conducts surveys asking questions about social, cultural, health, wellbeing, political, economic, and many other topics both in the U.S. and around the world, using a number of different methodologies and survey “vehicles.” All are focused on the goal of producing a scientific random sample of the population under study.

Here in the U.S., Gallup used in-person interviewing in randomly selected households in the first decades of our history. We switched to telephone interviewing in the late 1980s, and added in cell phones in January 2008. We have also used various telephone survey procedures over the years, including what we can call “stand-alone polls” which are self-contained, cross-sectional surveys usually conducted over a three- or four-day period, and various iterations of tracking polls, in which interviewing is conducted continuously day in and day out.

Gallup instituted a wonderful Gallup Daily tracking program in 2008, based on separate random surveys of 1,000 national adults conducted each day. Many of Gallup’s basic economic and political measures that are reported on a continuous basis have been contained on a random half sample of this tracking survey since 2008. Our Gallup Poll Social Series surveys reported each month, along with many of the surveys conducted in conjunction with USA Today, have been based on stand-alone surveys.

In all cases we are continually tweaking, modifying, and improving our methodology -- based on decisions made by an outstanding team of survey professionals and methodologists at Gallup. Domestically, we always have the objective of being able to accurately represent the target -- the adult population of the U.S.

One focus point over the past decade (for all of us in the survey profession) has been the need to stay consistent with changes in the communication behavior and habits of those we are interviewing. As noted, Gallup switched primarily to telephone interviewing a few decades ago based on the increased penetration of phones in American households and the increased costs of going into Americans’ homes for in-person interviewing. Now we know, based on government statistics (and what we observe around us), that Americans are shifting rapidly from reliance on landline phones to mobile devices. We first began to add cell phones to our samples in January 2008 and have been increasing the proportionate representation of cell phones in our samples on a periodic basis from that point forward. That’s based on the knowledge that there are more households with cell phones than landlines today in the U.S., or conversely, more households without landlines than without cell phones. We get updated estimates of telephone use from the U.S. government.

For our final month of political surveys before the Nov. 6 election, we are now conducting a separate daily tracking program consisting of interviews with a random sample of 500 U.S. adults each night. This provides us a survey vehicle focused just on the election and other political measures, particularly important in the current situation, in which we need to include a list of likely voter questions along with other political and election questions.

As we began this election tracking program on Oct.1, our methodologists also recommended modifying and updating several procedures. We increased the proportion of cell phones in our tracking to 50%, meaning that we now complete interviews with 50% cell phones and 50% landlines each night. This marks a shift from our Gallup Daily tracking, which has previously been 40% cell phones. This means that our weights to various phone targets in the sample can be smaller, given that the actual percentage of cell phones and cell-phone-only respondents in the sample is higher. We have instituted some slight changes in our weighting procedures, including a weight for the density of the population area in which the respondent lives. Although all Gallup surveys are weighted consistently to census targets on demographic parameters, we believe that these improvements provide a more consistent match with weight targets. The complete statement of survey methods is included at the end of each article we publish at

The fact that the election and political questions are included on a shorter, politically-focused survey each night instead of being included within a longer tracking survey has resulted in increased response rates -- a good thing. One hypothesis is that certain types of respondents may be more likely to stay on the line with a shorter political survey than with longer, more general surveys -- which, in turn, could affect not only response rates but the percentage of "don’t know’s" and refusals. (Gallup also typically sees an increase in response rates for political surveys conducted close to presidential elections in general.) All in all, it is possible that these changes in methods, which we believe increase the representation of our overall samples, may have some impact on political or other measures included in the surveys. Although it's too early to tell definitely, for example, we're looking to see if these improvements in the survey methods have, for example, had an impact on the average values of our presidential job approval rating.

We will most likely make other changes to our survey procedures as our survey methods and procedures evolve.

As noted, one reason for our investment in separate daily tracking during this political environment is the necessity to carry likely voter questions on the survey each night. Our traditional procedure has been to ask seven likely voter questions, and assign each respondent a score based on his or her answers to the questions and then isolate the pool of those who score higher on the scale and are thus -- based on historical trends -- more likely to vote. We have modified some of these questions slightly based on the increasing prevalence of early voting. At this point we are not attempting to precisely estimate real world turnout or to correlate that to our internal sample percentages, but rather making the assumption that those at the top end of the scale are the most likely voters.

As has always been the case, we do not attempt to weight the composition of the likely voter sample in any way -- such as by political party or race or age -- to approximate some guess of what we or others think it should look like demographically on Election Day. That approach is precarious given that the electorate can look quite different (especially looking at political parties) from one election to the next.  Also, party identification estimates are often based on exit poll results, which themselves are surveys using totally different methodologies, and we generally do not rely on judgment calls to predict what the ultimate electorate will look like. Our basic underlying sample of national adults is weighted to known population parameters on demographic and phone use variables, as noted, and the likely voter pool derives from that, based on how our randomly selected respondents answer the likely voter questions.

People ask about the order of the questions on our survey. We are following our historical procedures in this regard at this point in time -- asking the likely voter questions, then the presidential ballot, and then presidential job approval. This marks a slight change from prior to the addition of the likely voter questions, when the presidential ballot and the presidential job approval question did not have any questions before them on the survey, other than a registered voter screening item.

Wednesday, October 3, 2012

Obama's and Romney's Strengths Going Into Debates

Available data show that Mitt Romney’s strengths -- from the perspective of the average American -- appear to be his economic experience and his ability to handle the economy, his ability to deal with the federal budget deficit, knowing and being able to help investors and business, his ability to benefit the interests of men, and his skepticism about the scope and power of the federal government.

Barack Obama’s strengths appear to be his compassion for underdogs and those who are socio-economically disadvantaged, his compassion for minorities, his ability to help middle-income Americans, his ability to benefit the interests of women, his approach to social issues such as gay marriage and abortion, a likable personality, his ability to understand the problems of average Americans, and foreign policy experience. (See here and here for relevant data.) 

Americans have told us this year that the economy and employment/jobs are the number-one issues on their minds. Below economic concerns, Americans’ list of most important problems facing the U.S. continues with dissatisfaction with government and the way it runs, and concerns over the federal deficit and healthcare.

Romney has a slight edge over Obama as being better able to handle the top problem facing the country -- the economy. Romney has an edge on the issue of the federal government, since he is a) not in government and can therefore criticize it from the outside and b) his expressed concern about the size and power of the federal government and his concern about the dependency of Americans on government fits with the majority of public opinion. Romney has a clear edge over Obama on being able to handle the deficit. Romney operates at a 6-percentage-point deficit on healthcare vis-a-vis Obama, but his opponent’s signature approach to healthcare, the 2010 Affordable Care Act, is controversial with the American public and is generally opposed by as many as favor it.

Obama’s campaign has to this point attempted to reposition Romney’s strength on business and the economy as a minus, emphasizing that his skills in this arena are targeted just for those in the upper strata.

Romney’s campaign has not been as focused in its efforts to delegitimize Obama’s strengths, which are more personality based and thus harder to challenge through speeches and paid advertising.

Obama’s most likely plan of attack in the debates will be to continue to emphasize his strengths and blunt those of Romney. In the debates, we would thus expect Obama to continue to emphasize his compassion and understanding of the issues and problems of the disadvantaged, those who are struggling, and those in the middle class, and to continue his efforts to attack Romney’s economic strengths by defining them as narrowly based on helping investors and the upper strata.

Romney will most likely emphasize how his existing strength and experience with business and the economy play to the benefit of all Americans, not just the upper class, and will continue to press home the advantages of his conviction that using the federal government to solve major social and economic problems has liabilities. And Romney will emphasize his perceived advantage on the deficit.
Gallup’s long-term study of data on individual effectiveness has led to the conviction that it is better to build off of one’s strengths than it is to spend most of one’s time focusing on remedying one’s weaknesses. Along these lines, it will be difficult for Romney to transform his image and become “likable” in the course of the debates. Similarly, it will be difficult for Romney to delegitimize Obama’s perceived compassion and likable personality. In theory, Romney could attempt to acknowledge Obama’s compassion for underdogs, while emphasizing that the manifestation of this compassion with a belief in the power of government to solve their plight is not a viable long-term solution. 

Obama would want to come out of the debates having reinforced what he has been doing, playing to his compassion and likability strengths, while continuing to delegitimize Romney’s strengths as an approach that would not fix the economy for everyone, but only for the already privileged. Plus Obama may attempt to build off of his existing strengths on social issues by emphasizing that Romney would be more extreme in his positions on those issues.

Romney would optimally come out of the debate having convinced more Americans of why his economic strengths would play to their benefit, along with some counter effort to delegitimize Obama’s strengths. Romney in particular could play off the general American antipathy toward the federal government.

Each candidate has strengths with certain demographic groups. Obama’s vote coalition is built first and foremost on nonwhites, adding to that voters who are not married, who are not religious, who are young, who have low incomes, and, to a degree, who are women. Obama also has vote strength among those with low levels of education and those with postgraduate degrees. Conversely, Romney’s appeal is to whites in general, but more specifically whites who are older or male, whites who have some college or college degrees but no postgraduate degrees, those in the upper-income ranges, those who are married, and those who are very religious.

In terms of voter segments, Obama’s most obvious target in the debate would be whites with some college or college degrees. Romney’s most obvious expansion out of his base would be into lower-educated whites, whites with postgraduate degrees, and perhaps with one segment of nonwhites where Obama’s advantage is slightly less overwhelming -- Hispanics.

Copyright © 2010 Gallup, Inc. All rights reserved. | Terms of Use | Privacy Statement