I'm here in Chicago at the 65th annual meeting of the American Association for Public Opinion Research.
People are often surprised when I tell them that many pollsters and survey research professionals study their profession with the same zeal and thoroughness as professional doctors, architects, and engineers.
I say “many.” The pollsters and survey researchers who are members of AAPOR generally consider themselves professionals and have a high regard and respect for what it is they do for a living. By “professionals” I mean people who believe that they work in a field that does good for humanity, and a field with standards and norms of practice.
Other pollsters do not carry the same sense of responsibility. That's why we have bad polls and bad interpretation of polls.
Some AAPOR standards include disclosure of the ways in which surveys are done. The assumption is that details of survey research need to be made available -- as is the case for most scientific or scholarly research -- to be properly evaluated. One does not find medical studies in The New England Journal of Medicine without explicit details on exactly how the study was conducted and very specific details on the findings of the study, often in minute detail. This allows interested researchers to do a better job evaluating the quality of the results.
In survey research this is often not the case. Surveys are conducted and reported with very little detail on where the data came from -- even such rudimentary details as the dates of interviewing, the wording of the questions, the nature of the sample, and so forth. This makes it very difficult to evaluate the quality of the results.
So one of the major discussions at AAPOR over the last number of years has been this whole issue of disclosure. The out-going president of AAPOR, Dr. Peter Miller of Northwestern University, has promulgated an AAPOR Transparency Initiative, designed to allow researchers to voluntarily provide details of their research in a way promotes good practices.
Elsewhere at AAPOR conference, there are dozens and dozens of scholar papers of research in the methods and results of survey practice in the U.S. and around the world.
For your reading enjoyment, here's a sampling of just a few of the papers being given at this conference:
- Surveys on sensitive topics: Racism, intimate partner violence, material mortality
- Collecting biospecimens via surveys
- Polling on a fuzzy issue like the 2009 health care reform debate
- An examination of Internet users in Iraq
- Who are Latino cell phone only respondents?
- An eye-tracking study on survey question comprehension
- An experimental test of the impact of leaving voice messages in cell phone surveys
- Can post-stratification adjustments do enough to reduce bias in telephone surveys that do not sample cell phones?
- Do voters live vicariously through election results?
- Gender of interviewer: How does it influence responses in a gender-focused survey