Consequences of reducing nonresponse in a national telephone survey

Critics of public opinion polls often claim that methodological shortcuts t aken to collect timely data produce biased results. This study compares two random digit dial national telephone surveys that used identical questionn aires but very different levels of effort: a "Standard" survey conducted ov er a 5-day period that used a sample of adults who were home when the inter viewer called, and a "Rigorous" survey conducted over an 8-week period that used random selection from among all adult household members. Response rat es, computed according to AAPOR guidelines, were 60.6 percent for the Rigor ous and 36.0 percent for the Standard study. Nonetheless, the two surveys p roduced similar results. Across 91 comparisons, no difference exceeded 9 pe rcentage points, and the average difference was about 2 percentage points. Most of the statistically significant differences were among demographic it ems. Very few significant differences were found on attention to media and engagement in politics, social trust and connectedness, and most social and political attitudes, including even those toward surveys.