Subsampling callbacks to improve survey efficiency

The National Comorbidity Survey (NCS), a 1990-1992 nationwide face-to-fate survey of the U.S. population age 15-54 regarding the prevalence of psychia tric comorbidity, required, as with most large-scale surveys, multiple call backs to maximize response rates. As a consequence, it appeared that a subs tantial proportion of the data collection cost came from the many callbacks required to obtain a small proportion of interviews with difficult-to-reac h respondents. Thus we were led to explore whether the efficiency of data c ollection could be increased by subsampling a random proportion cr of the o riginally sampled units from the mth callback attempt forward, with case we ights used to remove bias. Similar considerations led the designers of the American Community Survey (ACS) to subsample households in the face-to-face portion of a three-mode postal, telephone, and face-to-face survey. In thi s article we determine the expected cost of a given interview under two str ategies-subsampling versus no subsampling-as a function of the probability of obtaining an interview at the kth callback, the cost of the kth callback , and the cost of an interview or cost of a refusal if obtained at the kth callback. We then determine the sample size required to maintain a constant variance of a mean estimated under the two sampling strategies. We give an efficiency ratio as the ratio of the total costs under the two sampling st rategies. This ratio is minimized as a function of m (the callback on which subsampling begins) and ct (the proportion of sampling units retained) und er the assumed cost and probability-of-interview structure. We then show th at subsampling becomes a potentially efficient strategy whenever (a) the pe r callback or per interview cost is increasing and (b) the probability of a successful interview attempt is decreasing. Although these conditions are routinely met in sample surveys, it appears that they must be met to a subs tantial degree for useful savings to occur. In the case of the NCS, we foun d that only trivial savings could be achieved. Although unfortunate in one sense, this finding is useful in proving that a strategy that seemed intuit ively reasonable offers only very modest practical benefit. Subsampling in the ACS seems to be more cost-effective; however, our analysis suggests tha t subsampling should begin at the telephone rather than the face-to-face ma de.