THE VERY DIFFERENT METHODS USED TO CONDUCT TELEPHONE SURVEYS OF THE PUBLIC

This review of the methods used by 83 leading marketing research firms in 17 countries shows enormous differences in the ways they design an d conduct telephone surveys of the public to obtain information about the population. While the biggest differences are between countries, t here are also many differences within countries. Indeed, no two firms, of the 83 surveyed, use identical methods. The primary purpose of thi s review is to provide comparative information and stimulate more disc ussion about different methodologies, not to criticise the methods use d. However, some of these are clearly open to criticism and are hard t o defend. Furthermore, the survey suggests that many of these firms ha ve not given much thought to the possible weaknesses of the methods th ey use or whether they should improve them. Overall, the most striking finding was the complete absence of consensus on almost all aspects o f sampling and weighting. Specifically: Four out often firms describe their methods as 'quota sampling,' 45% as 'probability sampling with w eighting'. Half use random digit dialling; half do not. No one method of selecting the individual within the household is used by more than the one-third, who say they use (a quota) to do this. The number of ca ll-backs usually made to telephone numbers which do not reply varies f rom none to more than ten. Approximately half the firms substitute num bers within geographic cells for non-response; the rest do not. Most f irms (including those that describe their methods as 'probability samp ling') use quotas to control their samples while they are in the field . The controls used vary greatly. Claimed refusal rates vary from less than 10% to over 60%. Some firms do not weight their telephone survey data at all; less than one-third always weight them. The variables us ed to weight telephone survey data also vary widely.