Randomized testing of alternative survey formats using anonymous volunteers on the World Wide Web

Consenting visitors to a health survey Web site were randomly assigned to a "matrix" presentation or an "expanded" presentation of survey response opt ions. Among 4,208 visitors to the site over 3 months, 1,615 (38 percent) pa rticipated by giving consent and completing the survey. During a pre-trial period, when consent was not required, 914 of 1,667 visitors (55 percent) p articipated (odds ratio 1.9, P < 0.0001). Mean response times were 5.07 min utes for the matrix format and 5.22 minutes for the expanded format (P = 0. 16). Neither health status scores nor alpha reliability coefficients were s ubstantially influenced by the survey format, but health status scores vari ed with age and gender as expected from U.S. population norms. In conclusio n, presenting response options in a matrix format may not substantially spe ed survey completion. This study demonstrates a method for rapidly evaluati ng interface design alternatives using anonymous Web volunteers who have pr ovided informed consent.