METHODOLOGICAL ISSUES IN JUDGMENT AND DECISION-MAKING RESEARCH - CONCURRENT VERBAL PROTOCOL VALIDITY AND SIMULTANEOUS TRACES OF PROCESS

This paper examines two dimensions of concurrent verbal protocol valid ity. First, whether verbalization affects process and outcome (reactiv ity) was examined by comparing concurrent verbal protocol traces with those from a computer search process tracing method, the latter being a complete trace of information acquisition from experimental material s. Earlier findings that verbalization affects time were confirmed. Ho wever, verbalization did not affect amount and pattern of acquisition or accuracy of judgments. Second, whether concurrent verbal protocols are complete was examined by comparing concurrent verbal protocol and computer traces that were simultaneously obtained in a treatment in wh ich subjects verbalized as they acquired information from the computer . The verbal traces less completely captured information acquisition b ehavior than computer search. This suggests that, although concurrent verbal protocols provide greater insight into decision behavior than c omputer search, the latter is a more reliable information-acquisition trace. Thus, if information acquisition is of primary interest and if computer search activities can be naturally integrated into performing the primary task, computer search is preferred to concurrent verbal p rotocols. However, if information use or retrieval from long-term memo ry is of primary interest, concurrent verbal protocols are preferred t o computer search. Finally, this paper examined whether the simultaneo us use of concurrent verbal protocols and computer search provides tra ces of information acquisition and use that are as complete as when ea ch method is independently applied. Although computer search tended to limit subjects' verbalizations of evaluative operators, this effect m ay be eliminated by practice on the computer prior to collecting data.