SEMIQUANTITATIVE METHODS FOR RESEARCH IMPACT ASSESSMENT

Three types of semiquantitative methods used by the federal government in research impact assessment are presented. These include the classi c retrospective method (Project Hindsight), another retrospective appr oach (Project TRACES and follow-ons), and accomplishments books used b y selected research-sponsoring organizations (Office of Naval Research , Air Force Office of Scientific Research, Department of Energy Office of Health and Environmental Research, Department of Energy High Energ y Physics Program, Defense Advanced Research Projects Agency). The str engths and weaknesses of each approach are discussed. One goal of all the studies presented was to identify the products of research and som e of their impacts. In addition, the Hindsight, TRACES, and DARPA stud ies tried to identify factors that influenced the productivity and imp act of research. The following general conclusions about the role and impact of basic research were reached: 1. The majority of basic resear ch events that directly impacted technologies or systems were non-miss ion-oriented and occurred many decades before the technology or system emerged. 2. The cumulative indirect impacts of basic research were no t accounted for by any of the retrospective approaches published. 3. A n advanced pool of knowledge must be developed in many fields before s ynthesis leading to an innovation can occur. 4. Allocation of benefits among researchers, organizations, and funding agencies to determine e conomic returns from basic research is very difficult and arbitrary, e specially at the micro level.