ASSESSING RESEARCH IMPACT - SEMIQUANTITATIVE METHODS

Three types of semiquantitative methods used by the federal government in research impact assessment are presented. These include the classi c retrospective method (Project Hindsight), another retrospective appr oach (Project TRACES and follow-ons) and accomplishments books used by selected research-sponsoring organizations (Office of Naval Research, Air Force Office of Scientific Research, Department of Energy Office of Health and Environmental Research, Department of Energy High Energy Physics Program Defense Advanced Research Projects Agency). The stren gths and weaknesses of each approach are discussed. One goal of all th e studies presented was to identify the products of research and some of their impacts. In addition, the Hindsight, the technology in retros pect and critical events in science (TRACES), and the Defense Advanced Research Projects Agency (DARPA) studies tried to identify factors th at influenced the productivity and impact of research. The following g eneral conclusions about the role and impact of basic research were re ached: (1) the majority of basic research events that directly affecte d technologies or systems were nonmission oriented and occurred many d ecades before the technology or the system emerged; (2) the cumulative indirect impacts of basic research were not accounted for by any of t he retrospective approaches published; (3) an advanced pool of knowled ge must be developed in many fields before synthesis leading to an inn ovation can occur; and (4) allocation of benefits among researchers, o rganizations, and funding agencies to determine economic returns from basic research is very difficult and arbitrary especially at the micro level.