The nature of indexing: how humans and machines analyze messages and textsfor retrieval. Part I: Research, and the nature of human indexing

Does human intellectual indexing have a continuing role to play in the face of increasingly sophisticated automatic indexing techniques? In this two-p art essay, a computer scientist and long-time TREC participant (Perez-Carba llo) and a practitioner and teacher of human cataloging and indexing (Ander son) pursue this question by reviewing the opinions and research of leading experts on both sides of this divide. We conclude that human analysis shou ld be used on a much more selective basis, and we offer suggestions on how these two types of indexing might be allocated to best advantage. Part one of the essay critiques the comparative research, then explores the nature o f human analysis of messages or texts and efforts to formulate rules to mak e human practice more rigorous and predictable. We find that research compa ring human vs automatic approaches has done little to change strongly held beliefs, in large part because many associated variables have not been isol ated or controlled. Part II focuses on current methods in automatic indexing, its gradual adopt ion by major indexing and abstracting services, and ways for allocating hum an and machine approaches. Overall, we conclude that both approaches to ind exing have been found to be effective by researchers and searchers, each wi th particular advantages and disadvantages. However automatic indexing has the over-arching advantage of decreasing cost, as human indexing becomes ev er more expensive. (C) 2001 Elsevier Science Ltd. All rights reserved.