The nature of indexing: how humans and machines analyze messages and textsfor retrieval. Part II: Machine indexing, and the allocation of human versus machine effort

Does human intellectual indexing have a continuing role to play in the face of increasingly sophisticated automatic indexing techniques? In this two-p art essay, a computer scientist and long-time TREC participant (Perez-Carba llo) and a practitioner and teacher of human cataloging and indexing (Ander son) pursue this question by reviewing the opinions and research of leading experts on both sides of this divide. We conclude that human analysis shou ld be used on a much more selective basis, and we offer suggestions on how these two types indexing might be allocated to best advantage. Part I of th e essay critiques the comparative research, then explores the nature of hum an analysis of messages or texts and efforts to formulate rules to make hum an practice more rigorous and predictable. We find that research comparing human versus automatic approaches has done little to change strongly held b eliefs, in large part because many associated variables have not been isola ted or controlled. Part II focuses on current methods in automatic indexing, its gradual adopt ion by major indexing and abstracting services, and ways for allocating hum an and machine approaches. Overall, we conclude that both approaches to ind exing have been found to be effective by researchers and searchers, each wi th particular advantages and disadvantages. However, automatic indexing has the over-arcking advantage of decreasing cost, as human indexing becomes e ver more expensive. (C) 2001 Elsevier Science Ltd. All rights reserved.