TERM DEPENDENCE - TRUNCATING THE BAHADUR LAZARSFELD EXPANSION

The performance of probabilistic information retrieval systems is stud ied where differing statistical dependence assumptions are used when e stimating the probabilities inherent in the retrieval model. Experimen tal results using the Bahadur Lazarsfeld expansion suggest that the gr eatest degree of performance increase is achieved by incorporating ter m dependence information in estimating Pr (d\rel). It is suggested tha t incorporating dependence in Pr (d\rel) to degree 3 be used; incorpor ating more dependence information results in relatively little increas e in performance. Experiments examine the span of dependence in natura l language text, the window of terms in which dependencies are compute d, and their effect on information retrieval performance. Results prov ide additional support for the notion of a window of +/- 3 to +/- 5 te rms in width; terms in this window may be most useful when computing d ependence.