APPLICATION OF PROBABILISTIC METHODS TO CHINESE TEXT RETRIEVAL

The use of text retrieval methods based on the probabilistic model wit h Chinese language material is discussed. Since Chinese text has no na tural word boundaries, we must either apply a dictionary-based word se gmentation method to the text, or index and search in terms of single Chinese characters. In either case, it becomes important to have a goo d way of dealing with phrases or contiguous strings of characters; the probabilistic model does not at present have such a facility. Some ad hoc modifications of the probabilistic weighting function and matchin g method are proposed for this purpose.