Unsupervised learning of word segmentation rules with genetic algorithms and inductive logic programming

This article presents a combination of unsupervised and supervised learning techniques for the generation of word segmentation rules from a raw list o f words. First, a language bias for word se mentation is introduced and a s imple genetic algorithm is used in the search for a segmentation that corre sponds to the best bias value. In the second phase, the words segmented by the genetic algorithm are used as an input for the first order decision lis t learner CLOG. The result is a set of first order rules which can be used for segmentation of unseen words. When applied on either the training data or unseen data, these rules produce segmentations which are linguistically meaningful, and to a large degree conforming to the annotation provided.