INCORPORATING PRIOR INFORMATION IN MACHINE LEARNING BY CREATING VIRTUAL EXAMPLES

One of the key problems in supervised learning is the insufficient siz e of the training set. The natural way for an intelligent learner to c ounter this problem and successfully generalize is to exploit prior in formation that may be available about the domain dr that can be learne d from prototypical examples. We discuss the notion of using using pri or knowledge by creating virtual examples and thereby expanding the ef fective training-set size. We show that in some contexts this ideals m athematically equivalent to incorporating the prior knowledge as a reg ularizer, suggesting that the strategy is well motivated. The process of creating virtual examples in real-world pattern recognition tasks i s highly nontrivial. We provide demonstrative examples from object rec ognition and speech recognition to illustrate the idea.