A BAYESIAN-APPROACH TO MODEL SELECTION IN HIERARCHICAL MIXTURES-OF-EXPERTS ARCHITECTURES

There does not exist a statistical model that shows good performance o n all tasks. Consequently, the model selection problem is unavoidable; investigators must decide which model is best at summarizing the data for each task of interest. This article presents an approach to the m odel selection problem in hierarchical mixtures-of-experts architectur es. These architectures combine aspects of generalized linear models w ith those of finite mixture models in order to perform tasks via a rec ursive ''divide-and-conquer'' strategy. Markov chain Monte Carlo metho dology is used to estimate the distribution of the architectures' para meters. One part of our approach to model selection attempts to estima te the worth of each component of an architecture so that relatively u nused components can be pruned from the architecture's structure. A se cond part of this approach uses a Bayesian hypothesis testing procedur e in order to differentiate inputs that carry useful information from nuisance inputs. Simulation results suggest that the approach presente d here adheres to the dictum of Occam's razor; simple architectures th at are adequate for summarizing the data are favored over more complex structures. (C) 1997 Elsevier Science Ltd. All Rights Reserved.