INPUT-OUTPUT HMMS FOR SEQUENCE PROCESSING

We consider problems of sequence processing and propose a solution bas ed on a discrete-state model in order to represent past context, We in troduce a recurrent connectionist architecture having a modular struct ure that associates a subnetwork to each state, The model has a statis tical interpretation we call input-output hidden Markov model (IOHMM). It can be trained by the estimation-maximization (EM) or generalized EM (GEM) algorithms, considering state trajectories as missing data, w hich decouples temporal credit assignment and actual parameter estimat ion, The model presents similarities to hidden Markov models (HMM's), but allows us to map input sequences to output sequences, using the sa me processing style as recurrent neural networks. IOHMM's are trained using a more discriminant learning paradigm than HMM's, while potentia lly taking advantage of the EM algorithm. We demonstrate that IOHMM's are well suited for solving grammatical inference problems on a benchm ark problem, Experimental results are presented for the seven Tomita g rammars, showing that these adaptive models can attain excellent gener alization.