RECURRENT NEURAL NETWORKS AND PRIOR KNOWLEDGE FOR SEQUENCE PROCESSING- A CONSTRAINED NONDETERMINISTIC APPROACH

The paper focuses on methods for injecting prior knowledge into adapti ve recurrent networks for sequence processing. In order to increase th e flexibility needed for specifying partially known rules, a nondeterm inistic approach for modelling domain knowledge is proposed. The algor ithms presented in the paper allow time-warping nondeterministic autom ata to be mapped into recurrent architectures with first-order connect ions. These kinds of automata are suitable for modeling temporal scale distortions in data such as acoustic sequences occurring in problems of speech recognition. The algorithms output a recurrent architecture and a feasible region in the connection weight space. It is demonstrat ed that, as long as the weights are constrained into the feasible regi on, the nondeterministic rules introduced using prior knowledge are no t destroyed by learning. The paper focuses primarily on architectural issues, but the proposed method allows the connection weights to be su bsequently tuned to adapt the behavior of the network to data.