NEURAL NETWORKS, RATIONAL FUNCTIONS, AND REALIZATION-THEORY

The problem of parametrizing single hidden layer scalar neural network s with continuous activation functions is investigated. A connection i s drawn between realization theory for linear dynamical systems, ratio nal functions, and neural networks that appears to be new. A result of this connection is a general parametrization of such neural networks in terms of strictly proper rational functions. Some existence and uni queness results are derived. Jordan decompositions are developed, whic h show how the general form can be expressed in terms of a sum of cano nical second order sections. The parametrization may be useful for stu dying learning algorithms.