Stationary and integrated autoregressive neural network processes

We consider autoregressive neural network (AR-NN) processes driven by addit ive noise and demonstrate that the characteristic roots of the shortcuts-th e standard conditions from linear time-series analysis-determine the stocha stic behavior of the overall AR-NN process. If all the characteristic roots are outside the unit circle, then the process is ergodic and stationary. I f at least one characteristic root lies inside the unit circle, then the pr ocess is transient. AR-NN processes with characteristic roots lying on the unit circle exhibit either ergodic, random walk, or transient behavior. We also analyze the class of integrated AR-NN (ARI-NN) processes and show that a standardized ARI-NN process "converges" to a Wiener process. Finally, le ast-squares estimation (training) of the stationary models and testing for nonstationarity is discussed. The estimators are shown to be consistent, an d expressions on the limiting distributions are given.