S. Nolfi et J. Tani, Extracting regularities in space and time through a cascade of prediction networks: The case of a mobile robot navigating in a structured environment, CONNECT SCI, 11(2), 1999, pp. 125-148
We propose that the ability to extract regularities from time series throug
h prediction learning can be enhanced if we use a hierarchical architecture
in which higher layers are trained to predict the internal state of lower
layers when such states change significantly. This hierarchical organizatio
n has two functions: (a) it forces the system to recode sensory information
progressively so as to enhance useful regularities and filter out useless
information; and (b) it progressively reduces the length of the sequences w
hich should be predicted going from lower to higher layers. This, in turn,
allows higher levels to extract higher-level regularities which are hidden
at the sensory level. By training an architecture of this type to predict t
he next sensory state of a robot navigating in an environment divided into
two rooms, toe show how the first-level prediction layer extracts low-level
regularities such as 'walls', 'corners' and 'corridors', while the second-
level prediction layer extracts higher-level regularities such as 'the left
side wall of the large room'. The extraction of these regularities allows
the robot to localize its position in the environment and to detect changes
in the environment (e.g, the presence of a new object or the fact that a d
oor has been closed).