The local minima of the error surface of the 2-2-1 XOR network

All local minima of the error surface of the 2-2-1 XOR network are describe d. A local minimum is defined as a point such that all points in a neighbou rhood have an error value greater than or equal to the error value in that point. It is proved that the error surface of the two-layer XOR network wit h two hidden units has a number of regions with local minima. These regions of local minima occur for combinations of the weights from the inputs to t he hidden nodes such that one or both hidden nodes are saturated for at lea st two patterns. However, boundary points of these regions of local minima are saddle points. It will be concluded that from each finite point in weig ht space a strictly decreasing path exists to a point with error zero. This also explains why experiments using higher numerical precision find less " local minima".