We argue that existing learning algorithms are often poorly equipped t
o solve problems involving a certain type of important and widespread
regularity that we call ''type-2 regularity''. The solution in these c
ases is to trade achieved representation against computational search.
We investigate several ways in which such a trade-off may be pursued
including simple incremental learning, modular connectionism, and the
developmental hypothesis of ''representational redescription''.