This paper examines the inductive inference of a complex grammar with neura
l networks-specifically, the task considered is that of training a network
to classify natural language sentences as grammatical or ungrammatical. the
reby exhibiting the same kind of discriminatory power provided by the Princ
iples and Parameters linguistic framework, or Government-and-Binding theory
. Neural networks are trained, without the division into learned vs. innate
components assumed by Chomsky, in an attempt to produce the same judgments
as native speakers on sharply grammatical/ungrammatical data. How a recurr
ent neural network could possess linguistic capability and the properties o
f various common recurrent neural network architectures are discussed. The
problem exhibits training behavior which is often not present with smaller
grammars and training was initially difficult. However, after implementing
several techniques aimed at improving the convergence df the gradient desce
nt backpropagation-through-time training algorithm, significant learning wa
s possible. It was found that certain architectures are better able to lear
n an appropriate grammar. The operation of the networks and their training
is analyzed. Finally, the extraction of rules in the form of deterministic
finite state automata is investigated.