Formal grammar and information theory: together again?

In the last 40 years, research on models of spoken and written language has been split between two seemingly irreconcilable traditions: formal linguis tics in the Chomsky tradition, and information theory in the Shannon tradit ion. Zellig Harris had advocated a close alliance between grammatical and i nformation-theoretic principles in the analysis of natural language, and ea rly formal-language theory provided another strong link between information theory and linguistics. Nevertheless, in most research on language and com putation, grammatical and information-theoretic approaches had moved far ap art. Today, after many years on the defensive, the information-theoretic approac h has gained new strength and achieved practical successes in speech recogn ition, information retrieval, and, increasingly, in language analysis and m achine translation. The exponential increase in the speed and storage capac ity of computers is the proximate cause of these engineering successes, all owing the automatic estimation of the parameters of probabilistic models of language by counting occurrences of linguistic events in very large bodies of text and speech. However, I will argue that information-theoretic and c omputational ideas are also playing an increasing role in the scientific un derstanding of language, and will help bring together formal-linguistic and information-theoretic perspectives.