EMPIRICALLY DESIGNING AND EVALUATING A NEW REVISION-BASED MODEL FOR SUMMARY GENERATION

We present a system for summarizing quantitative data in natural langu age, focusing on the use of a corpus of basketball game summaries, dra wn from on-line news services, to empirically shape the system design and to evaluate our approach. Our initial corpus analysis revealed cha racteristics of textual summaries that challenge the capabilities of c urrent language generation systems. In order to meet these challenges, we developed a revision-based model for summary generation and implem ented it in our prototype system STREAK. A second, detailed corpus ana lysis was used to identify and encode the revision rules of the system . Finally, we carried out a quantitative evaluation, using several tes t corpora, to measure the robustness of the new revision-based model. Our results show that our new model improves both coverage and extensi bility of the traditional language generation model.