This article describes an assessment carried our in collaboration with the
administrators of a large freshman English course. The assessment team work
ed with instructors to identify course goals and to design tasks that the i
nstructors felt would fairly assess the extent to which the students achiev
ed the goals. Students who did and did not take the course were both pre- a
nd posttested on five central goals: critical reading, argument identificat
ion, differentiation of summary and paraphrase, understanding of key terms
used in the course, and practical strategies for writing academic papers. R
esults of the assessment failed to indicate any substantial improvement on
any of the five course goals for students who took the course. These result
s contrasted with positive outcomes obtained by the same assessment team wi
th introductory history and statistics courses. The article concludes with
reflections on why instructors may fail to recognize that their courses are
not working.