Evaluating emerging software development technologies: Lessons learned from assessing aspect-oriented programming

Determining whether a new software development technique is useful and usab le is a challenging task. Various flavors of empirical study may be used to help with this task, including surveys, case studies, and experiments. Lit tle guidance is available within the software engineering community to help choose among these alternatives when assessing a new and evolving software development technique within some cost bounds. We faced this challenge whe n assessing a new programming technique called aspect-oriented programming. To assess the technique, we chose to apply both a case study approach and a series of four experiments because we wanted to understand and characteri ze the kinds of information that each approach might provide. In this paper , we describe and critique the evaluation methods we employed, and discuss the lessons we have learned. These lessons are applicable to other research ers attempting to assess new programming techniques that are in an early st age of development.