A comparative study of coarse- and fine-grained safe regression test-selection techniques

Regression test-selection techniques reduce the cost of regression testing by selecting a subset of an existing test suite to use in retesting a modif ied program. Over the past two decades, numerous regression test-selection techniques have been described in the literature. Initial empirical studies of some of these techniques have suggested that they can indeed benefit te sters, but so far, few studies have empirically compared different techniqu es. In this paper, we present the results of a comparative empirical study of two safe regression test-selection techniques. The techniques we studied have been implemented as the tools DejaVu and TestTube; we compared these tools in terms of a cost model incorporating precision (ability to eliminat e unnecessary test cases), analysis cost, and test execution cost. Our resu lts indicate, that in many instances, despite its relative lack of precisio n, TestTube can reduce the time required for regression testing as much as the more precise DejaVu. In other instances, particularly where the time re quired to execute test cases is long, DejaVu's superior precision gives it a clear advantage over TestTube. Such variations in relative performance ca n complicate a tester's choice of which tool to use. Our experimental resul ts suggest that a hybrid regression test-selection tool that combines featu res of TestTube and DejaVu may be an answer to these complications; we pres ent an initial case study that demonstrates the potential benefit of such a tool.