An investigation of machine learning based prediction systems

Traditionally, researchers have used either off-the-shelf models such as CO COMO, or developed local models using statistical techniques such as stepwi se regression, to obtain software effort estimates. More recently, attentio n has turned to a variety of machine learning methods such as artificial ne ural networks (ANNs), case-based reasoning (CBR) and rule induction (RI). T his paper outlines some comparative research into the use of these three ma chine learning methods to build software effort prediction systems. We brie fly describe each method and then apply the techniques to a dataset of 81 s oftware projects derived from a Canadian software house in the late 1980s. We compare the prediction systems in terms of three factors: accuracy, expl anatory value and configurability. We show that ANN methods have superior a ccuracy and that RI methods are least accurate. However, this view is somew hat counteracted by problems with explanatory value and configurability. Fo r example, we found that considerable effort was required to configure the ANN and that this compared very unfavourably with the other techniques, par ticularly CBR and least squares regression (LSR). We suggest that Further w ork be carried out, both to further explore interaction between the end-use r and the prediction system, and also to facilitate configuration, particul arly of ANNs. (C) 2000 Elsevier Science Inc. All rights reserved.