Measuring the effectiveness of reliability growth testing

Assessing the effectiveness of reliability growth testing allows decisions to be made about the management of the programme; for example, decisions to allocate resources in an attempt to realize improvements in system reliabi lity, or decisions to terminate testing when there is evidence that reliabi lity has matured. However, such assessments are commonly based on measures of change in the average time between failures, which do not always provide informative measures of effectiveness. We argue that a more appropriate ap proach is to focus on the identification and realization of faults, which i f removed are likely to lead to the improvement in system reliability. Ther efore we introduce a model that incorporates the concerns of engineering ex perts about the likely faults in the initial system design with the informa tion about faults that are realized on test. This model distinguishes betwe en the processes of detecting and removing faults and so captures the effec tiveness of test as well as the development of the system design. Using thi s model, the likely number of faults remaining in the system and the additi onal test time required to realize them can be estimated. This model was mo tivated by and assessed by engineers conducting growth tests for complex el ectronic systems. They consider our measures useful for supporting assessme nt of test effectiveness. Copyright (C) 1999 John Wiley & Sons, Ltd.