ON THE USE OF TESTABILITY MEASURES FOR DEPENDABILITY ASSESSMENT

Program ''testability'' is, informally, the probability that a program will fail under test if it contains at least one fault. When a depend ability assessment has to be derived from the observation of a series of failure-free test executions (a common need for software subject to ''ultra-high reliability'' requirements), measures of testability can -in theory-be used to draw inferences on program correctness (and henc e on its probability of failure in operation). In this paper, we rigor ously investigate the concept of testability and its use in dependabil ity assessment, criticizing, and improving on, previously published re sults. We first give a general descriptive model of program execution and testing, on which the different measures of interest can be define d. We propose a more precise definition of program testability than th at given by other authors, and discuss how to increase testing effecti veness without impairing program reliability in operation. We then stu dy the mathematics of using testability to estimate, from test results : 1)the probability of program correctness and 2) the probability of f ailures. To derive the probability of program correctness, we use a Ba yesian inference procedure and argue that this is more useful than der iving a classical ''confidence level.'' We also show that a high testa bility is not an unconditionally desirable property for a program. In particular, for programs complex enough that they are unlikely to be c ompletely fault-free, increasing testability may produce a program whi ch will be less trustworthy, even after successful testing.