Is proof more cost-effective than testing?

This paper describes the use of formal development methods on an industrial safety-critical application. The Z notation was used for documenting the s ystem specification and part of the design, and the SPARK(1) subset of Ada was used for coding. However, perhaps the most distinctive nature of the pr oject lies in the amount of proof that was carried out: proofs were carried out both at the Z level-approximately 150 proofs in 500 pages-and at the S PARK code level-approximately 9,000 verification conditions generated and d ischarged. The project was carried out under UK Interim Defence Standards 0 0-55 and 00-56, which require the use of formal methods on safety-critical applications. It is believed to be the first to be completed against the ri gorous demands of the 1991 version of these standards. The paper includes c omparisons of proof with the various types of testing employed, in terms of their efficiency at finding faults. The most striking result is that the Z proof appears to be substantially more efficient at finding faults than th e most efficient testing phase. Given the importance of early fault detecti on, we believe this helps to show the significant benefit and practicality of large-scale proof on projects of this kind.