Buying with Exact Confidence

We derive some results which may be helpful to buyers of software testing for faults, or to buyers of large lots screening for defectives. Suppose that a fixed but unknown number n of faults or defectives remain before testing. In the testing phase they are observed at random times, X1,X2,.,Xn, which are order statistics corresponding to n i.i.d. random variables. Since testing is usually an ongoing activity, this distribution is typically known. Under this assumption we derive a stopping criterion that guarantees, for any specified level . and integer m, that for all n>m, with probability exactly 1.., when stopping occurs, the software has no more than m faults remaining. We study various properties of this stopping rule, both finite and asymptotic, and show that it is optimal in a certain sense. We modify a conservative stopping rule proposed by Marcus and Blumenthal to make it exact, and we give some numerical comparisons.