Coverage estimation methods for stratified fault-injection

This paper addresses the problem of estimating fault tolerance coverage thr ough statistical processing of observations collected in fault-injection ex periments. In an earlier paper, various estimators based on simple sampling in the complete fault/activity input space and stratified sampling in a pa rtitioned space were studied; frequentist confidence limits were derived ba sed on a normal approximation. In this paper, the validity of this approxim ation is analyzed. The theory of confidence regions is introduced to estima te coverage without approximation when stratification is used. Three statis tics are considered for defining confidence regions. It is shown that one-a vectorial statistic-is often more conservative than the other two. However , only the vectorial statistic is computationally tractable. We then consid er Bayesian estimation methods for stratified sampling. Two methods are pre sented to obtain an approximation of the posterior distribution of the cove rage by calculating its moments. The moments are then used to identify the type of the distribution in the Pearson distribution system, to estimate it s parameters, and to obtain the coverage confidence limit. Three hypothetic al example systems are used to compare the validity and the conservatism of the frequentist and Bayesian estimations.