STATISTICAL SIGNIFICANCE AND NORMALIZED CONFUSION MATRICES

When assessing map accuracy, confusion matrices are frequently statist ically compared using kappa. While kappa allows individual matrix cate gories to be analyzed with respect to either omission or commission er ror rates, kappa is not used to compare individual matrix categories w ith respect to both rates concurrently. When this concurrent compariso n is desired, the matrices are typically normalized and then scrutiniz ed on a cell-by-cell basis by inspection. While no parametric test of significance exists for such a cell-by-cell examination, sampling dist ributions for these main diagonal entries can be estimated by repeated subsampling of the original sample data (i.e., bootstrapping), allowi ng inferences to be made about the population. In this research, the p rocedure for estimating the sampling distribution of normalized cell v alues is described. Three methods for determining the standard error o f normalized cell value sampling distributions are also outlined. Usin g these sampling distributions and their attendant standard error, the statistical comparison of cell values from two normalized confusion m atrices is illustrated. One illustrated method requires a mild paramet ric assumption, whereas the other is completely nonparametric. Neverth eless, the two distinct bootstrap methods produce nearly identical res ults.