T. Kanungo et al., A METHODOLOGY FOR QUANTITATIVE PERFORMANCE EVALUATION OF DETECTION ALGORITHMS, IEEE transactions on image processing, 4(12), 1995, pp. 1667-1674
We present a methodology for the quantitative performance evaluation o
f detection algorithms in computer vision. A common method is to gener
ate a variety of input images by varying the image parameters and eval
uate the performance of the algorithm, as algorithm parameters vary. O
perating curves that relate the probability of misdetection and false
alarm are generated for each parameter setting. Such an analysis does
not integrate the performance of the numerous operating curves. In thi
s paper, we outline a methodology for summarizing many operating curve
s into a few performance curves. This methodology is adapted from the
human psychophysics literature and is general to any detection algorit
hm. The central concept is to measure the effect of variables in terms
of the equivalent effect of a critical signal variable, which in turn
facilitates the determination of the breakdown point of the algorithm
. We demonstrate the methodology by comparing the performance of two-l
ine detection algorithms.