Assessing the accuracy of prediction algorithms for classification: an overview

We provide a unified overview of methods that currently are widely used to assess the accuracy of prediction algorithms, from raw percentages, quadrat ic error measures and other distances, ann correlation coefficients, and to information theoretic measures such as relative entropy and mutual informa tion. We briefly discuss the advantages and disadvantages of each approach. For classification tasks, we derive new learning algorithms for the design of prediction systems by directly optimising the correlation coefficient. We observe and prove several results relating sensitivity nod specificity o f optimal systems. While the principles are general, we illustrate the appl icability on specific problems such as protein secondary structure and sign al peptide prediction.