Quality assessment of image classification algorithms for land-cover mapping: a review and a proposal for a cost-based approach

The following issues relate to quality assessment of image classification: the classification methods as such, the methods to evaluate the classificat ion results, and the requirements of the application. In this paper, a numb er of evaluation methods are reviewed, and it is concluded that those based on confusion matrices and the KHAT analysis are the most suited if one is interested in comparing classifiers. The novelty of this paper is that much attention is given to the subjectivity present in every evaluation scheme, and that the concept of accuracy is extended to quality by creating the li nk between accuracy, objectives, and costs. A protocol is proposed for qual ity assessment related to the economical reality. An example based on a hyp othetical data set shows that the economic cost of misclassification can be high, and that it may be advantageous for the user to reconsider either th e objectives, the type of data used, or other aspects of the remote-sensing system that he uses to produce the map.