Abstract:

The evaluation of classifier performance in a cost-sensitive
setting is straightforward if the operating conditions
(misclassification costs and class distributions) are fixed and
known.  When this is not the case, evaluation requires a method
of visualizing classifier performance across the full range of
possible operating conditions.  This talk briefly reviews the
classic technique for classifier performance visualization --
the ROC curve -- and argues that it is inadequate for the needs
of researchers and practitioners in several important respects.
It then shows that a different way of visualizing classifier
performance -- the cost curve introduced by Drummond and Holte
at KDD'2000 -- overcomes these deficiencies. No familiarity with
ROC curves or cost curves is necessary, they will be fully
explained.

Joint work with Chris Drummond (National Research Council,
Ottawa)