After several months without any new version, here is finally pROC version 1.4!

This long-awaited version introduces three new notable features:

• Multi-class AUC support.
• Unpaired bootstrap and DeLong's test.
• Specificity and sensitivity tests (in roc.test).

Multiclass ROC curves

Multi-class AUC is a precious generalization of ROC curves for tests with more than one outcome. We used the approach by Hand and Till (2001)¹ that averages all the ROC curves than can be built with the given levels. Here is a basic example:

library(pROC)
data(aSAH)
multiclass.roc(aSAH$gos6, aSAH$s100b)

See ?multiclass.roc for more details.

Unpaired ROC tests

The second feature is the unpaired DeLong² and bootstrap ROC tests. Until now, only paired ROC curves (when measurements were done on the same sample) were available. Now even unpaired ROC curves are supported.

rocs100 <- roc(aSAH$outcome, aSAH$s100b)
rocwfns <- roc(aSAH$outcome, aSAH$wfns)
roc.test(rocs100, rocwfns, paired=FALSE)

Or with a truly unpaired context:

rocwfns.subset <- roc(aSAH$outcome[1:100], aSAH$wfns[1:100])
roc.test(rocs100, rocwfns.subset)

Specificity and sensitivity tests

The last feature introduced in pROC 1.4 is the specificity and sensitivity tests in roc.test. Instead of testing the AUC, it will test if the ROC curves have the same sensitivity/specificity at the given level of specificity/sensitivity, as described by Pepe et al. (2009)³. Here are two examples:

roc.test(roc1, roc2, method="specificity", specificity=0.9)
roc.test(roc1, roc2, method="sensitivity", sensitivity=0.9)

Conclusion

Here is the full change log:

• 'smooth' does not apply on ordered factors anymore.
• Multi-class AUC support (R only).
• Can choose how best thresold is determined (best.method and best.weights in coords and print.thres.best.method and print.thres.best.weights in plot.roc).
• Minor fixes in documentation (R and S+) and citation (S+ only).
• print now prints the response instead of response and more informative data in htests.
• Bootstrap with ci.auc consumes much less memory.
• Unpaired bootstrap and DeLong's test.
• Specificity and sensitivity tests (in roc.test).

As usual, you can find the new version on ExPASy and on the CRAN. To update, type update.packages() or install.packages("pROC") if you want to update pROC only.

• 1. David J. Hand and Robert J. Till (2001). “A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems“. Machine Learning 45, 171–186. DOI: 10.1023/A:1010920819831.
• 2. Elisabeth R. DeLong, David M. DeLong and Daniel L. Clarke-Pearson (1988) “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach”. Biometrics 44, 837–845.
• 3. Margaret Pepe, Gary Longton and Holly Janes (2009) “Estimation and Comparison of Receiver Operating Characteristic Curves”. The Stata journal 9, 1.