pROC 1.4 is here
After several months without any new version, here is finally pROC version 1.4!
This long-awaited version introduces three new notable features:
- Multi-class AUC support.
- Unpaired bootstrap and DeLong's test.
- Specificity and sensitivity tests (in
roc.test
).
Multiclass ROC curves
Multi-class AUC is a precious generalization of ROC curves for tests with more than one outcome. We used the approach by Hand and Till (2001)1 that averages all the ROC curves than can be built with the given levels. Here is a basic example:
library(pROC) data(aSAH) multiclass.roc(aSAH$gos6, aSAH$s100b)
See ?multiclass.roc
for more details.
Unpaired ROC tests
The second feature is the unpaired DeLong2 and bootstrap ROC tests. Until now, only paired ROC curves (when measurements were done on the same sample) were available. Now even unpaired ROC curves are supported.
rocs100 <- roc(aSAH$outcome, aSAH$s100b) rocwfns <- roc(aSAH$outcome, aSAH$wfns) roc.test(rocs100, rocwfns, paired=FALSE)
Or with a truly unpaired context:
rocwfns.subset <- roc(aSAH$outcome[1:100], aSAH$wfns[1:100]) roc.test(rocs100, rocwfns.subset)
Specificity and sensitivity tests
The last feature introduced in pROC 1.4 is the specificity and sensitivity tests in roc.test
. Instead of testing the AUC, it will test if the ROC curves have the same sensitivity/specificity at the given level of specificity/sensitivity, as described by Pepe et al. (2009)3. Here are two examples:
roc.test(roc1, roc2, method="specificity", specificity=0.9) roc.test(roc1, roc2, method="sensitivity", sensitivity=0.9)
Conclusion
Here is the full change log:
- 'smooth' does not apply on
ordered
factors anymore. - Multi-class AUC support (R only).
- Can choose how
best
thresold is determined (best.method
andbest.weights
incoords
andprint.thres.best.method
andprint.thres.best.weights
inplot.roc
). - Minor fixes in documentation (R and S+) and
citation
(S+ only). print
now prints the response instead of response and more informative data inhtest
s.- Bootstrap with
ci.auc
consumes much less memory. - Unpaired bootstrap and DeLong's test.
- Specificity and sensitivity tests (in
roc.test
).
As usual, you can find the new version on ExPASy and on the CRAN. To update, type update.packages()
or install.packages("pROC")
if you want to update pROC only.
- 1. David J. Hand and Robert J. Till (2001). “A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems“. Machine Learning 45, 171–186. DOI: 10.1023/A:1010920819831.
- 2. Elisabeth R. DeLong, David M. DeLong and Daniel L. Clarke-Pearson (1988) “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach”. Biometrics 44, 837–845.
- 3. Margaret Pepe, Gary Longton and Holly Janes (2009) “Estimation and Comparison of Receiver Operating Characteristic Curves”. The Stata journal 9, 1.
Xavier Robin
Publié le vendredi 21 janvier 2011 à 20:38 CET
Lien permanent : /blog/2011/01/21/proc-1.4-released
Tags :
pROC
Commentaires : 0