Two years after the last major release 1.5, pROC 1.6 is finally available. It comes with several major enhancements:

• Power ROC tests
• Confidence intervals for arbitrary coordinates
• Speed enhancements
• Dropped S+ support
• Other changes

Power ROC tests

This is probably the main feature of this version: power tests for ROC curves. It is now possible to compute sample size, power, significance level or minimum AUC with pROC.

library(pROC)
data(aSAH)

roc1 <- roc(aSAH$outcome, aSAH$ndka)
roc2 <- roc(aSAH$outcome, aSAH$wfns)

power.roc.test(roc1, roc2, power=0.9)

It is implemented with the methods proposed by Obuchowski and colleagues^{1, 2}, with the added possibility to use bootstrap or the DeLong³ method to compute variance and covariances. For more details and examples, see ?power.roc.test.

As a side effect, a new method="obuchowski" has been implemented in the cov and var functions. More details in ?var.roc and ?cov.roc.

Confidence intervals for arbitrary coordinates

It is now possible to compute confidence intervals of arbitrary coordinates, with a syntax much similar to that of the coords function.

library(pROC)
data(aSAH)

ci.coords(aSAH$outcome, aSAH$s100b, x="best")

# Or for much more information:
rets <- c("threshold", "specificity", "sensitivity", "accuracy", "tn", "tp", "fn", "fp", "npv", 
          "ppv", "1-specificity", "1-sensitivity", "1-accuracy", "1-npv", "1-ppv")
ci.coords(aSAH$outcome, aSAH$wfns, x=0.9, input = "sensitivity", ret=rets)

Speed enhancements

• A faster implemententation of the DeLong test was kindly contributed by Kazuki Yoshida. It is used in roc.test, ci, var and cov.
• Two new algorithms have been introduced to speed-up ROC analysis, and specifically the computation of sensitivity and specificity. The same code as before is used by default (algorithm=1), that goes in O(T*N) (N = number of data points and T = number of thresholds of the curve), is well tested and safe. If speed is an issue for you, you may want to consider the following alternatives:
  • algorithm=2 is a pure-R algorithm that goes in O(N) instead of O(T*N). It is typically faster when the number of thresholds of the ROC curve is above 1000, but slower otherwise.
  • algorithm=3 is a a C++ implementation of the standard algorithm of pROC, with a 3-5x speedup. It is typically the fastest for ROC curves with less than 3000-5000 thresholds.
  • The special values 0 means the fastest algorithm for the specific dataset will be determined with the microbenchmark package, while 4 is a debug feature that tests all 3 algorithms and ensures they produce the same results.

NOTE: because of this change, roc objects created with an earlier version will have to be re-created before they can be used in any bootstrap operation.

Dropped S+ support

S+ support was dropped, due to diverging code bases and apparent drop of support of S+ by TIBCO. A version 1.5.9 will be released in the next few days on ExPaSy with an initial work on ROC tests. It will work only on 32bits versions of S+ 8.2 for Windows.

Other changes

• coords (and ci.coords) now accepts a new ret value "1-accuracy"
• are.paired now also checks for identical levels
• Fixed a warning generated in the examples
• Fixed several bugs related with smooth.roc curves
• Additional input data sanity checks
• Now requires R >= 2.13 (in fact, since 1.5.1, thanks Emmanuel Curis for the report)
• Progress bars now defaults to text on Macs where 'tcltk' seems broken (thanks Gerard Smits for the report)

As usual, you will find the new version on ExPASy (please give a few days for the update to be propagated there) and on the CRAN. To update, type update.packages() or install.packages("pROC") if you want to update pROC only.

• 1. Nancy A. Obuchowski, Donna K. McClish (1997). “Sample size determination for diagnostic accurary studies involving binormal ROC curve indices”. Statistics in Medicine, 16, 1529–1542. DOI: 10.1002/(SICI)1097-0258(19970715)16:13<1529::AID-SIM565>3.0.CO;2-H.
• 2. Nancy A. Obuchowski, Micharl L. Lieber, Frank H. Wians Jr. (2004). “ROC Curves in Clinical Chemistry: Uses, Misuses, and Possible Solutions”. Clinical Chemistry, 50, 1118–1125. DOI: 10.1373/clinchem.2004.031823.
• 3. Elisabeth R. DeLong, David M. DeLong and Daniel L. Clarke-Pearson (1988) “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach”. Biometrics 44, 837–845.