Collection of metrics for evaluating models written in C++ using 'Rcpp'. Popular metrics include area under the curve, log loss, root mean square error, etc.
Tyler Hunt [email protected]
ModelMetrics is a much faster and reliable package for evaluating models. ModelMetrics is written in using Rcpp making it faster than the other packages used for model metrics.
You can install this package from CRAN:
install.packages("ModelMetrics")
Or you can install the development version from Github with devtools:
devtools::install_github("JackStat/ModelMetrics")
N = 100000Actual = as.numeric(runif(N) > .5)Predicted = as.numeric(runif(N)) actual = Actualpredicted = Predicted s1 <- system.time(a1 <- ModelMetrics::auc(Actual, Predicted))s2 <- system.time(a2 <- Metrics::auc(Actual, Predicted))# Warning message:# In n_pos * n_neg : NAs produced by integer overflows3 <- system.time(a3 <- pROC::auc(Actual, Predicted))s4 <- system.time(a4 <- MLmetrics::AUC(Predicted, Actual))# Warning message:# In n_pos * n_neg : NAs produced by integer overflows5 <- system.time({pp <- ROCR::prediction(Predicted, Actual); a5 <- ROCR::performance(pp, 'auc')}) data.frame( package = c("ModelMetrics", "pROC", "ROCR") ,Time = c(s1[[3]],s3[[3]],s5[[3]])) # MLmetrics and Metrics could not calculate so they are dropped from time comparison# package Time# 1 ModelMetrics 0.030# 2 pROC 50.359# 3 ROCR 0.358
glm
, lm
, randomForest
, merMod
, and glmerMod
auc
with data.table::frankv
gini
mcc
)mauc
)auc
(#10)