ModelMetrics
Rapid Calculation of Model Metrics
Collection of metrics for evaluating models written in C++ using 'Rcpp'. Popular metrics include area under the curve, log loss, root mean square error, etc.
Tyler Hunt [email protected]
Introduction
ModelMetrics is a much faster and reliable package for evaluating models. ModelMetrics is written in using Rcpp making it faster than the other packages used for model metrics.
Installation
You can install this package from CRAN:
install.packages("ModelMetrics")Or you can install the development version from Github with devtools:
devtools::install_github("JackStat/ModelMetrics")Benchmark and comparison
N = 100000Actual = as.numeric(runif(N) > .5)Predicted = as.numeric(runif(N)) actual = Actualpredicted = Predicted s1 <- system.time(a1 <- ModelMetrics::auc(Actual, Predicted))s2 <- system.time(a2 <- Metrics::auc(Actual, Predicted))# Warning message:# In n_pos * n_neg : NAs produced by integer overflows3 <- system.time(a3 <- pROC::auc(Actual, Predicted))s4 <- system.time(a4 <- MLmetrics::AUC(Predicted, Actual))# Warning message:# In n_pos * n_neg : NAs produced by integer overflows5 <- system.time({pp <- ROCR::prediction(Predicted, Actual); a5 <- ROCR::performance(pp, 'auc')}) data.frame( package = c("ModelMetrics", "pROC", "ROCR") ,Time = c(s1[[3]],s3[[3]],s5[[3]])) # MLmetrics and Metrics could not calculate so they are dropped from time comparison# package Time# 1 ModelMetrics 0.030# 2 pROC 50.359# 3 ROCR 0.358News
ModelMetrics 1.2.0
- added kappa statistic
- added s3 methods for
glm,lm,randomForest,merMod, andglmerMod - sped up
aucwithdata.table::frankv - added
gini
ModelMetrics 1.1.0
- added Matthews correlation coefficient (
mcc) - added multiclass auc (
mauc) - lots more tests
- fixed bug when rank ties were present in
auc(#10) - added code to handle different classes in functions
ModelMetrics 1.0.0
- Initializing package with basic metric functions