ROC-Optimizing Binary Classifiers

Implements ROC (Receiver Operating Characteristic)–Optimizing Binary Classifiers, supporting both linear and kernel models. Both model types provide a variety of surrogate loss functions. In addition, linear models offer multiple regularization penalties, whereas kernel models support a range of kernel functions. Scalability for large datasets is achieved through approximation-based options, which accelerate training and make fitting feasible on large data. Utilities are provided for model training, prediction, and cross-validation. The implementation builds on the ROC-Optimizing Support Vector Machines. For more information, see Hernàndez-Orallo, José, et al. (2004) , presented in the ROC Analysis in AI Workshop (ROCAI-2004).

roclab

An R package for ROC-Optimizing Binary Classifiers. It provides both linear and kernel models:

• Linear models provide various regularization penalties (ridge, lasso, alasso, elastic net, scad, mcp).
• Kernel models support a range of kernel functions (radial, polynomial, linear, laplace).
• Both models include a variety of surrogate loss functions (hinge, hinge2 (squared hinge), logistic, exponential).

For large datasets, scalability is achieved by approximating the empirical loss using incomplete U-statistics, and by applying a Nyström low-rank approximation to the kernel matrix. These approximations reduce computational cost while preserving accuracy, making ROC-Optimizing Binary Classifiers feasible in large-scale data.

In addition, efficient optimization is performed using Gradient Descent with the Adamax update rule —a variant of Adam, based on the infinity norm—for linear models with the ridge penalty and kernel models. For linear models with other penalties (i.e., those involving variable selection), Proximal Gradient Descent with an Adamax adaptive learning rate scheme is employed.

Installation

You can install the development version of roclab like so:

devtools::install_github("gimunBae/roclab")

Example

This is a basic example which shows you how to solve a common problem:

library(roclab)
set.seed(123)

# ========================
# Linear model example
# ========================
n_lin <- 1200
n_pos_lin <- round(0.2 * n_lin)
n_neg_lin <- n_lin - n_pos_lin

X_train_lin <- rbind(
  matrix(rnorm(2 * n_neg_lin, mean = -1), ncol = 2),
  matrix(rnorm(2 * n_pos_lin, mean =  1), ncol = 2)
)
y_train_lin <- c(rep(-1, n_neg_lin), rep(1, n_pos_lin))

n_test_lin <- 300
n_pos_test_lin <- round(0.2 * n_test_lin)
n_neg_test_lin <- n_test_lin - n_pos_test_lin
X_test_lin <- rbind(
  matrix(rnorm(2 * n_neg_test_lin, mean = -1), ncol = 2),
  matrix(rnorm(2 * n_pos_test_lin, mean =  1), ncol = 2)
)
y_test_lin <- c(rep(-1, n_neg_test_lin), rep(1, n_pos_test_lin))

# Fit a linear model
fit_lin <- roclearn(X_train_lin, y_train_lin, lambda = 0.1)

# Summary
summary(fit_lin)

# AUC on the test set
auc(fit_lin, X_test_lin, y_test_lin)

# Predict classes {-1, 1} on the test set
predict(fit_lin, X_test_lin, type = "class")

# ROC curve on the test set
y_score_lin <- predict(fit_lin, X_test_lin, type = "response")
plot_roc(y_test_lin, y_score_lin,summary=TRUE)

# 5-fold CV 
cvfit_lin <- cv.roclearn(
  X_train_lin, y_train_lin,
  lambda.vec = exp(seq(log(0.01), log(5), length.out = 20)),
  nfolds = 5
)

# Summarize cross-validation results
summary(cvfit_lin)

# Plot the cross-validation AUC across lambda values
plot(cvfit_lin)


# ========================
# Kernel model example
# ========================
n_ker <- 1200
r_train_ker <- sqrt(runif(n_ker, 0.05, 1))
theta_train_ker <- runif(n_ker, 0, 2*pi)
X_train_ker <- cbind(r_train_ker * cos(theta_train_ker), r_train_ker * sin(theta_train_ker))
y_train_ker <- ifelse(r_train_ker < 0.5, 1, -1)

n_test_ker <- 300
r_test_ker <- sqrt(runif(n_test_ker, 0.05, 1))
theta_test_ker <- runif(n_test_ker, 0, 2*pi)
X_test_ker <- cbind(r_test_ker * cos(theta_test_ker), r_test_ker * sin(theta_test_ker))
y_test_ker <- ifelse(r_test_ker < 0.5, 1, -1)

# Fit a kernel model
fit_ker <- kroclearn(X_train_ker, y_train_ker, lambda = 0.1, kernel = "radial")

# Summary
summary(fit_ker)

# AUC on the test set
auc(fit_ker, X_test_ker, y_test_ker)

# Predict classes {-1, 1} on the test set
predict(fit_ker, X_test_ker, type = "class")

# ROC curve on the test set
y_score_ker <- predict(fit_ker, X_test_ker, type = "response")
plot_roc(y_test_ker, y_score_ker,summary=TRUE)

# 5-fold CV 
cvfit_ker <- cv.kroclearn(
  X_train_ker, y_train_ker,
  lambda.vec = exp(seq(log(0.01), log(5), length.out = 20)),
  kernel = "radial",
  nfolds = 5
)

# Summarize cross-validation results
summary(cvfit_ker)

# Plot the cross-validation AUC across lambda values
plot(cvfit_ker)

License

License: MIT (see LICENSE file).

Citation

If you use this package in academic work, please cite as:

Bae, G., & Shin, S. J. (2025). roclab: ROC-Optimizing Binary Classifiers. R package version 0.1.4. Available at: https://github.com/gimunBae/roclab