Resampling Algorithms for 'mlr3' Framework

A supervised learning algorithm inputs a train set, and outputs a prediction function, which can be used on a test set. If each data point belongs to a subset (such as geographic region, year, etc), then how do we know if subsets are similar enough so that we can get accurate predictions on one subset, after training on Other subsets? And how do we know if training on All subsets would improve prediction accuracy, relative to training on the Same subset? SOAK, Same/Other/All K-fold cross-validation, can be used to answer these questions, by fixing a test subset, training models on Same/Other/All subsets, and then comparing test error rates (Same versus Other and Same versus All). Also provides code for estimating how many train samples are required to get accurate predictions on a test set.


Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("mlr3resampling")

2026.9.24 by Toby Hocking, 12 days ago


https://github.com/tdhock/mlr3resampling


Report a bug at https://github.com/tdhock/mlr3resampling/issues


Browse source code at https://github.com/cran/mlr3resampling


Authors: Toby Hocking [aut, cre] (ORCID: , Daniel Agyapong [ctb] , Michel Lang [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Bernd Bischl [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Jakob Richter [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Patrick Schratz [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Giuseppe Casalicchio [ctb] (ORCID: , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Stefan Coors [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Quay Au [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Martin Binder [ctb] , Florian Pfisterer [ctb] (ORCID: , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Raphael Sonabend [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Lennart Schneider [ctb] (ORCID: , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Marc Becker [ctb] , Author of mlr3 when Resampling/ResamplingCV was copied/modified) , Sebastian Fischer [ctb] (ORCID: , Author of mlr3 when Resampling/ResamplingCV was copied/modified)


Documentation:   PDF Manual  


LGPL-3 license


Imports data.table, R6, checkmate, paradox, mlr3, mlr3misc, methods, Rcpp

Suggests pbdMPI, geepack, ggplot2, mlr3tuning, lgr, future, future.apply, testthat, WeightedROC, nc, rpart, directlabels, mlr3pipelines, glmnet, mlr3learners, mlr3torch, torch, batchtools, mlr3batchmark, litedown

Linking to Rcpp, RcppArmadillo


See at CRAN