Comparison of Algorithms with Iterative Sample Size Estimation

Functions for performing experimental comparisons of algorithms using adequate sample sizes for power and accuracy.

Felipe Campelo ([email protected]) and Fernanda Takahashi ([email protected])
Operations Research and Complex Systems Laboratory - ORCS Lab
Universidade Federal de Minas Gerais
Belo Horizonte, Brazil

Implementation of R package CAISEr, with routines for automatically determining the sample size needed for performing comparative experiments with algorithms.

To install the most up-to-date version directly from Github, simply type:


The most recent CRAN release of the package is also available for installation directly from the R prompt, using:


For instructions and examples of use, please take a look at the vignette Adapting Algorithms for CAISEr, and at the package documentation, particularly functions run_experiment() and run_nreps2().

Please send any bug reports, questions, suggestions, chocolate (to Fernanda) or beers (to Felipe - we can always hope!) directly to the package authors listed at the top of this document.



CAISEr 0.3.3

  • fixed problem with printing version in the vignette.

CAISEr 0.3.2

  • fixed rare bug in calc_se() that resulted in NaN if two vectors with the same sample mean and same sample variance were passed as arguments.

CAISEr 0.3.1

  • Added function to consolidate partial results saved to file (consolidate.partial.results())
  • Minor improvements to saving partial results to file: users can now select arbitrary directory for saving

CAISEr 0.3

  • run_experiment() can now be run in parallel using multiple cores.
  • run_experiment() and calc_nreps2() can now save results to files.

CAISEr 0.2.4

  • run_experiment() now forces the use of all available instances if power >= 1.

CAISEr 0.2.3

  • Improved plot and summary functions for CAISErPowercurve objects.
  • Added options to calc_power_curve() to determine the range of effect sizes to consider.

CAISEr 0.2.2

  • Added new example and use case to calc_nreps2()

CAISEr 0.2.1

  • Minor fixes, particularly in printing function.

CAISEr 0.2.0

  • Initial release on CRAN.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.


1.0.5 by Felipe Campelo, 10 days ago

Report a bug at

Browse source code at

Authors: Felipe Campelo [aut, cre] , Fernanda Takahashi [ctb] , Elizabeth Wanner [ctb]

Documentation:   PDF Manual  

GPL-2 license

Imports assertthat, parallel, pbmcapply, ggplot2

Suggests MOEADr, smoof, knitr, rmarkdown, car, dplyr, pkgdown

See at CRAN