High Precise Measurement of R Expressions Execution Time

Provides infrastructure to accurately measure and compare the execution time of R expressions.


GitLab CI Build Status AppVeyror Build status Codecov Code Coverage License CRAN Status

Package benchr provides an infrastructure (framework) for precise measurement of R expressions execution time.

Key features:

  • Cross-platform implementation of the timer (the same code for all supported platforms);
  • High precision measurement of time intervals: usually nano or microseconds;
  • The reliability of the results due to a preliminary estimation of the timer error and subsequent correction of measurement results;
  • The stability of the results due to multiple repetitions of the measurements and the use of robust (resistant to outliers) statistics (quantile);
  • Informative output, including measurement accuracy, execution regime and descriptive statistics for each expression;
  • Various graphical representation of measurement results, including box plots, scatter plots and violin plots.

Installation

To install the package from the CRAN run the following command:

install.packages("benchr", repos = "https://cloud.r-project.org/")

To install the development version the following command can be used:

install.packages("benchr", repos = "https://artemklevtsov.gitlab.io/benchr/drat")

This package contains the compiled code, so to install it on Windows you will also need Rtools.

Usage

To measure execution time of arbitrary R code, benchr provides function benchmark(), as well as a number of additional methods for analysis and representation of results. Here's an example of time measurement for several expressions.

library(benchr)
benchmark(rep(1:100, each = 10), rep.int(1:100, rep.int(10, 100)))
#> Time units : microseconds 
#>                              expr n.eval   min lw.qu median  mean up.qu  max total relative
#>             rep(1:100, each = 10)    100 19.80 19.90  20.10 20.30 20.20 27.6  2030     7.16
#>  rep.int(1:100, rep.int(10, 100))    100  2.14  2.66   2.81  3.01  3.03 15.1   301     1.00
identical(rep(1:100, each = 10), rep.int(1:100, rep.int(10, 100)))
#> [1] TRUE

The resulting object can be saved as a variable and reused later in further methods:

res <- benchmark(NULL, {NULL}, {{{NULL}}})
summary(res)
#> Time units : nanoseconds 
#>              expr n.eval min lw.qu median   mean up.qu  max total relative
#>              NULL    100   5     7      9   8.68     9   24   868      1.0
#>          { NULL }    100  48    52     54  72.30    63 1200  7230      6.0
#>  { { { NULL } } }    100 132   145    150 158.00   155  644 15800     16.7

To present the results of measurements implemented additional methods for the class benchmark object:

  • mean -- means and confidence intervals for each R expression;
  • summary -- statistics (quantiles, means) for each R expression;
  • print -- text representation of results based on method summary;
  • plot -- scatter plot the execution time of each expression measure;
  • boxplot -- box plot the execution time of each expression.

For further details refer to the manual pages and vignettes:

help(package = "benchr")

Bug reports

Use the following command to go to the page for bug report submissions:

bug.report(package = "benchr")

Before reporting a bug or submitting an issue, please do the following:

  • Make sure that no error was found and corrected previously identified. You can use the search by the bug tracker;
  • Check the news list for the current version of the package. An error it might have been caused by changes in the package. This can be done with news(package = "benchr", Version == packageVersion("benchr")) command;
  • Make a minimal reproducible example of the code that consistently causes the error;
  • Make sure that the error triggered in the function from the benchr package, rather than in the code that you pass, that is other functions or packages;
  • Try to reproduce the error with the last development version of the package from the git repository.

When submitting a bug report please include the output produced by functions traceback() and sessionInfo(). This may save a lot of time.

License

The benchr package is distributed under GPLv2 license.

News

becnhr 0.2.2

  • Fix tests related with the ggplot2 3.0.0 release.

becnhr 0.2.1

  • Update Rcpp mininal version requirements to 0.12.11.
  • User drat repo to install devel version.

benchr 0.2.0

  • Added progress bar (using RcppProgress package) (#14).
  • Fixed limits when plotting with the log trnasformation (#19).
  • Some fixes related with R 3.3.4

benchr 0.1.0

  • Initial release.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("benchr")

0.2.2 by Artem Klevtsov, 8 months ago


https://gitlab.com/artemklevtsov/benchr


Report a bug at https://gitlab.com/artemklevtsov/benchr/issues


Browse source code at https://github.com/cran/benchr


Authors: Artem Klevtsov [aut, cre] , Anton Antonov [ctb] , Philipp Upravitelev [ctb]


Documentation:   PDF Manual  


GPL (>= 2) license


Imports Rcpp, RcppProgress, stats, graphics

Suggests testthat, ggplot2

Linking to Rcpp, RcppProgress

System requirements: C++11


Suggested by cubature.


See at CRAN