Track and report code coverage for your package and (optionally) upload the results to a coverage service like 'Codecov' < http://codecov.io> or 'Coveralls' < http://coveralls.io>. Code coverage is a measure of the amount of code being exercised by a set of tests. It is an indirect measure of test quality and completeness. This package is compatible with any testing methodology or framework and tracks coverage of both R code and compiled C/C++/FORTRAN code.
Track test coverage for your R package and view reports locally or (optionally) upload the results to codecov or coveralls.
install.packages("covr") # For devel versiondevtools::install_github("r-lib/covr")
The easiest way to setup covr on Travis-CI is with usethis.
usethis::use_coverage()
A coverage report can be used to inspect coverage for each line in your
package. Using report()
requires shiny.
library(covr) # If run with no arguments implicitly calls `package_coverage()`report()
covr also defines an RStudio Addin,
which runs report()
on the active project. This can be used via the addin
menu or by binding the action to a
shortcut, e.g.
Ctrl-Shift-C.
# if `getwd()` is the package's directory.package_coverage() # or a package in another directorycov <- package_coverage("/dir/lintr") # view results as a data.frameas.data.frame(cov) # zero_coverage() shows only uncovered lines.# If run within RStudio, `zero_coverage()` will open a marker pane with the# uncovered lines.zero_coverage(cov)
If you are already using Travis-CI add the
following to your project's .travis.yml
to track your coverage results
over time with Codecov.
r_github_packages: - r-lib/covr after_success: - Rscript -e 'covr::codecov()'
If you are using Appveyor CI, and are not using
Travis-CI at the same time, then you can add the
lines below to your project's appveyor.yml
:
on_success: - Rscript -e "covr::codecov()"
Don't forget to add covr
to the Suggests:
field of your package's
DESCRIPTION
file; possibly also to Remotes:
for r-lib/covr
.
For further details regarding Appveyor CI integration, also have a look at r-appveyor.
To use a different CI service or call codecov()
locally you can set the
environment variable CODECOV_TOKEN
to the token generated on codecov.io.
Codecov currently has support for the following CI systems (* denotes support
without needing CODECOV_TOKEN
).
You will also need to enable the repository on Codecov.
Alternatively you can upload your results to Coveralls
using coveralls()
.
r_github_packages: - r-lib/covr after_success: - Rscript -e 'covr::coveralls()'
For CI systems not supported by coveralls you need to set the COVERALLS_TOKEN
environment variable. It is wise to use a Secure Variable
so that it is not revealed publicly.
Also you will need to turn on coveralls for your project at https://coveralls.io/repos.
covr
supports a few of different ways of excluding some or all of a file.
A .covrignore
file located in your package's root directory can be used to
exclude files or directories.
The lines in the .covrignore
file are interpreted as a list of file globs to
ignore. It uses the globbing rules in Sys.glob()
. Any directories listed will
ignore all the files in the directory.
Alternative locations for the file can be set by the environment variable
COVR_COVRIGNORE
or the R option covr.covrignore
.
The .covrignore
file should be added to your .RBuildignore
file unless you
want to distribute it with your package. If so it can be added to
inst/.covrignore
instead.
The function_exclusions
argument to package_coverage()
can be used to
exclude functions by name. This argument takes a vector of regular expressions
matching functions to exclude.
# exclude print functionspackage_coverage(function_exclusions = "print\\.") # exclude `.onLoad` functionpackage_coverage(function_exclusions = "\\.onLoad")
The line_exclusions
argument to package_coverage()
can be used to exclude some or
all of a file. This argument takes a list of filenames or named ranges to
exclude.
# exclude whole file of R/test.Rpackage_coverage(line_exclusions = "R/test.R") # exclude lines 1 to 10 and 15 from R/test.Rpackage_coverage(line_exclusions = list("R/test.R" = c(1:10, 15))) # exclude lines 1 to 10 from R/test.R, all of R/test2.Rpackage_coverage(line_exclusions = list("R/test.R" = c(1, 10), "R/test2.R"))
In addition you can exclude lines from the coverage by putting special comments in your source code.
This can be done per line.
f1 <- function(x) { x + 1 # nocov}
Or by specifying a range with a start and end.
f2 <- function(x) { # nocov start x + 2} # nocov end
The patterns used can be specified by setting the global options
covr.exclude_pattern
, covr.exclude_start
, covr.exclude_end
.
Covr should be compatible with any testing package, it uses
tools::testInstalledPackage()
to run your packages tests.
Covr now supports Intel's icc
compiler, thanks to work contributed by Qin
Wang at Oracle.
Covr is known to work with clang versions 3.5+
and gcc version 4.2+
.
If the appropriate gcov version is not on your path you can set the appropriate
location with the covr.gcov
options. If you set this path to "" it will turn
off coverage of compiled code.
options(covr.gcov = "path/to/gcov")
covr
tracks test coverage by modifying a package's code to add tracking calls
to each call.
The vignette vignettes/how_it_works.Rmd contains a detailed explanation of the technique and the rationale behind it.
You can view the vignette from within R
using
vignette("how_it_works", package = "covr")
Because covr modifies the package code it is possible there are unknown edge cases where that modification affects the output. In addition when tracking coverage for compiled code covr compiles the package without optimization, which can modify behavior (usually due to package bugs which are masked with higher optimization levels).
shine()
has been removed. Instead use report()
.file_report()
added when viewing coverage for a single file (#308).
display_name()
is now exported, which can be useful to filter the coverage
object by filename.
environment_coverage()
added, mainly so it can be used for devtools::test_coverage_file()
.
gitlab()
function added to create a coverage report for GitLab using
GitLab's internal pages (@surmann, #327, #331).
The (optional) dependency on shiny has been removed. report()
can now be
built with only DT and htmltools installed.
Fix for gcc-8 gcov output producing lines with no coverage counts in them (#328)
impute_srcref()
now handles ...
and drop through arguments in switch
statements (#325).
tally_coverage()
now avoids an error when there are NA values in the source
references (#322).
covr(clean = TRUE)
now cleans the temporary library as well (#144)
package_coverage()
now returns the end of the file if there is a test error (#319)
report()
now handles reports in relative paths with subdirectories correctly (#329)
report()
reworked to look more like codecov.io and to display the overall
coverage (#302, #307).
DT explicitly loaded early in report()
so that failures will occur fast if
it is not installed. (#321, @renkun-ken).
shine()
has been deprecated in favor of report()
.Add support for .covrignore
files (#238), to exclude files from the coverage.
Support future versions of R which do not use parse data by default (#309).
Allow using trace_calls()
for manually adding functions to package trace
that are not found automatically (#295, @mb706).
Fix errors when R is not in the PATH
(#291)
Fix line computations when relative paths are being used (#242).
Fix for Coveralls Build processing error.
(#285) on pro accounts from
Travis CI (#306, @kiwiroy).
Keep attributes of function bodies (#311, @gaborcsardi)
Add an RStudio Addin for running a coverage report.
Never use mcexit fix on windows (#223).
Fix for a performance regression in parsing and reading parse data (#274).
Fix switch
support for packages, which was broken due to a bug in
how parse data is stored in packages.
Improve behavior of switch
coverage, it now supports default values and
fall through properly.
Add -p
flag to gcov command to preserve file paths. Fixes a bug where
gcov output didn't get reported when multiple compiled source files had
the same name (#271, @patperry)
R_COVR=true
when covr is running (#236, #268).filter_not_package_files()
now works if a source reference does not have a filename (#254, @hughjonesd).structure(NULL, *)
which is deprecated in R 3.4.0 (#260, #261, @renkun-ken).mcparallel:::mcexit()
automatically for packages using parallel (#195, @kforner).quit()
calls.function_coverage()
and package_coverage()
not use
non-standard evaluation.NULL
statements are analyzed for coverage (#156, @krlmlr).if
, while
and for
statements (#154, @krlmlr)..libPaths()
in subprocess to match those in calling process (#140, #147).