Interactive R Unit Tests

Simplifies regression tests by comparing objects produced by test code with earlier versions of those same objects. If objects are unchanged the tests pass, otherwise execution stops with error details. If in interactive mode, tests can be reviewed through the provided interactive environment.


Dependencies direct/recursive Project Status: WIP - Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.

TL;DR

unitizer simplifies creating, reviewing, and debugging unit tests in R. To install:

install.packages('unitizer')

Please keep in mind this is an experimental framework that has been thoroughly tested by one person.

unitizer bakes in a lot of contextual help so you can get started without reading all the documentation. Try the demo to get an idea:

library(unitizer)
demo(unitizer)

Or check out the screencast to see unitizer in action.

Why Another Testing Framework?

Automated Test Formalization

Are you tired of the deparse/dput then copy-paste R objects into test file dance, or do you use testthat::expect_equal_to_reference a lot?

With unitizer you review function output at an interactive prompt as you would with informal tests. You then store the value, conditions (e.g. warnings, etc.), and environment for use as the reference values in formal tests, all with a single keystroke.

Streamlined Debugging

Do you wish the nature of a test failure was more immediately obvious?

When tests fail, you are shown a proper diff so you can clearly identify how the test failed:

diffexample

Do you wish that you could start debugging your failed tests without additional set-up work?

unitizer drops you in the test environment so you can debug why the test failed without further ado:

reviewexample

Fast Test Updates

Do you avoid improvements to your functions because that would require painstakingly updating many tests?

The diffs for the failed tests let you immediately confirm only what you intended changed. Then you can update each test with a single keystroke.

Usage

unitizer stores R expressions and the result of evaluating them so that it can detect code regressions. This is akin to saving test output to a .Rout.save file as documented in Writing R Extensions, except that we're storing the actual R objects and it is much easier to review them.

To use unitizer:

  • Write test expressions as you would when informally testing code on the command line, and save them to a file (e.g. "my_file_name.R")
  • Run unitize("my_file_name.R") and follow the prompts
  • Continue developing your package
  • Re-run unitize("my_file_name.R"); if any tests fail you will be able to review and debug them in an interactive prompt

unitizer can run in a non-interactive mode for use with R CMD check.

Documentation

Related Packages

Acknowledgments

Thank you to:

About the Author

Brodie Gaslam is a hobbyist programmer based in the US East Coast.

News

unitizer NEWS

v1.4.7

  • Set RNGversion() in internal tests due to changes to R sampling mechanism.

v1.4.6

  • #258 Remove devtools as a suggests dependency (reported by @jimhester).
  • #257 Ensure all slots in S4 prototypes exist in the class definition (reported by B. Ripley).

v1.4.5

  • Colors in test browser working again.
  • #220 Fix vignette ordering on CRAN.
  • #225 Inferring file locations when not in pkg top level.
  • #237 Option to turn off diffs.
  • #239 Document issues with deparsing doubles in tests.
  • #242 Correctly parse newlines in prompt input.
  • #241 q() actually works now.
  • #231 Report parse warnings at prompt.
  • #234 Random seed issues.
  • #220 Re-order vignettes.
  • Wrap diffObj call in try to avoid killing unitizer section when diffObj errors.
  • Modify test_dir calls to use wrap on testthat versions greater than 0.1.2.

v1.4.4

  • Fix tests broken by R-devel 3.4.1 improvements to S4 deparsing
  • #232 document that .Last.value does not work
  • #228 more docs on handling RDS files
  • README typos (@eddelbuettel)
  • Improve vignette index

v1.4.3

  • #221 Crash on Re-Run due to poor comparison of parsed test file sources
  • #222 Allow loading of unitizers generated by later versions of the package in earlier version of the packages provided the object validates
  • #224 Slow re-eval in recommended state mode with lots of files

v1.4.1-2

Fixing CRAN submission errors.

v1.4.0

First Release to CRAN.

  • #213: Disable advanced state management by default to comply with CRAN
  • #203: Better reporting of search path state differences
  • Assorted bugfixes and enhancements: #214, #215, #208, #201, #208, #199, #197

v1.2.0 - v1.3.0

Interim release to get changes over past year out on the master branch.

Improvement / Changes:

  • #194: Performance improvements
  • #102: Ability to break out of multi file review with QQ
  • #186: If you type q() or quit() by mistake you can now cancel that action before unitizer terminates unceremoniously
  • #142: Ability to run unitize and unitize_dir without specifying any arguments (infer from working directory)
  • #168: Failing tests now will display all output and conditions
  • #171: Flush warnings in unitizer REPL
  • Improved integration of object diffs via .DIFF and .diff
  • Display improvements, including:
    • Cleaner separation of unitizer meta-output vs. test or command line
    • #164, #176: Streamline state difference display
  • Several other enhancements

Bugfixes:

  • #188: Search path issues in r-devel @richierocks
  • Fix unintended detaching of tools:rstudio, and many others
  • base.env now unique for each unitizer

v1.1.0

Improvement / Changes:

  • #161: Compare objects with diffobj::diffObj
  • #166: More systematic handling of library/attach/detach

Bugfixes:

  • Several unitizer prompt issues:
    • No longer capture prompt evals so debug is usable again
    • Parse errors reported correctly
    • #165: Confusing Help Prompt
  • Reference state properly preserved (previously would incorrectly use new state for reference tests kept in store)
  • Internal shimming of library/detach/attach more robust
  • Updated tests for changes in testthat, R

v1.0.0-1.0.9

Improvement / Changes:

  • More comprehensive state tracking and resetting:
    • options, random.seed, and wd are tracked in addition to search path
    • state is reset when reviewing tests as well as when executing them
    • you are alerted off state differences between new and reference tests on error
    • State control parameters are streamlined; API breaking
  • Whether an expression is ignored or not is now a function of whether the expression returns visibly or not
  • Pre and post test scripts
    • 'helper' directory renamed '_pre'
    • Can now use a '_post' directory to run cleanup
  • Interactive environment cleanup
    • Display tweaks
    • Contextual help tweaks
  • Vignette updates
  • Demo update
  • Added Rdiff_obj to run a tools::Rdiff directly on two R objects

Internal:

  • Reduced storage requirements for the unitizer stores
    • No longer storing assignments both as test value and object in environment
    • Calls recorded deparsed instead of as call objects
  • Shimming used for search path tracking is more lightweight
  • Text capture much more robust

Issues Fixed:

107, 106, 104, 103, 101, 99, 98, 94, 93, 90, 85, 84, 77, 74, 71, 67, 127, 115, 132, 134

v0.9.0

Improvements / Changes:

  • Complete restructure of internal test management to allow for much more robust unitize_dir behavior (#51)
  • Added testthat -> unitizer translation utilities (see ?testthat_translate_file)
  • Can now pre-load objects before unitizing; unitize_dir and unitize by default auto-preload files in subdir 'helper'
  • Renamed arg env.clean to par.env (technically API breaking, but since no one is using this package yet...)
  • Many usability fixes (#48, #68, #82, #83), and improved text display
  • Improved path inference to better guess desired unitizer based on partiallly specified file names (#72)

Other

  • unitize_dir works with empty dirs (#78)
  • Better management of file locations and names (#35, #75)

v0.8.1

Bugfixes

  • review now properly infers unitizer locations

v0.8.0

Improvements:

  • Added ability to accept multiple tests at once (Issue #45, use wisely...)
  • unitize can now infer partially specified test file names (in particular, will know to look in pkgname/tests/unitizer; see ?infer_unitizer_location)
  • parse_with_comments no longer run in non-interactive mode (#63)
  • Test call now part of output of test object show method (#54)

Bugfixes:

  • Comments inside unitizer_sect preserved (#64)
  • Ignored tests assigned to first non-ignored test section (#57)
  • Prompt display issues (#65, #66)

Internal:

  • search_path_cleanup more robust (#59)
  • get_text_capture tests added (#60)

v0.7.1

Improvements:

  • Reduced test execution and parsing overhead
  • Better handling of call recording for traceback and condition calls
  • editFunNames becomes editCalls and provides more comprehensive editing of calls (Issue #52)

Bufixes:

  • Comment handling in calls (Issues #56, #58)
  • Comment deparsing (Issues #39, #47, #52)

v0.7.0

Improvements:

  • Failed tests now automatically output a snippet of new and reference objects (Issue #34)
  • Text handling generally improved (better wrapping, etc. Issue #38)
  • Parsing speed improved (Issue #15)
  • Got rid of get* functions, instead, access test details with .NEW/.REF (Issue #29)
  • Implemented editFunNames to allow user to modify stored calls in unitizer so that tests can be re-used even if function names are changed

v0.6.5

Doc updates; should have been part of 0.6.4, but was too rushed to push...

v0.6.4

Improvements:

  • Comment parsing faster (issue #15)

Bugfixes:

  • Reference section logic improved (assume fixes #36 until there is evidence against)
  • Several parse errors fixed

Other:

  • Now depends on R 3.1.2 (not really, but that's what we are developing on and don't have bandwidth to test against multiple R versions)

v0.6.3

Bugfixes:

  • stderr now show in review mode (issue #43)
  • package startup messages suppressed (issue #23)
  • small demo bug

v0.6.2

Bugfixes:

  • Better whitespace wrapping in terminal mode (partially addresses #38)
  • Can now drop all items in review mode (issue #37)
  • Workaround an R parse bug (issue #41)
  • traceback() now works for stop(simpleError(...)) type stops

Behavior changes:

  • History is only subbed out if you need to type input (issue #40)

v0.6.1

Minor release, no substantive changes.

Bugfixes:

  • Loading a unitizer no longer automatically modifies it through upgrade
  • upgrade cleaned up and has tests now
  • calling functions in form pkg::fun or pkg:::fun no longer causes problems when checking for ignoredness

Behavior changes:

  • get no longer warns if unitizer ids don't match

v0.6.0

New Features:

  • Added a demo (demo(unitizer))
  • Broke up and updated vignettes
  • unitize_dir allows you to run all tests in a directory (issue #24)
  • review allows you to review and drop tests from an existing unitizer store (issue #21)
  • Test navigation mechanism improved (issue #26)
    • Typing R at the unitizer prompt now allows you to review all tests
    • You can skip ahead too
  • unitize(..., force.update=TRUE) will overwrite unitizer even if there were no changes recorded (issue #19)

Behavior changes:

  • unitize now runs with search.path.clean=TRUE by default

Bugfixes:

  • Comparison function warnings not captured (issue #14)
  • Search path restoration error messages fixed (issue #22)
  • Navigation regressions fixed (issue #30)

Other:

Summary titles cleaned up, interative prompts made clearer, package reload warn conflicts quieted (d2fe594c747, #23)

v0.5.0

New Features:

  • Can now run tests in clean environment (i.e. objects from .GlobalEnv will not be visible) (issue #13)
  • Can now run tests with clean search path (i.e. only the basic R libraries are loaded) (also issue #13), use unitize(..., search.path.clean=TRUE)
  • New vignette "Reproducible Tests" discusses the above features

Bugfixes:

  • Expressions printed as tests evaluated now truncated corretly (issue #4)
  • Incorrect displaying/hiding of ignored tests in some circumstances fixed

Other Improvements:

  • Summary no longer includes "removed" tests in matrix, since those are section- less
  • Other minor clean-up of the interactive environment prompting

v0.4.3

Many interactive use bug fixes:

  • LBB now parsed properly (issue #5)
  • Non interactive parse (issue #11)
  • Review and Back behavior consistent now in interactive use (issue #3)
  • Other interactive use cleanup (issues #6, 12, 10, 9)
  • Vignette now done properly

v0.4.2

  • Fixed setOldClass conflicts with RJSONIO (issue #1)
  • Fixed run_ls not finding base env under certain circumstances (issue #2)
  • Fixed conditionLists looping issue introduced when fixing issue #1

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("unitizer")

1.4.8 by Brodie Gaslam, a month ago


https://github.com/brodieG/unitizer


Report a bug at https://github.com/brodieG/unitizer/issues


Browse source code at https://github.com/cran/unitizer


Authors: Brodie Gaslam [aut, cre]


Documentation:   PDF Manual  


GPL (>= 2) license


Imports stats, utils, crayon, diffobj

Depends on methods

Suggests covr, knitr, rmarkdown, testthat


Suggested by fansi, oshka, vetr.


See at CRAN