A monadic solution to pipeline analysis. All operations -- and the errors, warnings and messages they emit -- are merged into a directed graph. Infix binary operators mediate when values are stored, how exceptions are handled, and where pipelines branch and merge. The resulting structure may be queried for debugging or report generation. 'rmonad' complements, rather than competes with, non-monadic pipeline packages like 'magrittr' or 'pipeR'. This work is funded by the NSF (award number 1546858).
make_local_cacher to take custom functions for saving, getting,
checking and deleting data.
crunch function to cache all values over a given size that are stored
tag field to
Rmonad objects and access functions:
tag - set the tags of the current head node (or nodes at given indices)
view - set the head of the graph to the given tagged node
views - get list of Rmonad objects matching a given tag
viewID - like view but with ID
viewIDs - like views but with ID
get_tag - get tags for nodes
has_tag - check whether nodes have tags
tag argument to
as_monad to allow tag setting at creation
tag argument to
get_* to allow access by tag
tag to caching functions, to allow tag and cache to be set together
give names to elements of lists returned by tag from
Node names are now reproducible. If you notice any internal nodes disappearing, please file a bug report, since this means we missed some case in our hashing system.
saveRDS, rather than
save, for local caching
Combine (and funnel) now stores failing inputs as NULL and does not delete the failing parent's cache
funnel, tagged parents are not deleted
Add automatic caching, if a function takes a long time to run, it is cached
Add option to turn toggle auto caching:
rmonad.auto_cache (default = FALSE)
Add option to set how long an expression can take before its result will be
rmonad.cache_maxtime (default = 3 seconds)
Add option to set cache directory:
rmonad.cache_dir (default = "cache")
Add option to set cache function:
rmonad.cacher (default =
Fix expression capture in
index the second positional argument of
has_value, as with other
Metadata lists are now evaluated in the natural lexical scope, with access to the function arguments and variables in the scope where the function was defined.
Add vignette showing integration with
magrittr pipe operator
Replace the internal ad hoc graph system with 'igraph'
Rmonad is now an S4 class
The raw igraph object is stored in the
The vertex attributes are stored in a list in the
Replace the concept of "branches" (e.g. the
%>^% operator) with
"children". These are just multiple children of a node in a directed graph.
Add a flexible cache system
size function that returns the number of nodes in a workflow.
ms_* accessors are renamed as
has_* vectorized accessors for the existence of fields (e.g.
plot.Rmonad is now a wrapper for
as_dgr_graph, now the underlying graph is already an igraph object,
which can be converted easily to a DiagrammeR graph, if desired
Remove all the
ms_* functions (replaced with
m_* functions. They provided access to the "head" of the
workflow, but this isn't really all that well defined, since
Completely remove deprecated
%v__% operator and the
Do not export
unnest (you don't need it)
report. The content of the report is updated somewhat,
but still should be considered as a template.
missues: Deprecate the
plot.Rmonad: pass '...' to
Evaluate metadata in the correct environment
nest_depth is set as the graph grows (in the past it wasn't added
mtabulate was run).
Free memory for values stored in the Rmonad objects when they are combined
funnel, previously these values were stored even when they should
have been removed.
Fixed lots of bugs and inconsistencies in the vectorized getters.
The following new features are experimental. I intend to keep them around in some form in the future, but the API will likely change.
Add rewriter functions that can reformat warning, error, and note message, or summarize the result, after an Rmonad is built. These rewriters are set in a node's metadata list. The following rewritters are currently supported:
format_warnings- format warning strings
format_error- format error strings
format_notes- format note (message) strings
summarize- store summaries of the data
cache- set the caching function
log- write log
Add a cache system, where values are stored as CacheManager objects which
have three slots:
chk. These are functions for getting,
deleting, and checking values in a cache, respectively. Internally, the
get functions may store values in memory (in a closure), locally as
a Rdata object, or not at all. The following new functions are exported:
make_local_cacher - build a cache function that stores data as Rdata
objects in a specified folder.
memory_cache - cache a value in memory
make_recacher - build a function to reset an
Rmonad object's cacher
no_cache - do not cache the value
clear_cache - removes all cached values (in memory or storage)
%v__% operator. Now the
%__% operator always stores the
final left-side result.
Fix linking to nested history.
Fix incorrect history linking bugs caused by the
Preserve the inputs to failing nested nodes. Now if a nested node fails, the input to that node, and the input to every ascending node, is preserved. (like a traceback where the function arguments are saved).
Add a new kind of edge. The
%__% associates two pipelines but does not
pass data or propagate errors. This was previously treated as one kind of
'parent' edge; now it is a 'prior' edge.
Add special plot handling for 'prior' edges.
combine now works exclusively on lists of Rmonad objects
Allow nesting of Rmonads,
as_monad no longer automatically unnests them
doc function. Instead use doc strings.
Docstrings and metadata in anonymous functions
Support for multivariate anonymous functions
as_monad now records time and space (previously were left as NA)
Rmonad class refactored as an R6 class
%*>% to take monad bound lists
m_value raises a warning (by default) when accessing an uncached value.
mreport - Generate of rudimentary Markdown reports
as_dgr_graph - Convert pipeline to DiagrammeR graph
is_rmonad - Tests if something is an
unnest - remove a level of nesting from a node
extract_metadata - get docstring and metadata from a function or block
m_nest - get or set a node's nested pipeline
m_nest_depth - get or set a node's nest depth
m_meta - get or set a node's metadata
m_id - get or set a node's id (changing an existing id raises a warning)
app_parents - add a list of parents to a node
m_delete_value - delete the value stored by a node
NULL can now be stored as a value and is distinguishable from Nothing
(i.e. an uncached value).
as.list now lists elements in the expected order
Errors raised are stored even if they are not non-empty strings. Previously
stop() would be reckoned as passing.
Put docstring first in the printed output