Sending functions to remote processes can be wasteful of resources because they carry their environments with them. With the carrier package, it is easy to create functions that are isolated from their environment. These isolated functions, also called crates, print at the console with their total size and can be easily tested locally before being sent to a remote.
The carrier package provides tools to package up functions so they can be sent to remote R sessions or to different processes, and tools to test your crates locally. They make it easy to control what data should be packaged with the function and what size your crated function is.
Currently, carrier only provides a strict function constructor that forces you to be explicit about the functions and the data your function depends on. In the future it will also provide tools to figure it out automatically.
crate()
is a function constructor that forces you to be explicit about
which data should be packaged with the function. You can create
functions using the standard R syntax:
crate(function(x) mean(x, na.rm = TRUE))#> <crate> 7 kB#> * function: 6.55 kB#> function(x) mean(x, na.rm = TRUE)
Or with a purrr-like lambda syntax:
crate(~mean(.x, na.rm = TRUE))#> <crate> 1.57 kB#> * function: 1.01 kB#> function (..., .x = ..1, .y = ..2, . = ..1)#> mean(.x, na.rm = TRUE)
The crated function prints with its total size in the header, so you know how much data you will send to remotes. The size of the bare function without any data is also printed in the first bullet, and if you add objects to the crate their size is printed in decreasing order.
crate()
requires you to be explicit about all dependencies of your
function. Except for base functions, you have to call functions with
their namespace prefix. You can test your function locally to make sure
you’ve been explicit enough. In the following example we forgot to
specify that var()
comes from the stats namespace:
fn <- crate(~var(.x))fn(1:10)#> Error in var(.x): could not find function "var"
So let’s add the namespace prefix:
fn <- crate(~stats::var(.x))fn(1:10)#> [1] 9.166667
If your function depends on global data, you need to declare it to make
it available to your crated function. Here we forgot to declare na_rm
:
na_rm <- TRUEfn <- crate(function(x) stats::var(x, na.rm = na_rm))fn(1:10)#> Error in stats::var(x, na.rm = na_rm): object 'na_rm' not found
There are two techniques for packaging data into your crate: passing data as arguments, and unquoting data in the function.
You can declare objects by passing them as named arguments to crate()
:
fn <- crate(function(x) stats::var(x, na.rm = na_rm),na_rm = na_rm)fn(1:10)#> [1] 9.166667
Note how the size of each imported object is displayed when you print the crated function:
fn#> <crate> 9.31 kB#> * function: 8.75 kB#> * `na_rm`: 56 B#> function(x) stats::var(x, na.rm = na_rm)
Another way of packaging data is to unquote objects with !!
. This
works because unquoting inlines objects in function calls. Unquoting can
be less verbose if you have many small objects to import inside the
function.
crate(function(x) stats::var(x, na.rm = !!na_rm))#> <crate> 7.86 kB#> * function: 7.42 kB#> function(x) stats::var(x, na.rm = !!na_rm)
However, be careful not to unquote large objects because:
Let’s unquote a data frame to see the noise caused by inlining:
# Subset a few rows so the call is not too noisydata <- mtcars[1:5, ]# Inline data in call by unquotingfn <- crate(~stats::lm(.x, data = !!data))
This crate will print with noisy inlined data:
fn#> <crate> 4.65 kB#> * function: 4.14 kB#> function (..., .x = ..1, .y = ..2, . = ..1)#> stats::lm(.x, data = list(mpg = c(21, 21, 22.8, 21.4, 18.7),#> cyl = c(6, 6, 4, 6, 8), disp = c(160, 160, 108, 258, 360),#> hp = c(110, 110, 93, 110, 175), drat = c(3.9, 3.9, 3.85,#> 3.08, 3.15), wt = c(2.62, 2.875, 2.32, 3.215, 3.44), qsec = c(16.46,#> 17.02, 18.61, 19.44, 17.02), vs = c(0, 0, 1, 1, 0), am = c(1,#> 1, 1, 0, 0), gear = c(4, 4, 4, 3, 3), carb = c(4, 4, 1, 1,#> 2)))
Same for the function call recorded by lm()
:
fn(disp ~ drat)#>#> Call:#> stats::lm(formula = .x, data = structure(list(mpg = c(21, 21,#> 22.8, 21.4, 18.7), cyl = c(6, 6, 4, 6, 8), disp = c(160, 160,#> 108, 258, 360), hp = c(110, 110, 93, 110, 175), drat = c(3.9,#> 3.9, 3.85, 3.08, 3.15), wt = c(2.62, 2.875, 2.32, 3.215, 3.44#> ), qsec = c(16.46, 17.02, 18.61, 19.44, 17.02), vs = c(0, 0,#> 1, 1, 0), am = c(1, 1, 1, 0, 0), gear = c(4, 4, 4, 3, 3), carb = c(4,#> 4, 1, 1, 2)), row.names = c("Mazda RX4", "Mazda RX4 Wag", "Datsun 710",#> "Hornet 4 Drive", "Hornet Sportabout"), class = "data.frame"))#>#> Coefficients:#> (Intercept) drat#> 952.3 -207.8
Initial release. The package currently only contains a single method for creating crates that requires users to be explicit about what data to pack in the crate. A future release will provide a method to figure out automatically what objects the crate depends on (with inevitable false positives and negatives).