Rapid Asynchronous and Distributed Computing

Package to tackle large-scale problems asynchronously across a distributed network. Employing a database centric model, rush enables workers to communicate tasks and their results over a shared 'Redis' database. Key features include low task overhead, efficient caching, and robust error handling. The package powers the asynchronous optimization algorithms in the 'bbotk' and 'mlr3tuning' packages.

rush

Package website: release | dev

rush is an R package for asynchronous and decentralized optimization. It uses a database-centric architecture in which workers communicate through a shared Redis database. To support high-throughput workloads, rush combines sub-millisecond per-task overhead with caching strategies that reduce database operations. The package integrates with the mlr3 ecosystem and serves as the backend for asynchronous optimization algorithms in bbotk and mlr3tuning.

Features

Database-centric architecture for decentralized optimization without a central controller.
Asynchronous communication between workers using Redis.
Sub-millisecond per-task overhead for high-throughput workloads.
Efficient caching mechanism that minimizes database read/write operations.
Task queue support for centralized tasks distribution when needed.
Fast data transformation from Redis to data.table.
Scales to large remote worker networks via the mirai package.
Worker logging directly into the Redis database using lgr.
Minimal dependencies for lightweight integration.

Install

Install the latest release from CRAN.

install.packages("rush")

Install the development version from GitHub.

pak::pak("mlr-org/rush")

And install Redis.

Test with Redis

To test the package, set the RUSH_TEST_USE_REDIS environment variable to true. The test suite deletes the Redis database before execution, so never run it against a production server.

Sys.setenv(RUSH_TEST_USE_REDIS = "true")

Citation

If you use rush, please cite our paper.

Becker M, Bischl B (2026). “rush: Scalable Asynchronous Distributed Computing via Shared State in R.” arXiv preprint arXiv:2606.21430. https://arxiv.org/abs/2606.21430.

@misc{becker2026rush,
  title = {rush: {{Scalable Asynchronous Distributed Computing}} via {{Shared State}} in {{R}}},
  shorttitle = {rush},
  author = {Becker, Marc and Bischl, Bernd},
  year = 2026,
  month = jun,
  number = {arXiv:2606.21430},
  eprint = {2606.21430},
  primaryclass = {cs.DC},
  publisher = {arXiv},
  doi = {10.48550/arXiv.2606.21430},
  archiveprefix = {arXiv}
}