Create and Emit 'OpenLineage' Events

Construct and validate run events that follow the 'OpenLineage' specification < https://openlineage.io/>. Model run lifecycles, datasets, and extensible facets with protocol-aware 'R' objects. Serialize events into deterministic JavaScript Object Notation (JSON) while preserving wire field names. Deliver events synchronously over Hypertext Transfer Protocol (HTTP) with authentication and retries, or use offline transports for local development and testing.

openlineage

openlineage is an R client for constructing, validating, serializing, and emitting OpenLineage run events. It targets the OpenLineage 2.0.2 schema and supports synchronous HTTP delivery as well as offline transports for development and testing.

Installation

Install the released package from CRAN with:

install.packages("openlineage")

Quick start

Use an accumulating transport to exercise a complete lifecycle without a server or network connection:

library(openlineage)

transport <- AccumulatingTransport$new()
client <- OpenLineageClient$new(transport = transport, disabled = FALSE)

run <- Run(new_run_id())
job <- Job("example", "daily-orders")

client$emit(RunEvent(run, job, event_type = "START"))
client$emit(RunEvent(run, job, event_type = "RUNNING"))
client$emit(RunEvent(run, job, event_type = "COMPLETE"))

length(transport$events)
to_openlineage_json(transport$events[[3]], pretty = TRUE)

See vignette("openlineage-lifecycle") for datasets, facets, configuration, authentication, and failure handling.

HTTP delivery and authentication

Create a client with an explicit endpoint. Constructing the client does not make a request; emit() performs synchronous delivery.

client <- OpenLineageClient$new(
  url = "https://lineage.example.com",
  api_key = "replace-with-your-secret",
  timeout = 10,
  max_retries = 3L
)

api_key produces a bearer Authorization header. Backend-specific headers can instead be supplied as a named character vector through headers. Do not combine api_key with a custom Authorization header. Credentials are kept out of events, printed clients, and package conditions.

The client recognizes these environment variables when the corresponding constructor value is absent:

Explicit values take precedence over environment values. An injected transport ignores HTTP environment settings. With no transport or URL, events are printed through ConsoleTransport.

Facets and extensions

Typed constructors cover common run, job, dataset, and statistics facets. Use ol_facet() for a schema without a typed R constructor:

custom <- ol_facet(
  "https://example.com/facets/QualityRunFacet.json",
  score = 0.98,
  method = "rules"
)
run <- Run(new_run_id(), facets = list(quality = custom))

Transport failures inherit from openlineage_transport_error. HTTP and authentication failures additionally use openlineage_http_error and openlineage_auth_error, respectively. Configuration and model failures use openlineage_config_error and openlineage_validation_error.