Token-Authenticated REST API Retrieval Toolkit

A small, dependency-light toolkit for talking to token-authenticated REST APIs. It manages authentication tokens in process environment variables (never written to disk), builds requests with configurable authentication and pagination strategies, and retrieves paginated data either one page at a time or in chunks combined into a single tibble. The design is API-agnostic: a single 'apifetch_api' profile describes an endpoint together with how it authenticates and paginates, so the same verbs work across different services.

apifetch

apifetch is a small, dependency-light toolkit for talking to token-authenticated REST APIs from R. It handles three recurring chores:

1. Token management — store/get/remove/list tokens in process environment variables (never written to disk), namespaced per service.
2. Request building — pluggable authentication and pagination strategies, bundled into a reusable API profile.
3. Data retrieval — fetch one page, or fetch everything in chunks row-bound into a single tibble.

It is the generic engine extracted from the BigDataPE package; BigDataPE is now just one use case (see vignette("bigdatape")). A Python sibling lives at apifetch-py.

Installation

Install the released version from CRAN:

install.packages("apifetch")

Or the development version from GitHub:

# install.packages("pak")
pak::pak("StrategicProjects/apifetch")

Usage

library(apifetch)

# 1. Describe the API once: where, how to authenticate, how to paginate.
api <- af_api(
  endpoint   = "https://api.example.com/v1/search",
  service    = "Example",
  auth       = af_auth_bearer(),          # "Authorization: Bearer <token>"
  pagination = af_paginate_offset("query")
)

# 2. Store a token (kept only in this session's environment).
af_store_token("reports", "my-secret-token", service = "Example")

# 3. Fetch.
one_page <- af_fetch(api, "reports", limit = 50)
everything <- af_fetch_all(api, "reports", chunk_size = 1000)

Strategies

Authentication: af_auth_bearer(), af_auth_raw(), af_auth_header(), af_auth_query().

Pagination: af_paginate_offset(where = "header" | "query"), af_paginate_none(). ```