Generate and Render Timed Subtitles Using 'FFmpeg'

Tools for generating timed subtitle files and rendering them into video with the 'FFmpeg' command-line interface (< https://ffmpeg.org/>). Converts structured transcription output (such as that produced by 'Whisper' speech-to-text models) into SubRip ('SRT') and Advanced SubStation Alpha ('ASS') subtitle files, with word-level caption timing, styling presets, and line-break control, and burns subtitles into video files using the 'FFmpeg' subtitles filter.

subtitles: FFmpeg-based Subtitle Authoring and Rendering in R

subtitles provides tools to generate standard subtitle files (SRT, ASS) from structured transcription outputs and to render those subtitles into videos using the FFmpeg command-line interface.

This package does not perform speech-to-text itself. Instead, it works with structured transcription results — for example, those produced by audio.whisper — to produce subtitle artifacts and to orchestrate FFmpeg for subtitle rendering.

The core design principles are:

• CLI-first: FFmpeg is the authoritative engine for subtitle rendering.
• Transparent: The package surfaces process invocation and errors directly.
• Artifact-focused: Subtitle files (.srt, .ass) are first-class citizens.

Quick Start

Prerequisites

You must have FFmpeg installed and available on your system PATH. A standard FFmpeg build with subtitle filter support (libass) is required.

Verify installation:

ffmpeg -version

Installation

This package is available on GitHub:

# install remotes if you don’t have it
install.packages("remotes")

# install subtitles from GitHub
remotes::install_github("cornball-ai/subtitles")

Basic Workflow

1. Generate structured transcription output

Use your preferred speech-to-text tool. For example, with audio.whisper:

srt_text <- predict(
  model,
  newdata = "audio.wav",
  language = "en",
  token_timestamps = TRUE
)

This returns a structured transcription object with segments and token timing.

2. Write an SRT subtitle file

whisper_to_srt(srt_text, file = "captions.srt")

This produces a valid SubRip (.srt) file representing the timed segments.

3. Optionally write an ASS subtitle file

ASS format supports richer styling and optional karaoke-style word highlighting:

whisper_to_ass(srt_text, file = "captions.ass", karaoke = TRUE)

4. Burn subtitles into a video

burn_subtitles(
  video     = "input.mp4",
  subtitles = "captions.ass",
  output    = "output_with_subtitles.mp4"
)

You can also pass style overrides:

burn_subtitles(
  "input.mp4",
  "captions.srt",
  "output.mp4",
  style = "FontSize=36,Outline=2,Shadow=1"
)

Function Reference

whisper_to_srt(x, file)

Converts a structured transcription object into a SubRip .srt file.

• x: Object returned from a transcription tool with segments.
• file: Path where the SRT file will be saved.

whisper_to_ass(x, file, karaoke = TRUE)

Converts a structured transcription object into an Advanced SubStation Alpha (.ass) file.

• karaoke: If TRUE, include token-level timing for word highlighting.

burn_subtitles(video, subtitles, output, style = NULL)

Invokes the FFmpeg CLI to render subtitles into a video.

• video: Path to the source video.
• subtitles: Path to .srt or .ass subtitle file.
• output: Path where the subtitled video will be saved.
• style: Optional ASS style overrides (passed to FFmpeg’s filter).

Call-record sidecars

Every file these functions write gets a JSON record at <output>.json: the resolved call (defaults filled in) plus a media block describing the delivered file (event count and time span for subtitles; fps, dimensions, and duration for video). Downstream tooling can lift these records into edit metadata. Disable with options(subtitles.sidecar = FALSE).

Why This Package Exists

Existing R packages for subtitle manipulation either focus on parsing (SRTtools) or linguistics. There is no dedicated, FFmpeg-first R tool for generating subtitle artifacts from structured transcriptions and invoking FFmpeg to render them.

subtitles fills that gap by providing:

• SRT / ASS subtitle generation
• Optional karaoke timing
• Transparent, reproducible video rendering

Philosophy

This package embraces the fact that the FFmpeg command line is the authoritative interface for video processing. Rather than hiding or abstracting it, subtitles builds on it:

• Arguments are surfaced directly
• Errors from FFmpeg are passed through
• Users can drop down to raw CLI if needed

This aligns with how video professionals work and ensures your R code stays in sync with FFmpeg documentation and community tooling.

System Requirements

• FFmpeg with subtitle filter support (libass)
• processx (for process execution)
• Compatible with Windows, macOS, Linux

Contributing

Contributions are welcome. Suggested areas:

• Additional styling presets
• Automatic line breaking
• Support for other transcription outputs (non-Whisper)
• CRAN packaging and documentation improvements

License

MIT

Acknowledgments

This package is not a speech-to-text provider. It is designed to integrate with tools such as:

• audio.whisper
• Other R wrappers around ASR systems