Detecting, Decomposing, and Stress-Testing Temporal Change in Repeated Decision Systems

Tools for detecting, decomposing, and stress-testing temporal drift in repeated binary decision systems. Complements the 'decisionpaths' package by shifting focus from path construction to system-level change over time. Implements five core analytic modules: (1) prevalence drift — did the overall decision rate change over time?; (2) transition drift — did the probability of switching or persisting change?; (3) entropy and stability trends — did path complexity evolve?; (4) group-differential drift — did the system drift differently across subgroups?; (5) change-point and regime-shift detection — did the system change abruptly after a policy or model update? Additionally provides a robustness module for testing stability of drift conclusions across analytic choices, and a sensitivity module for probing vulnerability to data problems including missingness, miscoding, and threshold shifts. Defines four original drift indices: the Decision Drift Index (DDI), Transition Drift Index (TDI), Group Differential Drift (GDD), and Cumulative Drift Burden (CDB). Applications include algorithmic audit, AI governance, education, health, and organisational research.

DecisionDrift

Detecting, Decomposing, and Stress-Testing Temporal Change in Repeated Decision Systems

Overview

DecisionDrift is an R package for auditing temporal drift in repeated binary decision systems — such as algorithmic screening tools, institutional review boards, or any AI system that issues repeated yes/no decisions over time.

It complements the decisionpaths package by shifting focus from path construction to system-level change:

Installation

From GitHub (recommended)

# install.packages("remotes")
remotes::install_github("causalfragility-lab/DecisionDrift")

From CRAN (once accepted)

install.packages("DecisionDrift")

Quick start

library(DecisionDrift)

# --- 1. Build a drift panel object -----------------------------------------
dat <- data.frame(
  id       = rep(1:100, each = 8),
  time     = rep(1:8, times = 100),
  decision = rbinom(800, 1, rep(seq(0.25, 0.55, length.out = 8), 100)),
  group    = rep(c("A", "B"), 50, each = 1)
)

dp <- dd_build(dat, id, time, decision,
               group      = group,
               event_time = 5L)   # known policy change at wave 5

# --- 2. Run the full audit in one call --------------------------------------
aud <- dd_audit(dp)
print(aud)
plot(aud)

# --- 3. Or call individual modules ------------------------------------------
dd_prevalence(dp)      # Was the overall decision rate stable?
dd_transition(dp)      # Did switching/persistence dynamics change?
dd_entropy_trend(dp)   # Did path complexity evolve?
dd_group_drift(dp)     # Did drift differ across groups?
dd_changepoint(dp)     # Where did the system break?

# --- 4. Summary drift indices -----------------------------------------------
dd_indices(dp)
# DDI = Decision Drift Index
# TDI = Transition Drift Index
# GDD = Group Differential Drift
# CDB = Cumulative Drift Burden

# --- 5. Stress-test conclusions ---------------------------------------------
dd_robustness(dp)      # Are conclusions stable across analytic choices?
dd_sensitivity(dp)     # How vulnerable are conclusions to data problems?

Package architecture

DecisionDrift ├── dd_build() Core: build drift_panel object │ ├── Module 1: dd_prevalence() Drift in decision rate → DDI ├── Module 2: dd_transition() Drift in transition structure → TDI ├── Module 3: dd_entropy_trend() Drift in path complexity / stability ├── Module 4: dd_group_drift() Group-differential drift → GDD ├── Module 5: dd_changepoint() Change-point & regime detection │ ├── dd_indices() All four indices in one call (DDI, TDI, GDD, CDB) │ ├── dd_robustness() Stability across: balanced panel, LOPO, │ LOGO, min_waves grid, bootstrap CIs │ ├── dd_sensitivity() Vulnerability to: miscoding, missingness, │ threshold shifts, composition shifts │ └── dd_audit() Flagship: 3-layer audit → verdict

The four drift indices

The three-layer audit

dd_audit() follows a deliberate three-layer logic:

1. Detection — Did the process drift? dd_prevalence(), dd_transition()

2. Decomposition — Was drift due to prevalence change, transition instability, subgroup divergence, or a regime shift? dd_entropy_trend(), dd_group_drift(), dd_changepoint()

3. Stress-testing — Are conclusions robust to noise, coding choices, and missing waves? dd_robustness(), dd_sensitivity()

The audit closes with a single verdict: "no drift detected", "marginal drift", "moderate drift", or "strong drift".

Relationship to decisionpaths

decisionpaths answers: What paths occurred, how stable, how entropic, how equitable?

DecisionDrift answers: Did the system itself change, when, how much, for whom, and is that conclusion robust?

They are designed to be used together: build paths with decisionpaths::dp_build(), audit drift with DecisionDrift::dd_audit().

A third package in the ecosystem, AIBias, will ask: Did inequality accumulate or amplify over repeated decisions?

Dependencies

Imports: cli, rlang, stats, tibble

Suggests (for plotting): ggplot2, patchwork

Citation

Hait, S. (2026). DecisionDrift: Detecting, decomposing, and stress-testing temporal change in repeated decision systems. R package version 0.1.0. https://github.com/causalfragility-lab/DecisionDrift

License

MIT © Subir Hait