Averaged Prediction Models

In panel data settings, specifies set of candidate models, fits them to data from pre-treatment validation periods, and selects model as average over candidate models, weighting each by posterior probability of being most robust given its differential average prediction errors in pre-treatment validation periods. Subsequent estimation and inference of causal effect's bounds accounts for both model and sampling uncertainty, and calculates the robustness changepoint value at which bounds go from excluding to including 0. The package also includes a range of diagnostic plots, such as those illustrating models' differential average prediction errors and the posterior distribution of which model is most robust.

apm: Averaged Prediction Models

Introduction

The apm package implements Averaged Prediction Models (APM), a Bayesian model averaging approach for controlled pre-post designs. These designs compare differences over time between a group that becomes exposed (treated group) and one that remains unexposed (comparison group). With appropriate causal assumptions, they can identify the causal effect of the exposure/treatment.

In APM, we specify a collection of models that predict untreated outcomes. Our causal identifying assumption is that the model’s prediction errors would be equal (in expectation) in the treated and comparison groups in the absence of the exposure. This is a generalization of familiar methods like Difference-in-Differences (DiD) and Comparative Interrupted Time Series (CITS).

Because many models may be plausible for this prediction task, we combine them using Bayesian model averaging. We weight each model by its robustness to violations of the causal assumption.

Installation

To install the package from CRAN, use

install.packages("apm")

To install the development version from GitHub, use:

# Install devtools if not already installed
install.packages("remotes")

remotes::install_github("tl2624/apm")

See vignette("apm") for details on using the package.