Conditional Predictive Impact

A general test for conditional independence in supervised learning algorithms as proposed by Watson & Wright (2021) . Implements a conditional variable importance measure which can be applied to any supervised learning algorithm and loss function. Provides statistical inference procedures without parametric assumptions and applies equally well to continuous and categorical predictors and outcomes.

Conditional Predictive Impact

David S. Watson, Marvin N. Wright

Introduction

The conditional predictive impact (CPI) is a measure of conditional independence. It can be calculated using any supervised learning algorithm, loss function, and knockoff sampler. We provide statistical inference procedures for the CPI without parametric assumptions or sparsity constraints. The method works with continuous and categorical data.

Installation

To install the ranger R package from CRAN, just run

install.packages("cpi")

To install the development version from GitHub using devtools, run

devtools::install_github("bips-hb/cpi")

Examples

Calculate CPI for random forest on iris data with 5-fold cross validation:

library(mlr3)
library(mlr3learners)
library(cpi)

cpi(task = tsk("iris"), 
    learner = lrn("classif.ranger", predict_type = "prob"),
    resampling = rsmp("cv", folds = 5), 
    measure = "classif.logloss", test = "t")

References

• Watson D. S. & Wright, M. N. (2021). Testing conditional independence in supervised learning algorithms. Machine Learning. DOI: 10.1007/s10994-021-06030-6.