Bayesian Additive Regression Trees using Bayesian Model Averaging

"BART-BMA Bayesian Additive Regression Trees using Bayesian Model Averaging" (Hernandez B, Raftery A.E., Parnell A.C. (2018) ) is an extension to the original BART sum-of-trees model (Chipman et al 2010). BART-BMA differs to the original BART model in two main aspects in order to implement a greedy model which will be computationally feasible for high dimensional data. Firstly BART-BMA uses a greedy search for the best split points and variables when growing decision trees within each sum-of-trees model. This means trees are only grown based on the most predictive set of split rules. Also rather than using Markov chain Monte Carlo (MCMC), BART-BMA uses a greedy implementation of Bayesian Model Averaging called Occam's Window which take a weighted average over multiple sum-of-trees models to form its overall prediction. This means that only the set of sum-of-trees for which there is high support from the data are saved to memory and used in the final model.

bartBMA

The goal of bartBMA is to provide an implementation of Bayesian Additive Regression Trees Using Bayesian Model Averaging (BART-BMA) (Hernandez et al. 2018)

Hernández, B., Raftery, A. E., Pennington, S. R., & Parnell, A. C. (2018). Bayesian additive regression trees using Bayesian model averaging. Statistics and computing, 28(4), 869-890.

Installation

library(devtools)
install_github("bartBMA")

Example

library(bartBMA)
## basic example code

N <- 100
p<- 100
set.seed(100)

epsilon <- rnorm(N)
xcov <- matrix(runif(N*p), nrow=N)
y <- sin(pi*xcov[,1]*xcov[,2]) + 20*(xcov[,3]-0.5)^2+10*xcov[,4]+5*xcov[,5]+epsilon

epsilontest <- rnorm(N)
xcovtest <- matrix(runif(N*p), nrow=N)
ytest <- sin(pi*xcovtest[,1]*xcovtest[,2]) + 20*(xcovtest[,3]-0.5)^2+10*xcovtest[,4]+5*xcovtest[,5]+epsilontest


bart_bma_example <- bartBMA(x.train = xcov,y.train=y,x.test=xcovtest, 
                    zero_split = 1, only_max_num_trees = 1,split_rule_node = 0)