Hyperparameter Search for Gradient Boosted Trees

An implementation of hyperparameter optimization for Gradient Boosted Trees on binary classification and regression problems. The current version provides two optimization methods: Bayesian optimization and random search. Instead of giving the single best model, the final output is an ensemble of Gradient Boosted Trees constructed via the method of ensemble selection.


An implementation of hyperparameter optimization for Gradient Boosted Trees on binary classification and regression problems. The current version supports two optimization methods: Bayesian optimization and random search. Instead of returning the single best model, the final output is an ensemble of Gradient Boosted Trees constructed via the method of ensemble selection.

Example

Binary Classification

data(german_credit)
train <- german_credit$train
test <- german_credit$test
target_idx <- german_credit$target_idx
pred_idx <- german_credit$pred_idx
 
# Train a GBT model with optimization on AUC
model <- gbts(train[, pred_idx], train[, target_idx], nitr = 200, pfmc = "auc")
 
# Predict on test data
yhat_test <- predict(model, test[, pred_idx])
 
# Compute AUC on test data
comperf(test[, target_idx], yhat_test, pfmc = "auc")

Regression

# Load Boston housing data
data(boston_housing)
train <- boston_housing$train
test <- boston_housing$test
target_idx <- boston_housing$target_idx
pred_idx <- boston_housing$pred_idx
 
# Train a GBT model with optimization on MSE
model <- gbts(train[, pred_idx], train[, target_idx], nitr = 200, pfmc = "mse")
 
# Predict on test data
yhat_test <- predict(model, test[, pred_idx])
 
# Compute MSE on test data
comperf(test[, target_idx], yhat_test, pfmc = "mse")

Installation

To get the current released version from CRAN:

install.packages("gbts")

Main Components

To see a list of functions and datasets provided by gbts:

help(package = "gbts")

News

gbts 1.2.0

  • Implemented ensemble selection to construct an ensemble of models in the output of gbts().

  • Revised the API of gbts() to simplify the specification of minimum and maximum values of hyperparameters.

  • Improved the display of optimization progress.

  • Terminated support for the R package "xgboost".

gbts 1.0.1

  • Modified access to gbm predict() to be compatible with the current and next version of gbm.

  • Revised the documentation of gbts() and the DESCRIPTION file to describe Bayesian optimization in replacement of active learning. This is merely a documentation change for the same algorithm.

  • Allowed the "srch" argument of gbts() to accept "bayes" for Bayesian optimization.

  • Revised the description of the "cutoff" argument of gbts().

  • Changed R (>= 3.3.1) in the DESCRIPTION file to R (>= 3.3.0) to address the installation error from OS X on CRAN, which uses R version 3.3.0 at the time.

gbts 1.0.0

  • This is the initial release.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("gbts")

1.2.0 by Waley W. J. Liang, 2 years ago


Browse source code at https://github.com/cran/gbts


Authors: Waley W. J. Liang


Documentation:   PDF Manual  


GPL (>= 2) | file LICENSE license


Imports doParallel, doRNG, foreach, gbm, earth

Suggests testthat


See at CRAN