An implementation of hyperparameter optimization for Gradient Boosted Trees on binary classification and regression problems. The current version provides two optimization methods: Bayesian optimization and random search. Instead of giving the single best model, the final output is an ensemble of Gradient Boosted Trees constructed via the method of ensemble selection.
An implementation of hyperparameter optimization for Gradient Boosted Trees on binary classification and regression problems. The current version supports two optimization methods: Bayesian optimization and random search. Instead of returning the single best model, the final output is an ensemble of Gradient Boosted Trees constructed via the method of ensemble selection.
data(german_credit)train <- german_credit$traintest <- german_credit$testtarget_idx <- german_credit$target_idxpred_idx <- german_credit$pred_idx# Train a GBT model with optimization on AUCmodel <- gbts(train[, pred_idx], train[, target_idx], nitr = 200, pfmc = "auc")# Predict on test datayhat_test <- predict(model, test[, pred_idx])# Compute AUC on test datacomperf(test[, target_idx], yhat_test, pfmc = "auc")
# Load Boston housing datadata(boston_housing)train <- boston_housing$traintest <- boston_housing$testtarget_idx <- boston_housing$target_idxpred_idx <- boston_housing$pred_idx# Train a GBT model with optimization on MSEmodel <- gbts(train[, pred_idx], train[, target_idx], nitr = 200, pfmc = "mse")# Predict on test datayhat_test <- predict(model, test[, pred_idx])# Compute MSE on test datacomperf(test[, target_idx], yhat_test, pfmc = "mse")
To get the current released version from CRAN:
install.packages("gbts")
To see a list of functions and datasets provided by gbts:
help(package = "gbts")
Implemented ensemble selection to construct an ensemble of models in the output of gbts().
Revised the API of gbts() to simplify the specification of minimum and maximum values of hyperparameters.
Improved the display of optimization progress.
Terminated support for the R package "xgboost".
Modified access to gbm predict() to be compatible with the current and next version of gbm.
Revised the documentation of gbts() and the DESCRIPTION file to describe Bayesian optimization in replacement of active learning. This is merely a documentation change for the same algorithm.
Allowed the "srch" argument of gbts() to accept "bayes" for Bayesian optimization.
Revised the description of the "cutoff" argument of gbts().
Changed R (>= 3.3.1) in the DESCRIPTION file to R (>= 3.3.0) to address the installation error from OS X on CRAN, which uses R version 3.3.0 at the time.