Stacked Gradient Boosting Machines

A minimalist implementation of model stacking by Wolpert (1992) for boosted tree models. A classic, two-layer stacking model is implemented, where the first layer generates features using gradient boosting trees, and the second layer employs a logistic regression model that uses these features as inputs. Utilities for training the base models and parameters tuning are provided, allowing users to experiment with different ensemble configurations easily. It aims to provide a simple and efficient way to combine multiple gradient boosting models to improve predictive model performance and robustness.

stackgbm

stackgbm offers a minimalist, research-oriented implementation of model stacking (Wolpert, 1992) for gradient boosted tree models built by xgboost (Chen and Guestrin, 2016), lightgbm (Ke et al., 2017), and catboost (Prokhorenkova et al., 2018).

Installation

The easiest way to get stackgbm is to install from CRAN:

install.packages("stackgbm")

Alternatively, to use a new feature or get a bug fix, you can install the development version of stackgbm from GitHub:

# install.packages("remotes")
remotes::install_github("nanxstats/stackgbm")

To install all potential dependencies, check out the instructions from manage dependencies.

Model

stackgbm implements a classic two-layer stacking model: the first layer generates "features" produced by gradient boosting trees. The second layer is a logistic regression that uses these features as inputs.

Related projects

For a more comprehensive and flexible implementation of model stacking, see stacks in tidymodels, mlr3pipelines in mlr3, and StackingClassifier in scikit-learn.

Code of Conduct

Please note that the stackgbm project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.