Gradient Boosting

Functional gradient descent algorithm for a variety of convex and non-convex loss functions, for both classical and robust regression and classification problems. See Wang (2011) , Wang (2012) , Wang (2018) , Wang (2018) .


            CHANGES in `bst' VERSION 0.2 (02-12-2011)

o multi-class HingeBoost via one-vs-all method

            CHANGES in `bst' VERSION 0.3-1 (08-8-2011)

o multi-class HingeBoost msvm o add misclassification error as a criteria in cv.msvm for multiclass problem

            CHANGES in `bst' VERSION 0.3-2 (01-17-2012)

o added trees size selection in multi-class HingeBoost msvm

            CHANGES in `bst' VERSION 0.3-3 (09-12-2013)

o function name mhinge was changed to mhingeova (multi-class hinge one-vs-all) o function name msvm was changed to mhingebst (multi-class hinge boosting)

            CHANGES in `bst' VERSION 0.3-4 (05-8-2014)

o change output name fraction to mstop in cv.bst

            CHANGES in `bst' VERSION 0.3-5 (07-14-2014)

o for non-numeric responses in mada, they are now forced to be numeric o fixed bugs in mada when xte and yte are missing o in DESCRIPTION, change Imports: gbm to Depends: gbm, the results can be different for mada

            CHANGES in `bst' VERSION 0.3-6 (12-23-2014)

o intercept estimated o hingeloss and hingengra were modified, they are for family="hinge" since version 0.3-11. Before version 0.3-6, they are for family="hinge2". o function rbst added: boosting plus difference of convex algorithm with truncated loss functions

            CHANGES in `bst' VERSION 0.3-7 (03-14-2015)

o add intercept estimate for family=gaussian and center=FALSE

            CHANGES in `bst' VERSION 0.3-8 (04-06-2015)

o add family="expo" in bst function and rfamily="texpo" in rbst function for exponential loss and truncated exponential loss, respectively o terminate boosting and robust boosting if loss value inreases

            CHANGES in `bst' VERSION 0.3-9 (06-19-2015)

o add robust multi-class boosting rmbst.R

            CHANGES in `bst' VERSION 0.3-10 (08-13-2015)

o change the algorithm with respect to base class in mbst.R o change bst function with twinboost=TRUE for learner="ls". In the previous versions, bst implemented the General Twin L2Boosting with general weak learner (Alg 1), rathter than Twin L2Boosting with componentwise linear least squares (Alg 2), cf: Buhlmann and Hothorn (2010). However, as Proposition 2 demonstrates, Alg 1 is not equal to Alg2 unless underlying regression model is orthonormal. The change is to match Alg 1.

            CHANGES in `bst' VERSION 0.3-11 (12-07-2015)

o add parallel computing for cross-validation in cv.bst function o add twintype argument for learner="ls" and twinboost=TRUE. If twintype=2, it does the same before and excluding version 0.3-10. If twintype=1, it does the same as version 0.3-10 o change argument "type" in bst and related functions to "threshold" o add vignette file o in function bst, boosting was terminated early if the fitted loss function value increased. This was changed and boosting only terminates for those loss functions related to truncated loss functions o add inst/extdata and two vignettes

            CHANGES in `bst' VERSION 0.3-12 (12-23-2015)

o add a data analysis example to khan.Rnw o add a new vignette mcl.Rnw o modify ex1data function o fix a bug in predict.mbst when newdata=NULL and newy=NULL

            CHANGES in `bst' VERSION 0.3-13 (01-07-2016)

o recall terminating boosting if loss value inreases in bst function, but still terminating for rbst if loss value increases o check if response is 1/-1 in bst and rbst for (margin based) binary classification problem o modify cv.bst to be computationally efficient using predict function more properly o add cv.rbst and cv.rmbst o change type=c("risk", "misc") to type=c("loss", "error") in cross validation related functions, to be consistent with predict functions o add cv.rbst o add index.html to inst/doc

            CHANGES in `bst' VERSION 0.3-14 (03-12-2016)

o add family="closs", "clossMM", "clossR", "clossRMM", "gloss", "glossMM", "qloss", "qlossMM" in bst function o add rfamily="closs", "gloss", "qloss" in rbst function o add family="closs", "clossMM" in mbst function o add rfamily="closs" in rmbst function o add vignette files for breast cancer data analysis

            CHANGES in `bst' VERSION 0.3-15 (07-31-2017)

o stop bst running if any fk=NULL and family="tgaussianDC", "thingeDC", "tbinomDC", "binomdDC", "texpoDC", "tpoissonDC", "thuberDC", "clossR", "clossRMM", "clossMM", "glossMM", "qlossMM")) o importFrom("doParallel", "stopImplicitCluster") o add stopImplicitCluster() after registerDoParallel o change the offset value for family="huber", "thuber" or "lar". This is the initial value of boosting in bst function o add a new parameter 'start' in bst_control, a logical value. If start=TRUE and fk in bst_control is a vector of values, then bst iterations begin with fk. Otherwise, bst iterations begin with the default values. This can be useful, for instance, in rbst for the MM boosting algorithm. o drop bst-package.Rd in man directory

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.


0.3-16 by Zhu Wang, a month ago

Browse source code at

Authors: Zhu Wang [aut, cre] , Torsten Hothorn [ctb]

Documentation:   PDF Manual  

Task views: Machine Learning & Statistical Learning

GPL (>= 2) license

Imports rpart, methods, foreach, doParallel, gbm

Suggests hdi, pROC, R.rsp, knitr, gdata

Imported by bujar, mpath.

Suggested by fscaret, mlr.

See at CRAN