Bayesian Logistic Regression with Heavy-Tailed Priors

Efficient Bayesian multinomial logistic regression based on heavy-tailed (hyper-LASSO, non-convex) priors. The posterior of coefficients and hyper-parameters is sampled with restricted Gibbs sampling for leveraging the high-dimensionality and Hamiltonian Monte Carlo for handling the high-correlation among coefficients. A detailed description of the method: Li and Yao (2018), Journal of Statistical Computation and Simulation, 88:14, 2827-2851, .

HTLR: Bayesian Logistic Regression with Heavy-tailed Priors

HTLR performs classification and feature selection by fitting Bayesian polychotomous (multiclass, multinomial) logistic regression models based on heavy-tailed priors with small degree freedom. This package is suitable for classification with high-dimensional features, such as gene expression profiles. Heavy-tailed priors can impose stronger shrinkage (compared to Guassian and Laplace priors) to the coefficients associated with a large number of useless features, but still allow coefficients of a small number of useful features to stand out with little punishment. Heavy-tailed priors can also automatically make selection within a large number of correlated features. The posterior of coefficients and hyperparameters is sampled with resitricted Gibbs sampling for leveraging high-dimensionality and Hamiltonian Monte Carlo for handling high-correlations among coefficients.

Installation

CRAN version (recommended):

install.packages("HTLR")

Development version on GitHub:

# install.packages("devtools")
devtools::install_github("longhaiSK/HTLR")

This package uses library Armadillo for carrying out most of matrix operations, you may get speed benefits from using an alternative BLAS library such as ATLAS, OpenBLAS or Intel MKL. Check out this post for the comparison and the installation guide.

Reference

Longhai Li and Weixin Yao (2018). Fully Bayesian Logistic Regression with Hyper-Lasso Priors for High-dimensional Feature Selection. 2018, 88:14, 2827-2851, the published version, or arXiv version.