Combining Tree-Boosting with Gaussian Process and Mixed Effects Models

An R package that allows for combining tree-boosting with Gaussian process and mixed effects models. It also allows for independently doing tree-boosting as well as inference and prediction for Gaussian process and mixed effects models. See < https://github.com/fabsig/GPBoost> for more information on the software and Sigrist (2022, JMLR) < https://www.jmlr.org/papers/v23/20-322.html> and Sigrist (2023, TPAMI) for more information on the methodology.

GPBoost R Package

This is the R package implementation of the GPBoost library. See https://github.com/fabsig/GPBoost for more information on the modeling background and the software implementation.

Table of Contents

• Examples
• Installation
  • Installation from CRAN
  • Installation from source
• Testing
• Preparing a CRAN package

Examples

• R examples
  • GPBoost / LaGaBoost algorithm for Gaussian ("regression") and non-Gaussian likelihoods (e.g., "classification", etc.)
  • Generalized linear Gaussian process and mixed effects models

Installation

Installation from CRAN

The gpboost package is available on CRAN and can be installed as follows:

install.packages("gpboost", repos = "https://cran.r-project.org")

Installation from source

It is much easier to install the package from CRAN. However, the package can also be build from source as described in the following. In short, the main steps for installation are the following ones:

• Install git
• Install CMake
• Install Rtools (for Windows only)
  • Choose the option 'add rtools to system PATH' or manually add it (-> 'Edit the system environment variables' -> 'Environment Variables' -> add, e.g., "C:\rtools45\usr\bin" to 'Path')
  • You might want to remove the cmake version that comes with Rtools as this might get priority, but it can be older than the required minimal versions
• Make sure that you have an appropriate C++ compiler (see below for more details). E.g. for Windows, simply download the free Visual Studio Community Edition and do not forget to select 'Desktop development with C++' when installing it
• Install the GPBoost package from the command line using:

git clone --recursive https://github.com/fabsig/GPBoost
cd GPBoost
Rscript build_r.R

Below is a more complete installation guide.

Preparation

You need to install git and CMake first. Note that 32-bit R/Rtools is not supported for custom installation.

Windows Preparation

NOTE: Windows users may need to run with administrator rights (either R or the command prompt, depending on the way you are installing this package).

Installing a 64-bit version of Rtools is mandatory.

After installing Rtools and CMake, be sure the following paths are added to the environment variable PATH. These may have been automatically added when installing other software.

• Rtools

  • If you have Rtools 3.x, for example:
    • C:\Rtools\mingw_64\bin
  • If you have Rtools 4.0, for example:
    • C:\rtools40\mingw64\bin
    • C:\rtools40\usr\bin
  • If you have Rtools 4.2+, for example:
    • C:\rtools42\x86_64-w64-mingw32.static.posix\bin
    • C:\rtools42\usr\bin
    • NOTE: this is, e.g., rtools43\ for R 4.3
  • NOTE:
    • Two Rtools paths are required from Rtools 4.0 onwards (see above)
    • When installing in R with install.packages(), these paths can be added locally in R, e.g., as follows prior to installation: Sys.setenv(PATH=paste0(Sys.getenv("PATH"),";C:\\Rtools\\mingw_64\\bin\\;C:\\rtools40\\usr\\bin\\"))
    • You might need to install the processx R package first for RTools 4.4

• CMake

  • example: C:\Program Files\CMake\bin

• R

  • example: C:\Program Files\R\R-3.6.1\bin

The default compiler is Visual Studio (or VS Build Tools) in Windows, with an automatic fallback to MingGW64 (i.e. it is enough to only have Rtools and CMake). To force the usage of MinGW64, you can add the --use-mingw (for R 3.x) or --use-msys2 (for R 4.x) flags (see below).

Mac OS Preparation

You can perform installation either with Apple Clang or gcc.

• In case you prefer Apple Clang, you should install OpenMP (details for installation can be found in the Installation Guide) first and CMake version 3.12 or higher is required. Only Apple Clang version 8.1 or higher is supported.
• In case you prefer gcc, you need to install it (details for installation can be found in the Installation Guide) and set some environment variables to tell R to use gcc and g++. If you install these from Homebrew, your versions of g++ and gcc are most likely in /usr/local/bin, as shown below.

# replace 8 with version of gcc installed on your machine
export CXX=/usr/local/bin/g++-8 CC=/usr/local/bin/gcc-8

Install

Build and install the R package with the following commands:

git clone --recursive https://github.com/fabsig/GPBoost
cd GPBoost
Rscript build_r.R

The build_r.R script builds the package in a temporary directory called gpboost_r. It will destroy and recreate that directory each time you run the script. That script supports the following command-line options:

• --skip-install: Build the package tarball, but do not install it
• --use-mingw: Force the use of MinGW toolchain, regardless of R version
• --use-msys2: Force the use of MSYS2 toolchain, regardless of R version

Note: for the build with Visual Studio/VS Build Tools in Windows, you should use the Windows CMD or PowerShell.

Build CUDA Version

GPBoost can optionally be built with CUDA support for the Gaussian process (GP) component. CUDA support must be enabled at compile time and requires a working NVIDIA CUDA toolchain.

Environment setup (recommended)

Example (Linux):

export CUDA_HOME=/usr/local/cuda
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

If CUDA is installed in a non-standard location:

export CUDAToolkit_ROOT=/path/to/cuda

Build

git clone --recursive https://github.com/fabsig/GPBoost
cd GPBoost
Rscript build_r.R --use-cudagp

For macOS and Windows users, the CUDA version is not supported.

CUDA library is needed: details for installation can be found in Installation Guide.

Testing

There is currently no integration service set up that automatically runs unit tests. However, any contribution needs to pass all unit tests in the R-package/tests/testthat directory. These tests can be run using the run_tests_coverage_R_package.R file. In any case, make sure that you run the full set of tests by specifying the following environment variable (see the run_tests_coverage_R_package.R file):

Sys.setenv(GPBOOST_ALL_TESTS = "GPBOOST_ALL_TESTS")

before running the tests in the R-package/tests/testthat directory.

Checks that need to be done for every change

Whenever something new is added or changed, the following checks should be run in addition to the unit tests above.

1. CRAN compatibility

Build the package with sh build-cran-package.sh (see Build a CRAN Package) and check it with

R CMD check --as-cran gpboost_*.tar.gz

2. Compiler warnings

Compile the C++ code with an extensive set of compiler warnings enabled:

sh helpers/check_compiler_warnings.sh

Use sh helpers/check_compiler_warnings.sh --summary to only get the counts per warning type. The script exits with a non-zero status if any warning is emitted.

3. AddressSanitizer (ASan) and UndefinedBehaviorSanitizer (UBSan) checks

The ASan and UBSan checks detect, e.g., buffer overflows, use-after-free, signed integer overflow, and invalid shifts. They are run in the same Docker containers that R-hub and CRAN use. First create the package tarball with sh build-cran-package.sh, then run the docker run ... commands documented in check_R_package_rhub.R. That file also documents how to install Docker, where the results are written, and how to interpret them. Note that these checks are slow (the entire C++ code is recompiled inside the container with instrumentation).

Preparing a CRAN package

This section is primarily for maintainers, but may help users and contributors to understand the structure of the R package. Most of GPBoost uses CMake to handle tasks like setting compiler and linker flags, including header file locations, and linking to other libraries. Because CRAN packages typically do not assume the presence of CMake, the R package uses an alternative method that is in the CRAN-supported toolchain for building R packages with C++ code: Autoconf. For more information on this approach, see "Writing R Extensions".

Build a CRAN Package

From the root of the repository, run the following:

sh build-cran-package.sh

This will create a file gpboost_${VERSION}.tar.gz, where VERSION is the version of GPBoost.

Installation of a CRAN package

After building the package, you can install it with the following command:

R CMD INSTALL gpboost_*.tar.gz

Testing a CRAN package

After building the package, you can test the CRAN package as follows:

R CMD check --as-cran gpboost_*.tar.gz