A Toolbox for Conditional Inference Trees and Random Forests

Additions to 'party' and 'partykit' packages : tools for the interpretation of forests (surrogate trees, prototypes, etc.), feature selection (see Gregorutti et al (2017) , Hapfelmeier and Ulm (2013) , Altmann et al (2010) ) and parallelized versions of conditional forest and variable importance functions. Also modules and a shiny app for conditional inference trees.

Variable Importance Measures for Multivariate Random Forests

Calculates two sets of post-hoc variable importance measures for multivariate random forests. The first set of variable importance measures are given by the sum of mean split improvements for splits defined by feature j measured on user-defined examples (i.e., training or testing samples). The second set of importance measures are calculated on a per-outcome variable basis as the sum of mean absolute difference of node values for each split defined by feature j measured on user-defined examples (i.e., training or testing samples). The user can optionally threshold both sets of importance measures to include only splits that are statistically significant as measured using an F-test.

A Unified Framework for Random Forest Prediction Error Estimation

Estimates the conditional error distributions of random forest predictions and common parameters of those distributions, including conditional misclassification rates, conditional mean squared prediction errors, conditional biases, and conditional quantiles, by out-of-bag weighting of out-of-bag prediction errors as proposed by Lu and Hardin (2021). This package is compatible with several existing packages that implement random forests in R.

Significance Level for Random Forest Impurity Importance Scores

Sets a significance level for Random Forest MDI (Mean Decrease in Impurity, Gini or sum of squares) variable importance scores, using an empirical Bayes approach. See Dunne et al. (2022) .

ROSE Random Forests for Robust Semiparametric Efficient Estimation

ROSE (RObust Semiparametric Efficient) random forests for robust semiparametric efficient estimation in partially parametric models (containing generalised partially linear models). Details can be found in the paper by Young and Shah (2024) .

Bootstrap Stacking of Random Forest Models for Heterogeneous Data

Generates and predicts a set of linearly stacked Random Forest models using bootstrap sampling. Individual datasets may be heterogeneous (not all samples have full sets of features). Contains support for parallelization but the user should register their cores before running. This is an extension of the method found in Matlock (2018) .

Projection Pursuit Oblique Decision Trees and Random Forests

Builds decision trees by splitting on linear combinations of randomly chosen variables. Projection pursuit is used to choose a projection of the variables that best separates the groups. Using linear combinations of variables to separate groups takes the correlation between variables into account, which allows the model to outperform a traditional decision tree when the separation between groups occurs in combinations of variables. Single trees can be assembled into random forests for improved accuracy. Implements projection pursuit classification trees (Lee, Cook, Park and Lee (2013) ) and projection pursuit forests (da Silva, Cook and Lee (2021) ), following the earlier 'PPforest' package.

Explaining and Visualizing Random Forests in Terms of Variable Importance

A set of tools to help explain which variables are most important in a random forests. Various variable importance measures are calculated and visualized in different settings in order to get an idea on how their importance changes depending on our criteria (Hemant Ishwaran and Udaya B. Kogalur and Eiran Z. Gorodeski and Andy J. Minn and Michael S. Lauer (2010) , Leo Breiman (2001) ).

Random Forest Regression with Arvind Distribution Error Model

Implements Random Forest regression under the Arvind distribution error model. Provides core distribution functions (density, cumulative distribution, quantile, random generation, hazard, survival), parameter estimation via Expectation-Maximization (EM) and Markov Chain Monte Carlo (MCMC), non-parametric bootstrap confidence intervals (at 90%, 95%, and 99% levels), Highest Posterior Density (HPD) intervals, model evaluation metrics (estimated values, bias, mean squared error, risk value), homoscedastic prediction intervals, and goodness-of-fit diagnostic tests (Kolmogorov-Smirnov and Anderson-Darling tests, Akaike Information Criterion, and Bayesian Information Criterion). References: Breiman (2001) ; Wright and Ziegler (2017) .

Mixed Effect Random Forests for Small Area Estimation

Mixed Effects Random Forests (MERFs) are a data-driven, nonparametric alternative to current methods of Small Area Estimation (SAE). 'SAEforest' provides functions for the estimation of regionally disaggregated linear and nonlinear indicators using survey sample data. Included procedures facilitate the estimation of domain-level economic and inequality metrics and assess associated uncertainty. Emphasis lies on straightforward interpretation and visualization of results. From a methodological perspective, the package builds on approaches discussed in Krennmair and Schmid (2022) and Krennmair et al. (2022) .