Evolutionary Feature Engineering

Automates feature engineering using evolutionary algorithms inspired by genetic programming. Starting from raw input features, the package evolves candidate transformation recipes through selection, crossover, and mutation, evaluating fitness via cross-validation or train/validation splits with gradient-boosted tree models ('LightGBM' or 'XGBoost'). Built-in transformers include arithmetic, logarithmic, and power operations, interaction terms, target encoding, quantile and log-based binning, principal component analysis, truncated singular value decomposition, Uniform Manifold Approximation and Projection (UMAP) dimensionality reduction, and minimum spanning tree (MST) graph-based clustering. The evolutionary search yields an optimised feature recipe that can be applied to new data for prediction. Methods are described in McInnes et al. (2018) , Ke et al. (2017) , Chen and Guestrin (2016) , Gagolewski (2021) , Gagolewski (2026) , and Gagolewski (2026) .

evoFE: Evolutionary Feature Engineering in R

evoFE (Evolutionary Feature Engineering) is an R package that uses a genetic algorithm to automatically discover, combine, and optimize feature transformations for tabular datasets. Instead of manually engineering interaction terms, ratios, or binning strategies, evoFE searches the space of possible feature recipes to maximize the predictive performance of LightGBM, XGBoost, or other ML models.

The final output is a reusable evo_recipe object that can be easily applied to new data at prediction time.

Features

• Genetic Algorithm Optimization: Searches the feature transformation space using selection, crossover, and mutation.
• Hierarchical Chaining: Evolved features can build on top of other proven features from previous generations (e.g., log(ratio(x1, x2))).
• Hybrid Active Feature Mask: The genetic search simultaneously selects which original raw features to include and what transformations to apply, guided by feature importances with temperature-scaled sigmoid sampling.
• 42 Built-in Transformers: Arithmetic, group-by aggregations, supervised encodings, dimensionality reduction (PCA, SVD, UMAP), and manifold/graph learning (Genie, Lumbermark, MST, Deadwood).
• Stateful Transformers: Includes PCA, SVD, UMAP, Genie Clustering, Lumbermark Clustering, and Deadwood Anomaly Detection, all fit on training data and cached for efficiency.
• Performance Caching: Features are cached using matrix-hashing to avoid redundant computations (like KNN search or UMAP projections) during cross-validation folds.
• Island Model: Run independent sub-populations with periodic recipe-level and gene-level migration for broader exploration of the search space.
• Flexible Evaluation: Supports both Cross-Validation (cv) and stratified Train/Validation/Holdout Split (split) strategies.
• Extensible Custom Registry: Register user-defined transformers with register_transformer() or custom ML backends with register_evaluator().
• Bayesian Hyperparameter Tuning: Wrap any registered evaluator in an mlr3mbo Bayesian optimization loop via make_tunable().
• Alternative & Custom Metrics: Optimize for standard metrics (LogLoss, AUC, F1, MAE, TS-Refinement) or pass a custom fitness function.
• Rich S3 Interface: print(), summary(), and plot() to inspect and visualize the evolution.
• Live Evolution Viewer: A real-time browser dashboard that streams generation-by-generation progress when record = TRUE.

Installation

You can install the released version of evoFE from CRAN with:

install.packages("evoFE")

Alternatively, you can install the development version directly from GitHub:

# Install devtools if you haven't already
# install.packages("devtools")

# Install evoFE from GitHub
devtools::install_github("tanopereira/evoFE", build_vignettes = TRUE)

macOS OpenMP Configuration (Recommended)

Several of evoFE's core transformers (like Genie and Lumbermark clustering) are implemented in C++ and parallelized using OpenMP. On macOS, R packages compile single-threaded by default. To enable multi-threading:

1. Install libomp via Homebrew:

   brew install libomp
   

2. Configure your ~/.R/Makevars file to use OpenMP:

   SHLIB_OPENMP_CFLAGS = -Xpreprocessor -fopenmp
   SHLIB_OPENMP_CXXFLAGS = -Xpreprocessor -fopenmp
   CPPFLAGS += -I/opt/homebrew/opt/libomp/include
   LDFLAGS += -L/opt/homebrew/opt/libomp/lib -lomp
   

3. Reinstall quitefastmst, genieclust, lumbermark, and deadwood from source:

   install.packages(c("quitefastmst", "genieclust", "lumbermark", "deadwood"), type = "source")
   

Quick Start

Here is a quick example using the mtcars dataset for a binary classification task:

library(evoFE)

data(mtcars)
df <- mtcars
df$am <- as.integer(df$am) # target: 0 = automatic, 1 = manual

# Evolve features
set.seed(42)
recipe <- evolve_features(
  data = df,
  target_col = "am",
  task = "classification",
  evaluator = "xgboost",
  generations = 5,
  pop_size = 8,
  cv_folds = 3,
  verbose = TRUE
)

# View the winning recipe overview and detailed summary
print(recipe)
summary(recipe)

# Plot the evolution fitness curve
plot(recipe, type = "fitness")

# Engineer features on new data
engineered_df <- predict(recipe, df[1:5, ])

# Run predictions using the trained model
predictions <- predict_model(recipe, df[1:5, ])

Supported Transformers

evoFE ships with 42 built-in transformers that the genetic algorithm can select from during evolution.

License

This project is licensed under the MIT License - see the LICENSE file for details.