skpr
Design of Experiments Suite: Generate and Evaluate Optimal Designs
Generates and evaluates D, I, A, Alias, E, T, and G optimal designs. Supports generation and evaluation of blocked and split/split-split/.../N-split plot designs. Includes parametric and Monte Carlo power evaluation functions, and supports calculating power for censored responses. Provides a framework to evaluate power using functions provided in other packages or written by the user. Includes a Shiny graphical user interface that displays the underlying code used to create and evaluate the design to improve ease-of-use and make analyses more reproducible. For details, see Morgan-Wall et al. (2021)
skpr 
Overview
skpr is an open source design of experiments suite for generating and evaluating optimal designs in R. Here is a sampling of what skpr offers:
- Generates and evaluates D, I, A, Alias, E, T, and G optimal designs, as well as user-defined custom optimality criteria.
- Supports generation and evaluation of split/split-split/.../N-split plot designs.
- Includes parametric and Monte Carlo power evaluation functions, and supports calculating power for censored responses.
- Provides an extensible framework for the user to evaluate Monte Carlo power using their own libraries.
- Includes a Shiny graphical user interface, skprGUI, that auto-generates the R code used to create and evaluate the design to improve ease-of-use and enhance reproducibility.
Installation
install.packages("skpr")# To install the latest version from Github:# install.packages("devtools")devtools::install_github("tylermorganwall/skpr")
Functions
gen_design()generates optimal designs from a candidate set, given a model and the desired number of runs.eval_design()evaluates power parametrically for linear models, for normal and split-plot designs.eval_design_mc()evaluates power with a Monte Carlo simulation, for linear and generalized linear models. This function also supports calculating power for split-plot designs using REML.eval_design_survival_mc()evaluates power with a Monte Carlo simulation, allowing the user to specify a point at which the data is censored.eval_design_custom_mc()allows the user to import their own libraries and use the Monte Carlo framework provided by skpr to calculate power.skprGUI()andskprGUIbrowser()opens up the GUI in either R Studio or an external browser.
If addition, the package offers two functions to generate common plots related to designs:
plot_correlations()generates a color map of correlations between variables.plot_fds()generates the fraction of design space plot for a given design.
skprGUI
skprGUI provides an graphical user interface to access all of the main features of skpr. An interactive tutorial is provided to familiarize the user with the available functionality. Type skprGUI() or skprGUIbrowser() to begin. Screenshots:
Usage
library(skpr)#Generate a candidate set of all potential design points to be considered in the experiment#The hypothetical experiment is determining what affects the caffeine content in coffeecandidate_set = expand.grid(temp = c(80,90,100),type = c("Kona","Java"),beansize = c("Large","Medium","Small"))candidate_set#> temp type beansize#> 1 80 Kona Large#> 2 90 Kona Large#> 3 100 Kona Large#> 4 80 Java Large#> 5 90 Java Large#> 6 100 Java Large#> 7 80 Kona Medium#> 8 90 Kona Medium#> 9 100 Kona Medium#> 10 80 Java Medium#> 11 90 Java Medium#> 12 100 Java Medium#> 13 80 Kona Small#> 14 90 Kona Small#> 15 100 Kona Small#> 16 80 Java Small#> 17 90 Java Small#> 18 100 Java Small#Generate the design (default D-optimal)design = gen_design(candidateset = candidate_set,model = ~temp + type + beansize,trials=12)design#> temp type beansize#> 1 80 Kona Medium#> 2 100 Java Small#> 3 80 Java Large#> 4 100 Kona Large#> 5 100 Java Medium#> 6 80 Kona Small#> 7 100 Kona Small#> 8 80 Kona Large#> 9 100 Java Large#> 10 80 Java Medium#> 11 100 Kona Medium#> 12 80 Java Small#Evaluate power for the design with an allowable type-I error of 5%eval_design(RunMatrix = design,model = ~temp + type + beansize,alpha=0.05)#> parameter type power#> 1 (Intercept) effect.power 0.8424665#> 2 temp effect.power 0.8424665#> 3 type effect.power 0.8424665#> 4 beansize effect.power 0.5165386#> 5 (Intercept) parameter.power 0.8424665#> 6 temp parameter.power 0.8424665#> 7 type1 parameter.power 0.8424665#> 8 beansize1 parameter.power 0.5593966#> 9 beansize2 parameter.power 0.5593966#Evaluate power for the design using a Monte Carlo simulation.#Here, we set the effect size (here, the signal-to-noise ratio) to 1.5.eval_design_mc(RunMatrix = design,model = ~temp + type + beansize,alpha=0.05,effectsize=1.5)#> parameter type power#> 1 (Intercept) parameter.power.mc 0.611#> 2 temp parameter.power.mc 0.623#> 3 type1 parameter.power.mc 0.625#> 4 beansize1 parameter.power.mc 0.347#> 5 beansize2 parameter.power.mc 0.338#Evaluate power for the design using a Monte Carlo simulation, for a non-normal response.#Here, we also increase the number of simululations to improve the precision of the results.eval_design_mc(RunMatrix = design,model = ~temp + type + beansize,nsim=5000,glmfamily = "poisson",alpha=0.05,effectsize=c(2,6))#> parameter type power#> 1 (Intercept) parameter.power.mc 0.9964#> 2 temp parameter.power.mc 0.9796#> 3 type1 parameter.power.mc 0.9766#> 4 beansize1 parameter.power.mc 0.8854#> 5 beansize2 parameter.power.mc 0.7088#skpr was designed to operate with the pipe (%>%) in mind.#Here is an example of an entire design of experiments analysis in three lines:library(dplyr)expand.grid(temp = c(80,90,100), type = c("Kona","Java"), beansize = c("Large","Medium","Small")) %>%gen_design(model = ~temp + type + beansize + beansize:type + I(temp^2), trials=24, optimality="I") %>%eval_design_mc(model = ~temp + type + beansize + beansize:type + I(temp^2), alpha=0.05)#> parameter type power#> 1 (Intercept) parameter.power.mc 0.900#> 2 temp parameter.power.mc 0.898#> 3 type1 parameter.power.mc 0.997#> 4 beansize1 parameter.power.mc 0.917#> 5 beansize2 parameter.power.mc 0.904#> 6 I(temp^2) parameter.power.mc 0.636#> 7 type1:beansize1 parameter.power.mc 0.909#> 8 type1:beansize2 parameter.power.mc 0.911
News
skpr v0.57.0 (Release date: 2018-09-21):
Major changes:
• Changed "RunMatrix" argument to "design" in evaluation functions.
Minor Changes/Bug fixes
• Fixed bug with model ordering when augmenting an existing design
• Reduce replication in code and improve documentation
skpr v0.56.1 (Release date: 2018-09-21):
Major changes:
• Major code clean-up to standardize style and spacing within code
• magrittr pipe %>% now imported by default
Minor changes/Bug fixes:
• Split-plot design generation bugfixes for interactions between whole and subplots
skpr v0.54.3 (Release date: 2018-08-08):
Major changes:
• Added Monte Carlo effect power through the car package's Anova function
Minor changes/Bug fixes:
• Change correlation plot palette to viridis
• Bugfix for disallowed combinations between nesting levels in split-plot designs
• Fixed missing varianceratios documentation in eval_design_mc
• Changed file paths to use file.path()
skpr v0.53.4 (Release date: 2018-07-11):
Bug fix:
• Fixed uninitialized memory issue in C++ code
skpr v0.53.3 (Release date: 2018-07-11):
Major changes:
• gen_design() Added design augmentation
• gen_design() Removed all limitations to split-plot design generation--all optimality criteria are now available
• gen_design() Added randomized argument if the user does not want the design randomized
• Added skprGUIserver(), the asynchronous version of skprGUI(). Implemented with a custom version of async based on the future package to support progress bars
Minor changes/Bug fixes:
• Moved progressBarUpdater argument into new advancedoptions argument for eval_design_mc()
• Added partial separation warning to console for non-GUI binomial monte carlo evaluations in eval_design_mc()
• Changed gen_design() so that user doesn't have to fill out splitplotsizes argument unless they want to manually specify the block sizes--skpr will try to create the most balanced design for the given number of trials
• Standardized skprGUI, skprGUIbrowser, and skprGUIserver
• Fixed bug with non-disappearing "block" pane in skprGUI when no htc factors.
• Fixed . treatment in GUI to work properly in all cases with split-plot designs.
• gen_design() Fixed bugs with I, A, and G-optimal split-plot design generation.
• Added seed to test_that script.
• gen_design() Fixed bug(s) in computing I and A-optimality for the design attributes.
• gen_design() Fixed bug related to disallowed combinations with split-plot designs.
• gen_design() Improved G-optimality search speed with rank-2 update formula.
• plot_correlations() Fixed missing variance-covariance matrix
• plot_correlations() Added drop=FALSE arguments to prevent conversion of dataframes to vectors
• plot_correlations() Updated to accept arguments to the par function in base R
Deprecations:
• eval_design_mc() Removed deprecated "binomialprobs" argument.
skpr v0.49.1 (Release date: 2018-05-30):
Bug fixes: Fixed memory access issue for the CRAN.
skpr v0.49.0 (Release date: 2018-05-16):
Major changes:
• Switched underlying linear algebra library from Armadillo to Eigen. Significant speed-up for all design searches. Added rank-2 update formula to speed-up Alias-optimal, I-optimal, and A-optimal designs.
• gen_design() now rearranges the levels for categorical factors from fewest to most number of observations for unbalanced designs (default ordering in R if equal).
• gen_design() argument "advancedoptions" now has an option to turn off alias-based tie breaking.
• Re-implemented . handling in formulas to align more with lm().
Minor changes/Bug fixes:
• Fixed Alias-tie breaking non-linearity.
• Updated GUI code pane to show actual model being used (instead of dots).
• Improved error handling of code pane.
• Fixed conservative anticipated coefficient bug that occurred when 3+ parameter powers were equal.
• Fixed bug with generation of anticipated coefficients for interactions with quadratic effects.
Deprecations:
• quad(.) function support removed.
skpr v0.47.4 (Release date: 2018-03-16):
• Implemented several optimizations that vastly improve speed and performance for D-optimal design generation.
• Added support for bookmarking in the GUI.
• Added advanced options argument to control tolerances for the design search and optimal tie-resolution, and an option to specify which power of aliasing should be used in resolving ties.
• Removed references to deprecated delta argument.
• Fixed conservative anticipated coefficients bug.
• Fixed bug with comparison of floats
• Fixed issue with non-reproducibility of generating parallel designs by using the doRNG package.
• Updated documentation with changes.
skpr v0.45.2 (Release date: 2018-01-22):
Major changes:
• Changed optimal search algorithm to return design with best aliasing structure (as determined by the alias matrix) when there is a tie between optimal designs. Increases likelyhood of producing an orthogonal design.
• Added support for mixture designs in evalulation functions. Generation is already supported in gen_design by removing the intercept from the model.
• Added support for backticked variables in evaluation functions and plotting functions. Made non-support for backticked variables explicit in gen_design.
• Added user-defined "custom" optimality option, where the user can define their own optimality criterion as an R function of the model matrix. This will be slower than the built-in criteria, but provides a great deal of flexibility--especially because the optimality functions can be written without any C. Function needs to be named "customOpt" and be in the global environment.
• Updated split-plot design generation to support Alias-optimal, T-optimal, and user-defined custom optimal designs. For custom designs, user needs to create a function named "customBlockedOpt" that is a function of the model matrix, as well as the variance-covariance matrix vInv. Should look like the following: customBlockedOpt = function(customdesign, vInv) {calculate optimality criterion}
• Implemented support for plot_correlations and plot_fds in output of eval_design_mc. Added required attributes to output.
Minor changes:
• Changed minDopt to 0.8 to return better screening designs.
• Changed default number of repeats to 20.
• Changed plot_correlations to use two-color scheme by default.
• Changed G-optimal design search to start from a D-optimal design.
• Added color to GUI design. Added ability to change color palettes.
• Added option to remove design randomization in GUI.
• Major UI updates for the GUI, changing the style away from default Shiny app.
Bug fixes:
• Changed singularity testing to use more robust function.
• Fixed E-optimal split-plot design generation.
• Fixed bug in split-plot non-singular design generation.
• Fixed is.numeric bug with plot_correlations in Shiny.
• Fixed bug with missing contrast list with hard-to-change factors. Fixed bug where factors were converted to integers with split-plot designs.
• Fixed bug in eval_design_survival_mc with contrast generation after reducing the run matrix.
• Fixed subsetting bug in automatic blocking column detection, turning on drop=FALSE. Fixed bug when external blocking columns were not ordered correctly. Changed name of internal block columns (Block# to skprBlock#) to make sure they don't interfere with user-generated columns.
• Fixed uninformative error message when optimality criterion not recognized when generating designs. Standardized input for optimality criterion to always use upper case.
• Updated/fixed/added unit tests.
skpr v0.40.1 (Release date: 2017-10-20):
• Fixed significant bug with parallel design generation that broke automatic detection of the number of cores available.
• Changed non-split-plot D-optimal design search to use modified Fedorov's algorithm with delta function (changed from modified Fedorov's, without delta function). Decreases search time for complex designs or designs with many trials.
• Turned off OPENMP warning for all c++ files.
skpr v0.39.0 (Release date: 2017-10-15):
• All the evaluation functions will now respect pre-set contrasts for categorical factors. If a contrast isn't set for a factor, it will use the contrast set in the contrasts argument.
• Added the ability for the user to specify the number of cores manually by setting options(cores=). Otherwise, gen_design will use all cores available if parallel=TRUE.
• Added the ability for the user to interrupt the design search in gen_design for both single core and multicore searches. When user interrupts parallel search, gen_design will properly stop and close all connections.
• Added calculation of effect power for interaction terms and higher order terms.
• Changed eval_design so that the output always reports parameter power even if all factors are continuous.
• Fixed bug in eval_design_custom_mc which resulted in imported designs overwriting the generated list of contrasts with NULL.
• Changed delta argument to more descriptive "effectsize". Updated documentation and codebase to reflect this change. Effect size now accepts a length-two vector to either explicitly specify the difference in the high and low levels, or to specify GLM-family specific effect sizes. Updated GUI to reflect effectsize change.
• Changed Monte Carlo simulated estimates plot for exponential GLMs to show mean parameter rather than rate.
• Fixed bug when tibble was converted to data frame and didn't convert the character columns to factors. Turned off OPENMP warning in compilation
• Added support for ~.*. operator, allows user to specify full interaction model (no quadratic terms).
• Updated documentation.
skpr v0.35.1 (Release date: 2017-08-17):
Initial release.