Multiverse Analysis of Multinomial Processing Tree Models

Statistical or cognitive modeling usually requires a number of more or less
arbitrary choices creating one specific path through a 'garden of forking paths'.
The multiverse approach (Steegen, Tuerlinckx, Gelman, & Vanpaemel, 2016,
) offers a principled alternative in which results
for all possible combinations of reasonable modeling choices are reported.
MPTmultiverse performs a multiverse analysis for multinomial processing tree
(MPT, Riefer & Batchelder, 1988, ) models combining
maximum-likelihood/frequentist and Bayesian estimation approaches with
different levels of pooling (i.e., data aggregation). For the
frequentist approaches, no pooling (with and without parametric or nonparametric
bootstrap) and complete pooling are implemented using
MPTinR < https://cran.r-project.org/package=MPTinR>.
For the Bayesian approaches, no pooling, complete pooling, and three different
variants of partial pooling are implemented using
TreeBUGS < https://cran.r-project.org/package=TreeBUGS>. The main function is
fit_mpt() who performs the multiverse analysis in one call.

MPTmultiverse is an R package that provides functions for a multiverse analysis of multinomial processing tree (MPT) models. Note that the package is currently work in progress and should be considered alpha. If you experience problems, open an issue.

Install

To install MPTmultiverse, make sure you already installed the devtools package via install.packages("devtools"). Moreover, you also need a to have JAGS installed: Go to http://mcmc-jags.sourceforge.net/ for instructions on how to install JAGS on your machine.

If these prerequisites are met, type devtools::install_github("mpt-network/MPTmultiverse") in your R console to install MPTmultiverse together with all required packages that it depends on. To make sure that you are using the latest versions of all packages, you should also run update.packages(ask = FALSE).

Usage

Create a new folder that contains the following three files
(cf. the subfolder vignettes/):

The MPT model in the .eqn-format

The model should be parameterized including all equality constraints.

To encode fixed parameters (e.g., g=.50), replace the parameter
in the eqn-file by constants.

The data with individual frequencies as a .csv-file

The file analysis.rmd(copied from the vignettes subfolder).

Adjust the input options in analysis.rmd in the section
"MPT model definition & Data". You have to specify the correct file names
and the names of the columns in your data that contain a subject identifier and
between-subjects conditions.

Optionally, set some options (e.g., the number of bootstrap samples) via mpt_options()

Run the analysis script (e.g., by knitting the .rmd file).

Current limitations

For the Bayesian models with "no-pooling" and "complete-pooling", no additional
MCMC samples are drawn to achieve the desired level of convergence (e.g., Rhat < 1.05).
This might be addressed in future versions of TreeBUGS.
As a remedy, the number of MCMC iterations can be increased a priori (via mpt_options()).