Tools for Type S (Sign) and Type M (Magnitude) Errors
Provides tools for working with Type S (Sign) and
Type M (Magnitude) errors, as proposed in Gelman and Tuerlinckx (2000)
Gelman & Carlin (2014) .
In addition to simply calculating the probability of
Type S/M error, the package includes functions for calculating these errors
across a variety of effect sizes for comparison, and recommended sample size
given "tolerances" for Type S/M errors. To improve the speed of these
calculations, closed forms solutions for the probability of a Type S/M error
from Lu, Qiu, and Deng (2018)
are implemented. As of 1.0.0, this includes
support only for simple research designs. See the
package vignette for a fuller exposition on how Type S/M errors arise in
research, and how to analyze them using the type of design analysis proposed
in the above papers.
Tools for calculating and working with Gelman et al's Type S (sign), and M (magnitude or exaggeration ratio) errors for analyzing hypothesis tests in R.
Functions for calculating Power and Type S/M errors across variety of effect sizes, building on code provided in Gelman and Carlin (2014, see below).
Graphics function for visualizing these errors with or without ggplot
Implementation of Lu et al's (2018, see below) closed form solution for Type M error, providing a speed-up.
This will be on CRAN soon, but in the meantime, you can install with devtools using:
You can find an online version of retrodesign's vignette on my website, which provides an introduction to both
retrodesign and Type S/M errors.
More Reading on Type S/M Errors
- Gelman and Tuerlinckx's Type S error rates for classical and Bayesian single and multiple comparisons
procedures (2000): A comparison of the properties of Type S errors of frequentist and Bayesian confidence statements. Useful for how this all plays out in a Bayesian context. Bayesian confidence statements have the desirable property of being more conservative than frequentist ones.
- Gelman and Carlin's Assessing Type S and Type M Errors
(2014): Gelman and Carlin compare their suggested design analysis, as we've written about above,
to more traditional design analysis, through several examples, and discuss the
desirable properties it has in more depth than I do here. It is also the source of the original
retrodesign() function, which I re-use in the package with permission.
- Lu et al's A note on Type S/M errors in hypothesis testing
(2018): Lu and coauthors go further into the mathematical properties of Type S/M errors, and prove the closed form solutions implemented in
- McShane et al's Abandon Statistical Significance (2017): If you
want a starting point on the challenges with NHST that have led many statisticians
to argue for abandoning NHST all together, and starting points for alternative
ways of doing science.