Aligned Rank Transform

The aligned rank transform for nonparametric factorial ANOVAs as described by Wobbrock, Findlater, Gergle, and Higgins (2011) . Also supports aligned rank transform contrasts as described by Elkin, Kay, Higgins, and Wobbrock (2021) .

ARTool: R Package for the Aligned Rank Transform for Nonparametric Factorial ANOVAs

BuildStatus CRAN_Status_Badge GPL >=2 DOI DOI

Matthew Kay, University of Washington [email protected]
Jacob O. Wobbrock, University of Washington [email protected]

ARTool is an R package implementing the Aligned Rank Transform for conducting nonparametric analyses of variance on factorial models. This implementation is based on the ART procedure as used in the original implementation of ARTool by Wobbrock et al.

The package automates the Aligning-and-Ranking process using the art function. It also automates the process of running a series of ANOVAs on the transformed data and extracting the results of interest. It supports traditional ANOVA models (fit using lm), repeated measures ANOVAs (fit using aov), and mixed effects models (fit using lmer); the model used is determined by the formula passed to art.

Note: The documentation of this package assumes some level of familiarity with when and why you may want to use the aligned rank transform; the ARTool page provides a more in-depth (and highly approachable) introduction to the aligned rank transform and the motivation for its use.


You can install the latest released version from CRAN with this R command:


Or, you can install the latest development version from GitHub with these R commands:



The general approach to using ART is to transform your data using art , verify the ART procedure is appropriate to the dataset using summary , and then run an anova on the transformed data using anova .

First, let us load some example data:


Higgins1990Table5 is a data frame from an experiment in which the effects of Moisture and Fertilizer on DryMatter in peat pots was tested. Four pots were placed on each Tray , with Moisture varied between Tray s and Fertilizer varied within Tray s. We can see the basic structure of the data:

##  $ Tray      : Factor w/ 12 levels "t1","t2","t3",..: 1 1 1 1 2 2 2 2 3 3 ...
##  $ Moisture  : Factor w/ 4 levels "m1","m2","m3",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ Fertilizer: Factor w/ 4 levels "f1","f2","f3",..: 1 2 3 4 1 2 3 4 1 2 ...
##  $ DryMatter : num  3.3 4.3 4.5 5.8 4 4.1 6.5 7.3 1.9 3.8 ...
head(Higgins1990Table5, n=8)
##   Tray Moisture Fertilizer DryMatter
## 1   t1       m1         f1       3.3
## 2   t1       m1         f2       4.3
## 3   t1       m1         f3       4.5
## 4   t1       m1         f4       5.8
## 5   t2       m1         f1       4.0
## 6   t2       m1         f2       4.1
## 7   t2       m1         f3       6.5
## 8   t2       m1         f4       7.3

Step 1: Transform the data

To analyze this data using the aligned rank transform, we first transform the data using art . We specify the response variable (DryMatter ), the fixed effects and all of their interactions (Moisture*Fertilizer, or equivalently Moisture + Fertilizer + Moisture:Fertilizer), and any grouping terms if present (here, (1|Tray) ).

While (1|Tray) has no effect on the results of the aligned rank transformation, it will be used by anova to determine the type of model to run: when grouping terms are present, mixed effects models are run using lmer. If you wish to use a repeated measures ANOVA instead of a mixed effects model, you can use an Error term instead (see below for an example of this). If you do not having repeated measures, do not include any grouping terms or error terms.

m <- art(DryMatter ~ Moisture*Fertilizer + (1|Tray), data=Higgins1990Table5)

Step 2: Verify appropriateness of ART

To verify that the ART procedure was correctly applied and is appropriate for this dataset, we can look at the output of summary :

## Aligned Rank Transform of Factorial Model
## Call:
## art(formula = DryMatter ~ Moisture * Fertilizer + (1 | Tray), 
##     data = Higgins1990Table5)
## Column sums of aligned responses (should all be ~0):
##            Moisture          Fertilizer Moisture:Fertilizer 
##                   0                   0                   0 
## F values of ANOVAs on aligned responses not of interest (should all be ~0):
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##       0       0       0       0       0       0

We see that the columns sums of aligned responses and the F values of ANOVAs on aligned responses not of interest are all ~0, indicating that the alignment correctly “stripped out” effects not of interest. Thus, we can apply the ANOVA on the transformed data.

Step 3: Run the ANOVA

ARTool automatically selects the model to be used for the ANOVA. Because we have included a grouping term, (1|Tray), ARTool will fit mixed effects models using lmer and run the ANOVAs on them:

## Analysis of Variance of Aligned Rank Transformed Data
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df) 
## Model: Mixed Effects (lmer)
## Response: art(DryMatter)
##                             F Df Df.res     Pr(>F)    
## 1 Moisture             23.832  3      8 0.00024200 ***
## 2 Fertilizer          122.402  3     24 1.1124e-14 ***
## 3 Moisture:Fertilizer   5.118  9     24 0.00064664 ***
## ---
## Signif. codes:   0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Alternative model: Repeated Measures ANOVA

This particular study could also be analyzed using a repeated measures ANOVA, yielding the same results (note that repeated measures ANOVAs and mixed effects models will not always yield the same results). To instead run a repeated measures ANOVA, add an Error term to the model as you might for a call to aov:

m <- art(DryMatter ~ Moisture*Fertilizer + Error(Tray), data=Higgins1990Table5)
## Analysis of Variance of Aligned Rank Transformed Data
## Table Type: Repeated Measures Analysis of Variance Table (Type I) 
## Model: Repeated Measures (aov)
## Response: art(DryMatter)
##                       Error Df Df.res F value     Pr(>F)    
## 1 Moisture             Tray  3      8  23.833 0.00024199 ***
## 2 Fertilizer          Withn  3     24 122.402 1.1124e-14 ***
## 3 Moisture:Fertilizer Withn  9     24   5.118 0.00064665 ***
## ---
## Signif. codes:   0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Contrast tests

For an example of how to run contrast tests on an art model, see this vignette:


This vignette is also available here.


Should you encounter any issues with this package, contact Matthew Kay ([email protected]). If you have found a bug, please file it here with minimal code to reproduce the issue.


Kay M and Wobbrock J (2019). ARTool: Aligned Rank Transform for Nonparametric Factorial ANOVAs. R package version 0.10.6, DOI: 10.5281/zenodo.594511.

Wobbrock J, Findlater L, Gergle D and Higgins J (2011). “The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures.” In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ’11), Vancouver, British Columbia (May 7-12, 2011). New York: ACM Press, pp. 143-146. DOI: 10.1145/1978942.1978963.


ARTool 0.10.6

Minor changes:

  • Minor changes to ensure tests pass due to changes in output from lme4::lmer

ARTool 0.10.5

Minor changes:

  • Replace lsmeans with emmeans in code and docs due to lsmeans being deprecated
  • Use for Cohen's d CIs in effect size vignette

ARTool 0.10.4

Minor changes:

  • Reference phia vignette using function call instead of non-canonical URL
  • Cautionary note about standardized effect sizes in effect size vignette
  • Added testInteractions example as alternative in contrasts vignette


  • Dependency fixes for failed test at testthat/test.artlm.R:35 (our use of lsmeans in that test requires some additional packages only declared as "suggests" in lsmeans, so now we "suggest" them as well).

ARTool 0.10.1

New features:

  • New vignette describing effect size estimates
  • Using lsmeans interactions argument instead of phia for interaction contrasts vignette


  • Require R >= 3.2 and lsmeans >= 2.22 to fix some bugs in earlier versions

ARTool 0.10.0

New features:

  • Support for Error() terms in model formulas (resulting models are run using aov())
  • Checks for numeric variables passed into formulas that may cause incorrect results (if the user intended data to be treated as categorical)
  • More detailed ANOVA tables
  • New vignette describing contrast tests, particularly for interactions (vignette("art-contrasts"))


  • Formulas now correctly support arbitrary expressions as terms (rather than just column names).

ARTool 0.9.5

Testing fix for changes in upcoming version of lsmeans: round lsmeans p value tests to 5 decimal places to accommodate changes to Tukey adjustment

ARTool 0.9.4

Minor changes to testing based on updated version of testthat (0.10.0):

  • More closely follow recommended testthat usage
  • Skip tests requiring lsmeans if it is not installed

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.


0.11.1 by Matthew Kay, 14 days ago

Report a bug at

Browse source code at

Authors: Matthew Kay [aut, cre] , Lisa A. Elkin [aut] , James J. Higgins [aut] , Jacob O. Wobbrock [aut]

Documentation:   PDF Manual  

GPL (>= 2) license

Imports lme4, car, plyr, magrittr, dplyr, emmeans

Suggests testthat, knitr, rmarkdown, ggplot2, tidyr, pander, lmerTest, cluster, phia, survival, psych, stringi, DescTools, tibble, covr

See at CRAN