Test of No Group Discrimination Using the WMW Statistic

Implements a wmwAUC test of H0: AUC = 1/2 for continuous, discrete, or mixed random variables, based on the Wilcoxon-Mann-Whitney (WMW) statistic. The classic WMW test is calibrated under H0: {(F, G): F = G} which does not match the set {(F, G): AUC = 1/2}, implied by the test statistic, and consequently leads to erroneous inferences. wmwAUC is calibrated under the correct null and implements two finite-sample corrected p-value methods: an Exact Unbiased (EU) method and a Bias-Corrected (BC) method, both valid for any tie pattern. Methods are described in M. Grendar (2025) "Wilcoxon-Mann-Whitney Test of No Group Discrimination" .

wmwAUC Test of No Group Discrimination

wmwAUC

The classical Wilcoxon-Mann-Whitney (WMW) test is calibrated under the null hypothesis $\mathrm{H_0\colon F = G}$. However, the WMW test statistic, when standardized, is the empirical AUC (eAUC), and hence the test in fact assesses the set ${\mathrm{(F,G)\colon AUC = 0.5}}$ – which does not match $\mathrm{H_0\colon F = G}$. This incoherence causes that WMW test provides erroneous inferences.

Through Monte Carlo analysis of zero-mean heteroskedastic Gaussians and corresponding asymptotic theory, we demonstrate this mismatch directly: the eAUC concentrates on 0.5 even when $\mathrm{F \neq G}$, showing that the classical test’s calibration cannot be tracking distributional equality. Moreover, the traditional alternative hypothesis of stochastic dominance is unnecessarily restrictive; the WMW statistic is consistent against the broader alternative $\mathrm{H_1\colon AUC \neq 0.5}$, as established by Van Dantzig (1951).

wmwAUC implements a properly calibrated test of $\mathrm{H_0\colon AUC = 0.5}$ based on the WMW statistic; for details, see (Grendár 2025). The package provides both asymptotic inference and two finite-sample bias-correction methods:

Exact Unbiased (EU) Method: Universal approach handling data with arbitrary tie patterns through the mid-rank kernel and exact finite-sample unbiased variance estimation from Hoeffding decomposition theory. Reduces correctly to the continuous case when no ties are present.
Bias-Corrected (BC) Method: Alternative using individual
placement-component bias correction with $O(n^{-1})$ finite-sample corrections and Welch-Satterthwaite degrees of freedom. Derived for continuous data (Sect. 4 of the paper); the implementation extends this to ties via the same mid-rank kernel as EU, with conservative behavior confirmed by simulation to persist under ties.

The EU method serves as the default implementation, providing:

Universal applicability (handles any data type - continuous, discrete, or mixed)
Exact finite-sample unbiasedness (not asymptotic approximation)
Theoretically principled tie handling through mid-rank kernel
Retained real power at small sample sizes where BC’s power can be close to zero

BC remains available for users who specifically want its more conservative behavior.

Key functions include:

wmwAUC_test(): Main testing function, defaulting to the EU method, with an option to select BC instead
wmwAUC_pvalue_BC(): WMW AUC p-values via the BC method
wmwAUC_pvalue_EU(): WMW AUC p-values via the EU method, valid for any tie pattern
wmwAUC_pseudomedian_ci(): Confidence interval for the AUC-equalizing shift (pseudomedian), obtained by inverting the test

(The old names wmw_test(), wmw_pvalue(), wmw_pvalue_ties(), and pseudomedian_ci() are not retained as functional aliases. Calling any of them raises an informative error naming the current function to use instead.)

Installation

You can install the development version of wmwAUC using

devtools::install_github('grendar/wmwAUC')

Simulation 1

Consider the setting of two zero-mean different-scale gaussians. Then the traditional $\mathrm{H_0\colon F = G}$ of the classical WMW test is false and $\mathrm{H_0\colon F \neq G}$ holds.

The Monte Carlo simulation demonstrates that the normalized test statistic $U/(n_1n_2)$ which is just eAUC, concentrates asymptotically on 0.5 - the value expected under a true null hypothesis.

If the classical WMW test tested distributional equality, the test statistic should not concentrate on its null value when distributions clearly differ.

Also note that under $\mathrm{H_0\colon F \neq G}$, p-values should concentrate near zero, yet the observed distribution is nearly uniform with a slightly elevated first bins, consistent with testing a true null hypothesis ($\mathrm{H_0\colon AUC = 0.5}$) using miscalibrated variance estimation.

#############################################################################
#
# Simulation 1: H0: F=G is erroneously too broad
#
#############################################################################
#
# This simulation takes several minutes to complete
# N = 10000
# n = 1000
# set.seed(123L)
# pval_wt = pval_wmwAUC = eauc = numeric(N)
# for (i in 1:N) {
#
#   x = rnorm(n, sd = 0.1)
#   y = rnorm(n, sd = 3)
#   # wilcox.test() of H0: F = G
#   wt = wilcox.test(x, y)
#   pval_wt[i] = wt$p.value
#   # wmwAUC_test() of H0: AUC = 0.5
#   pval_wmwAUC[i] = wmwAUC_pvalue_BC(x, y)
#   # eAUC
#   eauc[i] = wt$statistic/(n*n)
#   #
# }
data(simulation1)  # List eauc, pval_wt, pval_wmwAUC
#

Empirical AUC centered at 0.5 despite $\mathrm{F \neq G}$.

Traditional p-values under $\mathrm{H_1}$ should concentrate near 0.

Correct p-values for testing $\mathrm{H_0\colon AUC = 0.5}$.

Simulation 2

The two zero-mean different-scale gaussians setting does not satisfy the traditional $\mathrm{H_1}$ of the stochastic dominance. But, as proved by Van Dantzig in 1951, the WMW statistic is consistent for the broader $\mathrm{H_1\colon AUC \neq 0.5}$.

#############################################################################
#
# Simulation 2: H1: F stoch. dominates G is too narrow
#               WMW is consistent for broader H1: AUC != 0.5   
#
#############################################################################
#
#
# This simulation takes several minutes to complete
# N = 10000
# n = 1000
# set.seed(123L)
# pval_wt = pval_wmwAUC = eauc = numeric(N)
# for (i in 1:N) {
# #
#   # gaussians with different location and scale
#   # does not satisfy stochastic dominance
#   x = rnorm(n, 0, sd = 0.1)
#   y = rnorm(n, 0.5, sd = 3)
#   # wilcox.test H0: F = G vs H1: (F stochastically dominates G) OR (G stochastically dominates F)
#   wt = wilcox.test(x, y)
#   pval_wt[i] = wt$p.value
#   # wmwAUC_test H0: AUC = 0.5 vs H1: AUC neq 0.5
#   pval_wmwAUC[i] = wmwAUC_pvalue_BC(x, y)
#   # eAUC
#   eauc[i] = wt$statistic/(n*n)
# #
# }
data(simulation2)  # List of eauc, pval_wt, pval_wmwAUC
# WMW detects broader alternatives than traditional stochastic dominance

Example 1

Real data analyzed by wmwAUC test of no group discrimination.

data(gemR::MS)
da <- MS

# preparing data frame
class(da$proteins) <- setdiff(class(da$proteins), "AsIs")
df <- as.data.frame(da$proteins)
df$MS <- da$MS