When to Use the Mann-Whitney U Test Instead of a t-Test: Four Inputs That Decide

Shapiro–Wilk returns p = .03 on 12 mice per group, and the reviewer’s note says the data are not normal, please use a non-parametric test. The reflex is to switch from the t-test to Mann–Whitney instead and move on. Sometimes that reflex is right, but the normality test is the weakest of the four things that should decide when to use one test over the other.

Deciding when to use the Mann–Whitney U test instead of a t-test turns on sample size, distribution shape, what kind of data you collected, and what the switch itself costs. Before any of those, it is worth being precise about what the test compares, because the most repeated description of it is wrong.

What Mann–Whitney Actually Compares

Mann–Whitney does not compare medians. It converts every value to a rank and asks whether values drawn from one group tend to exceed values drawn from the other. R’s own documentation for wilcox.test() is unusually blunt about it: the null hypothesis is that the two distributions “differ by a location shift of mu,” and the accompanying estimator “does not estimate the difference in medians (a common misconception)”.

The practical consequence: two groups can have identical medians and still produce p < .05, because the ranks above and below those medians are distributed differently. Mann–Whitney becomes a median comparison only if you add an assumption — that the two distributions have the same shape and differ only by a shift. That assumption is doing more work than most Methods sections admit.

One naming trap while we are here. “Wilcoxon rank-sum” and “Mann–Whitney U” are the same test for two independent groups. “Wilcoxon signed-rank” is the paired test, and it carries a different assumption again: symmetry of the distribution of within-pair differences. Mixing up the two Wilcoxons is a real source of Methods-section errors.

The Four Inputs That Decide Whether to Use Mann–Whitney Instead of a t-Test

1. Sample size, because the assumption is about the mean, not the data

The t-test does not require your data to be normal. It requires the sampling distribution of the mean difference to be approximately normal, and the Central Limit Theorem delivers that for non-normal data once samples are moderate — roughly n ≥ 30 per group as a working threshold. At n = 30 per group, skew has to be severe before the t-test misbehaves. At n = 12 per group, as in the scenario above, you are in the range where the distinction genuinely matters.

This inverts the usual reading of a normality test. Shapiro–Wilk has little power to detect non-normality at small n, which is exactly where non-normality would hurt you; at large n it flags trivial departures that change nothing. The test is least informative precisely where you most want an answer. Our guide to interpreting normality tests before choosing a test covers the sample-size dependence in more detail.

2. Whether the two distributions have the same shape

If one group is roughly symmetric and the other has a long right tail, a significant Mann–Whitney result tells you the groups differ, but not that their centres differ. Shape differences alone can drive the statistic. Plot both distributions before you decide. When shapes differ badly, a t-test on a transformed scale often answers a cleaner question than a rank test on the raw scale.

3. Ordinal data versus non-normal continuous data

These are two different justifications that happen to point at the same test, and conflating them causes trouble. Ordinal outcomes — Likert items, severity grades, histology scores — are rank data by construction. A rank test is the right model for them at any sample size, and no amount of n rescues a mean computed on a 5-point scale.

Skewed continuous data is a different situation. Concentrations, titres, and expression values are usually continuous and log-normal, and the informative move is often to work on the log scale rather than to discard the magnitudes by ranking. Ranking a concentration series throws away the fact that 480 pg/mL is eight times 60 pg/mL, not merely one rank higher.

4. What the switch costs, and what the two-step procedure costs

Switching is cheaper than most people assume. When the data genuinely are normal, the Wilcoxon rank-sum test retains an asymptotic relative efficiency of 3/π ≈ 0.955 against the t-test — about 4.5% less power, which translates to roughly 5% more participants for equivalent power. When normality is violated, the rank test frequently has more power, sometimes substantially so for strongly skewed data.

The more expensive part is the sequence. Running a normality test and then choosing your primary test based on its outcome is a conditional procedure, and the two-stage combination does not carry the nominal α of either stage taken alone. Deciding on design grounds before you see the data avoids this entirely, and it is also what a pre-registration or statistical analysis plan asks you to do.

Very Small Samples Have a p-Value Floor

With tiny groups, Mann–Whitney cannot reach conventional significance no matter how cleanly the groups separate, because there are only so many ways to arrange the ranks. With 3 per group there are C(6,3) = 20 possible allocations of the six ranks. The single most extreme arrangement gives a one-tailed p of 1/20 = .05, so the smallest attainable two-tailed p is 2/20 = .10.

The Decision, Compressed

Note the last row. Running pairwise Mann–Whitney tests across three or more groups re-ranks the data within each pair, so the comparisons are not consistent with the omnibus test and the family-wise error rate is uncontrolled. Dunn’s test uses the ranks from the original Kruskal–Wallis, which is why it is the correct follow-up — the same logic that makes post-hoc tests rather than repeated t-tests the right move after an ANOVA, covered in our guide to choosing between Tukey, Bonferroni, and FDR corrections.

Note also that the parametric default in the table is Welch’s, not Student’s. Welch’s handles unequal variances and unequal group sizes correctly and costs almost nothing when variances are equal, which is why methodologists have argued it should be the default choice (Delacre, Lakens, & Leys, 2017, International Review of Social Psychology). The reasoning is worked through in our comparison of Welch’s and Student’s t-tests.

Report an Effect Size Either Way

Reviewers increasingly reject p-value-only reporting, and non-parametric tests are where this bites hardest, because most software will not compute an effect size for you. For Mann–Whitney, report the rank-biserial correlation r. Commonly quoted bands, translated from Cohen’s d conventions, are approximately 0.11 (small), 0.28 (medium), and 0.43 (large) — useful as rough calibration rather than as thresholds to defend.

Also report the Hodges–Lehmann estimate with its confidence interval, which is what R returns with conf.int = TRUE. It estimates the median of all pairwise differences between groups, and that is a quantity a reader can interpret on the original measurement scale. Reporting it is the closest you can get to the “difference in medians” that everyone assumes Mann–Whitney provides.

Planning n When You Expect to Run Mann–Whitney

Formal power analysis for rank tests generally requires simulation, because the answer depends on the distribution you assume. The practitioner shortcut leans on the efficiency figure above: compute the sample size for the equivalent t-test, then add roughly 5% to cover the efficiency loss under normality. If you expect substantial skew, the rank test will do better than that estimate, so it is a conservative starting point rather than a precise one.

Our free sample size and power calculator covers the independent-samples t-test case and reports the achieved power alongside n, so you can compute the parametric baseline and apply the adjustment yourself. It also emits an APA-formatted methods snippet, which saves rewriting the justification later. For the underlying two-group mechanics, including the Cohen’s d reference table, see t-test power analysis for two-group comparisons.

The decision worth making early is not which test survives a normality check. It is what you want to claim — that the groups differ somewhere, or that a specific central quantity differs by a specific amount — and then choosing the test that supports that claim on the measurement scale the reader cares about.