Chi-Square Test of Independence: Assumptions, Expected Counts, and When to Use Fisher's Exact

Your data are categorical — two variables, each with a few levels, and you want to know whether they’re associated. You reach for a chi-square test of independence. The software runs it, prints a p-value, and you move on. Except a footnote at the bottom of the output says “X cells have expected count less than 5” and you don’t know what that means or whether it’s a problem.

This walks through the assumptions practitioners actually need to check, the rule of thumb for small expected counts, and when to switch to Fisher’s exact test. The worked example is in R, but the assumption logic transfers to SPSS, Stata, Prism, and JASP.

What the Chi-Square Test of Independence Tests

The test asks: are two categorical variables associated, or are the observed counts consistent with the variables being independent? The procedure builds a contingency table of observed counts, computes the counts you would expect under independence, and sums a discrepancy statistic across cells:

where O_ij is the observed count in cell (i, j), E_ij is the expected count under independence, R_i is the row total, C_j is the column total, and N is the grand total. Degrees of freedom = (rows − 1) × (columns − 1). Larger χ² values give smaller p-values; the test is one-sided in the sense that you only reject for χ² values larger than expected.

What it does not tell you: the direction or strength of the association. For a 2×2 table you need to look at the cell counts and probably report an odds ratio with a CI. For larger tables, standardized residuals and Cramér’s V quantify which cells drive the result and how strong the association is.

Assumption 1: Independence of Observations

Each observation must contribute to exactly one cell. This is the most easily violated assumption because it’s about data structure, not data values.

• Violated when: the same subjects are measured at two time points and you cross-tabulate the time-1 and time-2 responses (use McNemar’s test instead). Repeated samples from the same patient. Counting cells within an animal and treating them as independent (clustered data — chi-square will be anticonservative).
• Symptom: row totals or column totals don’t reflect distinct subjects. If N in the table is larger than the number of unique individuals, independence is broken.

Assumption 2: Expected Counts — Cochran’s Rule

The chi-square approximation breaks down when expected cell counts are too small. The conventional rule, due to Cochran (1954), has two parts:

1. No expected count below 1.
2. No more than 20% of cells have expected counts below 5.

For a 2×2 table, “no more than 20%” means zero cells (one cell would be 25%). For a 3×3 table you have 9 cells, so up to 1 cell can be below 5. For a 4×5 table you have 20 cells, so up to 4 can be below 5.

Cochran’s rule is conservative — later simulation work (Camilli & Hopkins, 1978; Kroonenberg & Verbeek, 2018) shows the chi-square approximation often holds with expected counts down to 1 in a few cells. But the 20%/5 rule is still what reviewers ask about, so design your reporting around it.

Worked Example: A 2×3 Table

Suppose you randomized 120 patients across three treatment groups (A, B, C) and recorded whether each had a defined clinical response (Yes/No). The observed counts:

Expected count under independence: E_A,Yes = 40 × 63 / 120 = 21, E_A,No = 40 × 57 / 120 = 19. All six expected counts: 21, 19, 21, 19, 21, 19. All ≥ 5, so Cochran’s rule is satisfied.

The chi-square statistic for the table:

tab <- matrix(c(28, 21, 14, 12, 19, 26), nrow = 3,
              dimnames = list(group = c("A","B","C"),
                              response = c("Yes","No")))
chisq.test(tab)

Computing by hand: χ² = (28−21)²/21 + (12−19)²/19 + (21−21)²/21 + (19−19)²/19 + (14−21)²/21 + (26−19)²/19 = 2.33 + 2.58 + 0 + 0 + 2.33 + 2.58 = 9.83. With df = (3−1)×(2−1) = 2, the p-value is .0073. There is evidence of association between group and response.

What’s missing for a complete write-up: post-hoc analysis. Chi-square is an omnibus test — rejection tells you something differs, not which groups. Standardized residuals identify which cells drive the result (|z| > 2 is typically considered notable); pairwise 2×2 sub-tables with FDR adjustment locate the differences. For a deeper treatment of pairwise testing after an omnibus test, see how to choose between Tukey, Bonferroni, and FDR; the same logic applies here.

When to Switch to Fisher’s Exact Test

Fisher’s exact test computes the exact p-value by enumerating tables with the same marginal totals, rather than approximating with the chi-square distribution. Use it when the chi-square approximation can’t be trusted:

• Any expected cell count below 1. Always switch.
• More than 20% of cells with expected count below 5 in tables larger than 2×2. Switch.
• 2×2 table with any expected count below 5. Switch.
• Very small total N (e.g., N < 30). Switch by default.

Fisher’s exact works fine for larger tables in current software (R’s fisher.test() uses a network algorithm), so the historical “Fisher’s only works for 2×2” restriction is no longer the case. For very large tables with one or two sparse cells, the computation can be slow; simulate.p.value = TRUE falls back to Monte Carlo.

Effect Size: Don’t Skip This

Chi-square’s p-value scales with sample size — a tiny association becomes “significant” at N = 10,000. Effect size answers the practical question: how strong is the association?

• 2×2 table: Phi (φ) = √(χ²/N), interpreted on the same scale as a correlation (0.1 small, 0.3 medium, 0.5 large). Equivalent to the absolute Pearson correlation for two binary variables. Also report the odds ratio with a 95% CI — it’s the standard effect-size metric for 2×2 in clinical and epidemiological work.
• Larger tables: Cramér’s V = √(χ² / (N × (k − 1))), where k = min(rows, columns). Same 0.1/0.3/0.5 interpretation. Bounded between 0 and 1.

Cramér’s V for the worked example: √(9.83 / (120 × 1)) = 0.286, a medium-sized association. Reviewers in 2026 will ask for this; APA 7th edition requires effect sizes for chi-square tests of association.

When the Right Test Isn’t Chi-Square

A few cases push you to a different procedure:

• Paired/matched categorical data: McNemar’s test for 2×2 matched-pairs designs, or its generalization (Stuart-Maxwell / Bowker’s test) for larger square tables. Example: same patients before and after a diagnostic intervention.
• Ordered categories: Cochran-Armitage trend test or linear-by-linear association. Chi-square treats “mild / moderate / severe” as nominal categories; trend tests use the ordering and have more power when an actual trend exists.
• Stratified analysis: Cochran-Mantel-Haenszel test for 2×2 tables across strata (e.g., comparing two treatments across multiple sites while controlling for site). Avoids Simpson’s paradox.
• Many predictors: switch to logistic regression. Chi-square handles one binary or categorical outcome with one categorical predictor; logistic regression handles the same outcome with multiple predictors (continuous or categorical) and lets you adjust for covariates.

If you’re unsure which test fits your design, the test-selection decision guide walks through the variable-type and group-structure questions in order.

Pre-Submission Checklist

Before the methods section is final:

• Independence of observations confirmed (no within-subject repeated measures cross-tabbed, no nested data treated as independent)
• Expected counts checked and reported in a supplementary table if any are below 5
• Decision to use chi-square vs. Fisher’s exact justified by the expected-count rule, not by “sample size large/small”
• Effect size (phi, Cramér’s V, or odds ratio with CI) reported alongside the test statistic and p-value
• Test statistic written as χ²(df, N = n) = value (APA convention)
• Post-hoc analysis (standardized residuals or pairwise sub-tables with FDR) included when the omnibus test is significant and the table is larger than 2×2

The combination — chi-square or Fisher’s exact, plus effect size, plus post-hoc when needed — is what reviewers in clinical, behavioral, and observational research expect. Skipping effect size in favor of bare p-values is the single most common D2 finding in this area; surface it in your draft tables before submission.

If you want to confirm a chi-square vs. Fisher’s decision against your own table without re-running counts by hand, GraphHelix surfaces the expected-count check inline with the test recommendation, so the rule-violation case is caught before it reaches the manuscript.