
Methods answering "is there a difference?"
Discrimination (difference) tests answer a binary question: is a difference between samples perceived? All of them are forced-choice procedures: the assessor must choose an answer, and the statistics rely on the guessing probability p₀.
| Method | Assessor's task | p₀ | Standard | When to use |
|---|---|---|---|---|
| Triangle test | From 3 samples (2 identical), pick the odd one | 1/3 | ISO 4120:2021 · GOST R 53159-2008 | Universal choice: ingredient substitution, batch control. Caution with a strong aftertaste |
| Tetrad | From 4 samples (2+2), form two pairs | 1/3 | ASTM E3009 | More powerful than the triangle at the same n; less receptor fatigue |
| Duo-trio | Which of 2 samples matches the reference | 1/2 | ISO 10399:2026 · GOST R ISO 10399-2012 | Strong aftertaste; a stable reference exists |
| Paired comparison (2-AFC) | Which of 2 samples is more intense in a given attribute | 1/2 | ISO 5495:2005 · GOST R 53160-2008 | Directional difference in one attribute ("which is sweeter?") |
| 3-AFC | From 3 samples, pick the signal one (attribute known in advance) | 1/3 | ISO 13301:2018 | Thresholds, directional differences; more powerful than the triangle |
| Two-out-of-five | From 5 samples (3+2), identify the 2 identical ones | 1/10 | ASTM E2162 | Very weak differences; high fatigue — short series |
| "A"–"not A" | Identify a sample as "A" or "not A" | — | ISO 8588:2017 | Samples of different appearance; recognition tests; analysis via a 2×2 table and d′ |
| Difference from control | Rate the magnitude of deviation from the reference on a scale | — | QC practice | Routine control: "how far has the batch drifted from the reference" |
Why p₀ matters: the lower the guessing probability, the more powerful the test at the same number of assessors. Two-out-of-five (p₀ = 1/10) needs half as many responses as duo-trio (p₀ = 1/2) — but it is physiologically harder: five tastings fatigue the receptors. The choice is always a compromise between statistical power and sensory load.
Design parameters: α, β and p_d
Every discrimination test is fixed by four parameters — without them the result is uninterpretable:
- α-risk (alpha) — the probability of falsely concluding that a difference exists when it does not (type I error). Usually α = 0.05.
- β-risk (beta) — the probability of missing an existing difference (type II error). Usually β = 0.10–0.20.
- p_d — the maximum allowable proportion of assessors able to discriminate the samples (proportion of discriminators): < 25 % — small values, 25–35 % — medium, > 35 % — large.
- Mode: "difference" proves that the samples are distinguishable; "similarity" — that the differences are so small that the products are interchangeable. These are different tests with different tables: "difference not proven" ≠ "similarity proven".
The Thurstonian model: why the tetrad is more powerful than the triangle test
The binomial model answers the question "is the difference proven at a proportion of discriminators p_d", but does not connect the methods to each other: "30 % discriminators" in a triangle test and in a tetrad are sensory differences of different magnitude. The Thurstonian model describes perception as a normally distributed variable; the difference between products is measured by the sensitivity index d′ (d-prime) — the distance between the distributions in units of their standard deviation. The probability of a correct response p_c in each method is a function of d′ (the method's psychometric function), and it differs between methods.
The table below is a numerical simulation of the Thurstonian model (2·10⁶ trials per point; exact formulas for 2-AFC and duo-trio). The key point is visible: at the same sensory difference d′, the tetrad yields noticeably more correct responses than the triangle test, at the same guessing level of 1/3.
| d′ | 2-AFC | 3-AFC | Triangle | Duo-trio | Tetrad |
|---|---|---|---|---|---|
| 0.5 | 0.638 | 0.483 | 0.356 | 0.522 | 0.378 |
| 1.0 | 0.761 | 0.633 | 0.418 | 0.582 | 0.494 |
| 1.5 | 0.856 | 0.765 | 0.507 | 0.663 | 0.641 |
| 2.0 | 0.921 | 0.866 | 0.604 | 0.747 | 0.777 |
How to read this in practice — the minimum panel for 80 % power at α = 0.05 and a difference of d′ = 1.0 (exact binomial calculation):
| Method | p_c at d′ = 1 | n for 80 % power | Critical value |
|---|---|---|---|
| 3-AFC | 0.634 | 9 | 5 |
| 2-AFC | 0.760 | 13 | 9 |
| Tetrad | 0.494 | 48 | 21 |
| Triangle test | 0.418 | 180 | 70 |
| Duo-trio | 0.582 | 207 | 115 |
The practical consequences: AFC methods are the most efficient, but they require the nature of the difference to be known in advance ("which sample is sweeter?"). Among the methods "without a specified attribute", the tetrad at d′ = 1 needs 3.7 times fewer assessors than the triangle test (48 vs 180) — which is why ASTM E3009 and the industry are gradually moving to it. Duo-trio, despite its high p_c, loses because of the 1/2 guessing level. The calculation is reproducible:
# p_c of the methods at a given d' (Thurstonian model, Monte Carlo) # tetrad: 2 "strong" and 2 "weak" samples; the decision is to pair them by similarity import numpy as np rng = np.random.default_rng(42); N, d = 2_000_000, 1.0 s1, s2 = rng.normal(d, 1, N), rng.normal(d, 1, N) n1, n2 = rng.normal(0, 1, N), rng.normal(0, 1, N) ok = (np.abs(s1-s2) + np.abs(n1-n2) < np.minimum(np.abs(s1-n1)+np.abs(s2-n2), np.abs(s1-n2)+np.abs(s2-n1))) pc = ok.mean() # → 0.494 # minimum panel for 80 % power (exact binomial calculation) from scipy.stats import binom p0 = 1/3 n = 4 while True: crit = int(binom.ppf(0.95, n, p0)) + 1 while binom.sf(crit-1, n, p0) <= 0.05: crit -= 1 if binom.sf(crit-1, n, pc) >= 0.80: break n += 1 print(n, crit) # → 48 21
Statistical calculator for discrimination tests
A reproducible example from the study guide: triangle test, n = 24, x = 13 → critical value 13 (Table A.1 of ISO 4120:2021), p-value = P(X ≥ 13 | p₀ = 1/3) ≈ 0.0284 < 0.05 — the difference is proven. Press "Calculate" — the calculator is already set to this example.
How many assessors are needed
Benchmarks for the triangle test (α = 0.05, β = 0.10): p_d = 50 % → ~20–25 assessors; p_d = 30 % → 53; p_d = 20 % → 113. The weaker the difference that must not be missed, the faster the panel grows.
Method chooser
Method features — details
Triangle test (ISO 4120:2021)
The workhorse of discrimination analysis. 6 balanced triads (AAB, ABA, BAA, BBA, BAB, ABB), three-digit codes, mandatory forced choice. The limitation is products with a strong aftertaste: three tastings in a row are fatiguing. The 2021 edition added guidance on the Thurstonian model (d′) alongside the classical guessing model.
Tetrad (ASTM E3009)
Four samples (two + two), the task is to group them into pairs. The same p₀ = 1/3, but higher statistical power at equal n and less sensory load per tasting (no third comparison to remember). Suitable for "fatiguing" products and for small panels.
Duo-trio (ISO 10399:2026)
A reference is served first, then two samples — which one matches the reference. p₀ = 1/2, less powerful than the triangle, but easier for the assessor with persistent tastes. The reference can be constant (a stable standard) or balanced (half the panel gets reference A, half gets B). The current edition is the 2026 one, replacing ISO 10399:2017.
"A"–"not A" (ISO 8588:2017)
The assessor gets familiar with the reference "A", then identifies a stream of samples as "A"/"not A". The data are summarized in a 2×2 contingency table (stimulus × response) and analysed with χ²; for threshold tasks — signal detection theory: sensitivity d′ and response criterion. The method is convenient when samples cannot be served simultaneously (different appearance, tableware, temperature).
Replicates and the beta-binomial model
If assessors repeat the test (replicates), one person's answers are not independent: the ordinary binomial model underestimates the variance and overstates significance. Correct approaches are the beta-binomial model (Ennis & Bi) or adjusted tests (Brockhoff & Schlich). The rule: if you plan replicates, plan a model that accounts for panel heterogeneity.
Further reading
- Ennis J. M., Jesionka V. The Power of Sensory Discrimination Methods Revisited. Journal of Sensory Studies, 2011 — the power calculations for the tetrad vs the triangle test that started the industry's mass move to the tetrad. article
- Brockhoff P. B., Christensen R. H. B. Thurstonian Models for Sensory Discrimination Tests as Generalized Linear Models. Food Quality and Preference, 2010 — the Thurstonian model as a GLM: estimating d′ with standard errors. article
- The sensR package for R (Christensen, Brockhoff) — d′, power, panel size, beta-binomial models for all discrimination methods; free. tool
- ASTM E3009 — Standard Test Method for Sensory Analysis — Tetrad Test — the current edition of the tetrad standard. standard
Test yourself
Four questions on discrimination tests
1. Triangle test, n = 30, x = 13 correct obtained. The critical value at α = 0.05 is 15. Conclusion?
13 < 15: no significant difference. But the conclusion "similar" requires a separate similarity test with specified β and p_d. Check in the calculator above.
2. Why is two-out-of-five more powerful than duo-trio at the same n?
Power grows as p₀ falls. The price is fatigue: five tastings per response. That is why the method is reserved for weak differences and short series.
3. Task: prove that changing the cocoa supplier did not alter the taste (it is acceptable for no more than 30 % of assessors to feel the difference). This is…
The wording "no more than p_d discriminators" is the signature of a similarity test: the critical value is computed from p_d and β, not from α.
4. At p_d = 0.30, α = 0.05, β = 0.10, the triangle test needs…
Check with the panel-size calculator: n = 53 is the value from Table B.1 of ISO 4120:2021. "By-eye" estimates understate the required size by a factor of 2–3.
Try it in practice
The interactive Sensostat demo lets you run a triangle test end-to-end: constructor, serving plan, questionnaire, conclusion.