Sensostat Try the demo

Textbook · Module 04

Discrimination (Difference) Methods

Triangle test, tetrad, duo-trio, paired comparison, 2-AFC/3-AFC, two-out-of-five, "A"–"not A": design, statistics, critical-value calculator.

Module 04 of 11~35 min readinteractive + quiz
A triangle-test triad: three coded cups on a tray, water and crackers for palate cleansing
A triangle-test triad: two samples are identical, one is different. Three-digit codes, water and a neutral cracker between samples.

Methods answering "is there a difference?"

Discrimination (difference) tests answer a binary question: is a difference between samples perceived? All of them are forced-choice procedures: the assessor must choose an answer, and the statistics rely on the guessing probability p₀.

MethodAssessor's taskp₀StandardWhen to use
Triangle testFrom 3 samples (2 identical), pick the odd one1/3ISO 4120:2021 · GOST R 53159-2008Universal choice: ingredient substitution, batch control. Caution with a strong aftertaste
TetradFrom 4 samples (2+2), form two pairs1/3ASTM E3009More powerful than the triangle at the same n; less receptor fatigue
Duo-trioWhich of 2 samples matches the reference1/2ISO 10399:2026 · GOST R ISO 10399-2012Strong aftertaste; a stable reference exists
Paired comparison (2-AFC)Which of 2 samples is more intense in a given attribute1/2ISO 5495:2005 · GOST R 53160-2008Directional difference in one attribute ("which is sweeter?")
3-AFCFrom 3 samples, pick the signal one (attribute known in advance)1/3ISO 13301:2018Thresholds, directional differences; more powerful than the triangle
Two-out-of-fiveFrom 5 samples (3+2), identify the 2 identical ones1/10ASTM E2162Very weak differences; high fatigue — short series
"A"–"not A"Identify a sample as "A" or "not A"—ISO 8588:2017Samples of different appearance; recognition tests; analysis via a 2×2 table and d′
Difference from controlRate the magnitude of deviation from the reference on a scale—QC practiceRoutine control: "how far has the batch drifted from the reference"

Why p₀ matters: the lower the guessing probability, the more powerful the test at the same number of assessors. Two-out-of-five (p₀ = 1/10) needs half as many responses as duo-trio (p₀ = 1/2) — but it is physiologically harder: five tastings fatigue the receptors. The choice is always a compromise between statistical power and sensory load.

Design parameters: α, β and p_d

Every discrimination test is fixed by four parameters — without them the result is uninterpretable:

  • α-risk (alpha) — the probability of falsely concluding that a difference exists when it does not (type I error). Usually α = 0.05.
  • β-risk (beta) — the probability of missing an existing difference (type II error). Usually β = 0.10–0.20.
  • p_d — the maximum allowable proportion of assessors able to discriminate the samples (proportion of discriminators): < 25 % — small values, 25–35 % — medium, > 35 % — large.
  • Mode: "difference" proves that the samples are distinguishable; "similarity" — that the differences are so small that the products are interchangeable. These are different tests with different tables: "difference not proven" ≠ "similarity proven".
A trap case: "We ran a triangle test, 24 assessors, 11 correct against a critical value of 12 — so the new sweetener is indistinguishable." No: the difference was merely not proven. A similarity conclusion needs a similarity test with specified β and p_d — and usually a larger panel.

The Thurstonian model: why the tetrad is more powerful than the triangle test

The binomial model answers the question "is the difference proven at a proportion of discriminators p_d", but does not connect the methods to each other: "30 % discriminators" in a triangle test and in a tetrad are sensory differences of different magnitude. The Thurstonian model describes perception as a normally distributed variable; the difference between products is measured by the sensitivity index d′ (d-prime) — the distance between the distributions in units of their standard deviation. The probability of a correct response p_c in each method is a function of d′ (the method's psychometric function), and it differs between methods.

The table below is a numerical simulation of the Thurstonian model (2·10⁶ trials per point; exact formulas for 2-AFC and duo-trio). The key point is visible: at the same sensory difference d′, the tetrad yields noticeably more correct responses than the triangle test, at the same guessing level of 1/3.

d′2-AFC3-AFCTriangleDuo-trioTetrad
0.50.6380.4830.3560.5220.378
1.00.7610.6330.4180.5820.494
1.50.8560.7650.5070.6630.641
2.00.9210.8660.6040.7470.777

How to read this in practice — the minimum panel for 80 % power at α = 0.05 and a difference of d′ = 1.0 (exact binomial calculation):

Methodp_c at d′ = 1n for 80 % powerCritical value
3-AFC0.63495
2-AFC0.760139
Tetrad0.4944821
Triangle test0.41818070
Duo-trio0.582207115

The practical consequences: AFC methods are the most efficient, but they require the nature of the difference to be known in advance ("which sample is sweeter?"). Among the methods "without a specified attribute", the tetrad at d′ = 1 needs 3.7 times fewer assessors than the triangle test (48 vs 180) — which is why ASTM E3009 and the industry are gradually moving to it. Duo-trio, despite its high p_c, loses because of the 1/2 guessing level. The calculation is reproducible:

# p_c of the methods at a given d' (Thurstonian model, Monte Carlo)
# tetrad: 2 "strong" and 2 "weak" samples; the decision is to pair them by similarity
import numpy as np
rng = np.random.default_rng(42); N, d = 2_000_000, 1.0
s1, s2 = rng.normal(d, 1, N), rng.normal(d, 1, N)
n1, n2 = rng.normal(0, 1, N), rng.normal(0, 1, N)
ok = (np.abs(s1-s2) + np.abs(n1-n2) < np.minimum(np.abs(s1-n1)+np.abs(s2-n2),
                                                  np.abs(s1-n2)+np.abs(s2-n1)))
pc = ok.mean()            # → 0.494
# minimum panel for 80 % power (exact binomial calculation)
from scipy.stats import binom
p0 = 1/3
n = 4
while True:
    crit = int(binom.ppf(0.95, n, p0)) + 1
    while binom.sf(crit-1, n, p0) <= 0.05: crit -= 1
    if binom.sf(crit-1, n, pc) >= 0.80: break
    n += 1
print(n, crit)            # → 48 21
Important: d′ is a currency shared by all methods. If a report gives not only "p = 0.02" but also a d′ estimate with a confidence interval (e.g. via the sensR package for R), the results of triangle tests and tetrads become comparable with each other and across laboratories.

Statistical calculator for discrimination tests

Test result → conclusion interactive

Exact binomial calculation (no normal approximations) — the very core of sensostat-stats, verified by reference tests against ISO 4120 and ASTM. Choose a method and a mode, and enter the session result.

"Similarity" — for justifying interchangeability

A reproducible example from the study guide: triangle test, n = 24, x = 13 → critical value 13 (Table A.1 of ISO 4120:2021), p-value = P(X ≥ 13 | p₀ = 1/3) ≈ 0.0284 < 0.05 — the difference is proven. Press "Calculate" — the calculator is already set to this example.

How many assessors are needed

Panel size calculation interactive

Exact power enumeration: the minimum n at which the probability of detecting a difference with a proportion of discriminators p_d is no lower than 1 − β at a given α (the same logic as in Table B.1 of ISO 4120:2021).

Benchmarks for the triangle test (α = 0.05, β = 0.10): p_d = 50 % → ~20–25 assessors; p_d = 30 % → 53; p_d = 20 % → 113. The weaker the difference that must not be missed, the faster the panel grows.

Method chooser

Which test to choose interactive

Answer three questions about your task to get a method recommendation with a rationale.

Method features — details

Triangle test (ISO 4120:2021)

The workhorse of discrimination analysis. 6 balanced triads (AAB, ABA, BAA, BBA, BAB, ABB), three-digit codes, mandatory forced choice. The limitation is products with a strong aftertaste: three tastings in a row are fatiguing. The 2021 edition added guidance on the Thurstonian model (d′) alongside the classical guessing model.

Tetrad (ASTM E3009)

Four samples (two + two), the task is to group them into pairs. The same p₀ = 1/3, but higher statistical power at equal n and less sensory load per tasting (no third comparison to remember). Suitable for "fatiguing" products and for small panels.

Duo-trio (ISO 10399:2026)

A reference is served first, then two samples — which one matches the reference. p₀ = 1/2, less powerful than the triangle, but easier for the assessor with persistent tastes. The reference can be constant (a stable standard) or balanced (half the panel gets reference A, half gets B). The current edition is the 2026 one, replacing ISO 10399:2017.

"A"–"not A" (ISO 8588:2017)

The assessor gets familiar with the reference "A", then identifies a stream of samples as "A"/"not A". The data are summarized in a 2×2 contingency table (stimulus × response) and analysed with χ²; for threshold tasks — signal detection theory: sensitivity d′ and response criterion. The method is convenient when samples cannot be served simultaneously (different appearance, tableware, temperature).

Replicates and the beta-binomial model

If assessors repeat the test (replicates), one person's answers are not independent: the ordinary binomial model underestimates the variance and overstates significance. Correct approaches are the beta-binomial model (Ennis & Bi) or adjusted tests (Brockhoff & Schlich). The rule: if you plan replicates, plan a model that accounts for panel heterogeneity.

Further reading

Test yourself

module 04 quiz

Four questions on discrimination tests

1. Triangle test, n = 30, x = 13 correct obtained. The critical value at α = 0.05 is 15. Conclusion?

13 < 15: no significant difference. But the conclusion "similar" requires a separate similarity test with specified β and p_d. Check in the calculator above.

2. Why is two-out-of-five more powerful than duo-trio at the same n?

Power grows as p₀ falls. The price is fatigue: five tastings per response. That is why the method is reserved for weak differences and short series.

3. Task: prove that changing the cocoa supplier did not alter the taste (it is acceptable for no more than 30 % of assessors to feel the difference). This is…

The wording "no more than p_d discriminators" is the signature of a similarity test: the critical value is computed from p_d and β, not from α.

4. At p_d = 0.30, α = 0.05, β = 0.10, the triangle test needs…

Check with the panel-size calculator: n = 53 is the value from Table B.1 of ISO 4120:2021. "By-eye" estimates understate the required size by a factor of 2–3.

Try it in practice

The interactive Sensostat demo lets you run a triangle test end-to-end: constructor, serving plan, questionnaire, conclusion.

Open the demo →