Statistics is how we tell signal from noise
This module brings the whole statistical toolkit of sensory science together in one place. The Sensostat rule: every numerical conclusion is accompanied by a test, a significance level and an interpretation in the client's language.
The binomial model of discrimination tests
If there is no difference, the assessor guesses with probability p₀ (triangle/tetrad — 1/3, duo-trio/paired — 1/2, two-out-of-five — 1/10). The number of correct answers X ~ Binomial(n, p₀). A difference is proven if X ≥ the critical value (the smallest x for which P(X ≥ x) ≤ α).
"Significant / not significant" is not the end: a confidence interval for p_d
The p-value says the difference is proven, but not how large it is. The proportion of discriminators is estimated as p̂_d = (p̂_c − p₀) / (1 − p₀), and its precision is given by the Clopper–Pearson confidence interval (the "exact" interval for a binomial proportion) rescaled to p_d:
Example: a triangle test, n = 48, x = 22 correct (the critical value is 22, i.e. p ≈ 0.048, a "borderline" difference). Then p̂_d = (22/48 − 1/3) / (2/3) = 0.188, and the 95 % Clopper–Pearson interval:
| Quantity | Estimate | 95 % CI |
|---|---|---|
| p_c (proportion correct) | 0.458 | [0.314 ; 0.608] |
| p_d (proportion of discriminators) | 0.188 | [0.000 ; 0.412] |
# Clopper–Pearson interval and conversion to p_d (triangle test) from scipy.stats import beta n, x, p0 = 48, 22, 1/3 lo = beta.ppf(0.025, x, n - x + 1) # 0.3137 hi = beta.ppf(0.975, x + 1, n - x) # 0.6083 pd_hat = (x/n - p0) / (1 - p0) # 0.188 pd_ci = (max(0, (lo - p0)/(1 - p0)), (hi - p0)/(1 - p0)) # (0.000 ; 0.412)
Replicates: why "48 answers from 24 assessors" ≠ "48 answers"
In protocols with replicates (each assessor repeats the test r times) the observations within one person are dependent: one is consistently attentive, another is not. The binomial model treats all n·r answers as independent and underestimates the variance. The correct model is the beta-binomial one: each assessor has their own probability of a correct answer, beta-distributed across the panel; the parameter γ (overdispersion) measures the excess variability.
The factor (1 + (r−1)γ) is variance inflation. Example (triangle test, 24 assessors × 2 replicates = 48 answers, p₀ = 1/3):
| Model | Var(X) | SD | Comment |
|---|---|---|---|
| Binomial (γ = 0) | 10.67 | 3.27 | all answers independent |
| Beta-binomial, γ = 0.05 | 11.20 | 3.35 | weak panel heterogeneity |
| Beta-binomial, γ = 0.10 | 11.73 | 3.43 | typical practice |
| Beta-binomial, γ = 0.20 | 12.80 | 3.58 | the panel is highly heterogeneous |
At γ = 0.10–0.20 the critical value grows by 1–2 answers — so a protocol that "passed" by the binomial table may turn out non-significant. For the triangle test there is also a chance-corrected version of the model (chance-corrected beta-binomial): it estimates the proportion of discriminators p_d directly, accounting for heterogeneity. Implementation — the betabin functions of the sensR package; in Sensostat the model is applied automatically as soon as replicates are specified in the design.
Ranking: the Friedman test
Ranking (ISO 8587:2006 / GOST R ISO 8587-2015) is analysed by the rank analogue of two-way ANOVA — the Friedman test: χ²F = 12/(n·k·(k+1)) · ΣRᵢ² − 3n(k+1), where Rᵢ are the sample rank sums.
Example. 12 assessors rank 4 coffees by bitterness (1 = least bitter). Rank sums: A=20, B=22, C=32, D=46.
χ²F = 12/(12·4·5)·(20²+22²+32²+46²) − 3·12·5 = 0.05·4024 − 180 = 21.2 > χ²0.05;3 = 7.815 (p < 0.001) → the samples differ. Post-hoc: a difference of rank sums is significant if it exceeds LSD ≈ t·√(n·k(k+1)/6) ≈ 12.7. Conclusion: D is significantly more bitter than A and B; A, B and C do not differ from each other.
ANOVA and mixed models for descriptive data
The basic panel model: Y = product + assessor + product×assessor (+ replicate). Assessor is a random factor; product significance is tested against the product×assessor interaction, not against the residual (see the analysis in module 05). Post-hoc — Tukey HSD.
Worked example: variance decomposition by hand
A QDA session, attribute "bitterness" (0–10 scale): 3 products × 4 assessors × 2 replicates. The total sum of squares decomposes into four components:
df: 23 = 2 + 3 + 6 + 12
| Source | SS | df | MS | F | p |
|---|---|---|---|---|---|
| Product | 29.08 | 2 | 14.54 | 12.03¹ | 0.008 |
| Assessor | 2.13 | 3 | 0.71 | 1.55² | 0.254 |
| Product × Assessor | 7.25 | 6 | 1.21 | 2.64² | 0.072 |
| Error (replicates) | 5.50 | 12 | 0.46 | ||
| Total | 43.96 | 23 |
¹ against the interaction MS (mixed model, assessor random) · ² against the error MS.
See what the correct denominator does: the same product effect against the error MS would give F = 31.7 and p = 0.000016 — four orders of magnitude "stronger"! This is an illusion: the error MS measures only the replicate noise of one person, while we generalise the conclusion to the population of assessors. A significant interaction (here p = 0.072 — borderline) warns: the panel is not fully in agreement, assessors rank the products differently, and the product means should be read with caution.
Product means: A = 3.25; B = 5.75; C = 5.38. Tukey's post-hoc (honestly significant difference, HSD) based on the interaction:
Differences: B−A = 2.50 > 1.69 — significant; C−A = 2.13 > 1.69 — significant; B−C = 0.38 < 1.69 — no difference. Conclusion for the client: "The products differ in bitterness (mixed ANOVA, F(2,6) = 12.03; p = 0.008): recipe A is significantly milder than both competitors; B and C do not differ. Panel agreement is borderline (interaction p = 0.072) — we recommend a calibration session on the bitterness scale".
# The same calculation — reproducible (Python, statsmodels) import pandas as pd, statsmodels.formula.api as smf from statsmodels.stats.anova import anova_lm df = pd.DataFrame({ 'y':[3,4,4,3,2,3,3,4, 6,7,4,5,7,6,5,6, 5,6,6,5,4,5,6,6], 'prod': ['A']*8+['B']*8+['C']*8, 'ass': ['J1']*2+['J2']*2+['J3']*2+['J4']*2}) m = anova_lm(smf.ols('y ~ C(prod)*C(ass)', df).fit(), typ=2) F_prod = m.loc['C(prod)','MS'] / m.loc['C(prod):C(ass)','MS'] # 12.03
PCA: the product map
From the "panel means: products × attributes" matrix, a PCA (principal component analysis) is built with centring and scaling of the attributes. The biplot shows products (points) and attributes (vectors) simultaneously: the vector length is the attribute contribution, the direction — which products it is associated with.
PCA biplot: five chocolates, six attributes (module 05 data)
Related methods: MFA (multiple factor analysis) for napping and multi-block data; GPA (generalised Procrustes analysis) for free-choice profiling; the RV coefficient for comparing configurations (RV ≥ 0.8 — configurations are similar); AHC — product clustering.
The mathematics of PCA in three steps
The eigenvector v_j is a direction (a principal component), the eigenvalue λ_j is the variance along it. Product scores are the projections X·v_j, loadings are the attributes' links with the component. The share of explained variance: λ_j / Σλ. In the biplot above, the first two components explain the bulk of the differences between the five chocolates: PC1 separates the dark ones from the white and milk (cocoa, bitterness versus sweetness and vanilla), PC2 — ruby with its acidity. Practical rules: components with λ < 1 (the Kaiser rule) are usually noise; attributes whose arrows point the same way duplicate each other — candidates for trimming the lexicon.
The family is wider than PCA: CA (correspondence analysis) — for CATA frequencies; MFA — for data blocks from different respondents (napping, module 07); GPA (generalized Procrustes analysis) — when each assessor has their own coordinate system (free choice profiling); PLSR — when sensory data explain hedonic response (preference mapping, module 06).
Consumer statistics
- Hedonic tests: means ± CI, mixed ANOVA (product fixed, respondent random), Tukey; preference segmentation — hierarchical clustering (Ward).
- CATA: Cochran's Q test per attribute, post-hoc — pairwise McNemar with correction; the "products × attributes" map — correspondence analysis (CA).
- JAR: penalty analysis — mean drop and t-test/bootstrap, the ≥ 20 % rule (module 06).
- Preference mapping: external (PLSR: liking ~ sensory coordinates) and internal (MDPREF) — liking drivers.
Cochran's Q for CATA — a worked example
Four yogurts, 12 consumers, the attribute "creamy" (0/1 ticks). Product sums: A = 10, B = 4, C = 9, D = 3. The statistic:
where Cⱼ are the product sums, Rᵢ the respondent sums, k = 4 products. Here Q = 10.57; χ²(3): p = 0.014 — the citation frequencies differ significantly. Post-hoc — pairwise McNemar on discordant pairs: A–D: 8/1, p = 0.039; C–D: 7/1, p = 0.070; A–B: 8/2, p = 0.109.
Power: how much data is enough
Panel size is chosen before the study, not after. For discrimination tests — from (α, β, p_d): an exact search is implemented in the module 04 calculator. Benchmarks for the triangle test at α=0.05, β=0.10: p_d=50 % → n≈20–25; p_d=30 % → n=53; p_d=20 % → n=117.
For consumer tests, power is computed from the smallest meaningful difference of means (Δ liking) and the variance — using t-test formulas.
Further reading
- Brockhoff P. B., Christensen R. H. B. Thurstonian Models as GLM. Food Quality and Preference, 2010 — binomial and beta-binomial models of discrimination tests in the modern formulation. article
- Lê S., Josse J., Husson F. FactoMineR: An R Package for Multivariate Analysis. Journal of Statistical Software, 2008 — PCA/CA/MFA in R; open access. article · OA
- SensoMineR and sensR — open packages used to check this module's calculations: from panel ANOVA to d′. tools
- Lawless & Heymann, 2nd ed. — statistical appendices I–V — from binomial tables to test power, written in a sensory scientist's language. book
Test yourself
Three questions on statistics
1. In the mixed panel model, the significance of the "product" factor is tested…
Assessor is a random factor, so the product F-test goes against MS(product×assessor). Against the residual — only if there is no interaction in the model.
2. In the Friedman example χ²_F = 21.2 against a critical value of 7.815. Conclusion?
21.2 > 7.815 → the null hypothesis of equality is rejected at the 0.05 level (in fact p < 0.001).
3. Two principal components in the chocolate example explain 98.6 % of the variance. This means…
High explained variance of two components is a sign that the biplot adequately reflects the data structure.
From theory to industries
Module 10: how sensory science works in the food industry, cosmetics, QC and marketing.