Sensostat Try the demo

Textbook · Module 02

Assessors and Sensory Panels: Selection, Training, Monitoring

Panel types according to ISO 8586:2023, sensitivity screening (ISO 3972, ISO 5496), the training programme, and panel performance monitoring to ISO 11132.

Module 02 of 11~25 min readinteractive + quiz
Panel training: assessors in booths evaluate coded samples while the panel leader compares the scores with the consensus
A panel training session: each assessor's scores are compared with the group consensus — this is how a "measuring instrument" is calibrated out of people.

The assessor as a measuring instrument

In sensory analysis the "instrument" is a human being, and the result depends on who exactly evaluates the product. ISO 8586:2023 "Sensory analysis — Selection and training of sensory assessors" distinguishes qualification levels:

Assessor typeWho they areSuitable tasks
Naive assessorA person with no selection criteria or experienceConsumer tests (hedonics, CATA)
Initiated assessorHas already taken part in sensory testsPreliminary discrimination tests, QC in/out
Selected assessorHas passed sensory-ability screeningDiscrimination tests, batch screening
Trained assessorTrained in specific methods and scalesDescriptive analysis (QDA, profiling)
Expert sensory assessorTrained + proven long-term performance and deep category knowledgeCertification of reference standards, lexicon development, arbitration of disputed batches

Typical panel sizes: a trained descriptive panel — 8–12 assessors (2–3 replicates), discrimination tests — 24–30 selected assessors, consumer hedonic tests — from 100 respondents with no training. Training an expert panel takes months — it is an investment, not a one-off campaign.

Screening: whom to admit to the panel

Selecting candidates is a multi-stage filter. Each step has an objective pass criterion.

  1. Questionnaire and interview — health, allergies, medication, smoking, motivation, time availability. Contraindications and unreliable attendance are screened out.
  2. Basic tastes (ISO 3972) — recognition of sweet (sucrose), sour (citric acid), salty (NaCl), bitter (caffeine) and umami (monosodium glutamate) in aqueous solutions; ranking concentration series by intensity.
  3. Smell (ISO 5496) — identification and ranking of odours (vanillin, lemon, clove, etc.), including with a pinched nose to separate taste from retronasal olfaction.
  4. Discrimination ability — a series of triangle tests on real products: the score is evaluated with a binomial criterion (calculator below).
  5. Descriptive ability — the skill of verbalizing sensations: "this is not just sweet, it is a caramel sweetness with a note of burnt sugar".

Screening calculator interactive

The classical selection criterion: a candidate completes a series of triangle tests, and their result is compared with the critical number of correct responses at α = 0.05 (exact binomial criterion, as in the tables of ISO 4120:2021).

Set the series parameters and press "Evaluate the candidate".
Do not confuse: the screening criterion "10 correct out of 15" is not a "two-thirds pass mark" but a statistical threshold: at p₀ = 1/3 the probability of randomly scoring 10+ out of 15 is ≈ 0.008 — well below α = 0.05.

Training to ISO 8586:2023

Training turns a selected assessor into a "calibrated instrument". The programme builds from the general to the specific:

Stage 1

Sensory ABC: basic tastes and odours, the concept of intensity, working with scales, booth rules (no perfume, no talking, spitting).

Stage 2

Discrimination practice: triangle tests and ranking on category products with increasing difficulty; calibration on deliberately different samples.

Stage 3

Lexicon and references: the panel collectively builds a descriptor vocabulary (ISO 11035) and anchors a physical reference standard and its scale position for each descriptor.

Stage 4

Method training: practising the specific method (profiling, TDS) on training samples; comparing individual scores with the panel consensus.

Stage 5

Qualification: a candidate is admitted to working sessions if their scores are reproducible (repeatability across replicates), agree with the panel and discriminate between products. Then comes permanent monitoring.

The full cycle of preparing a descriptive panel usually takes from 2–3 months to half a year of regular sessions (1–3 per week). Saving on training means measuring noise instead of the product.

Panel performance monitoring: ISO 11132

A trained panel does not "work forever" — its metrology is controlled per ISO 11132:2021 through three criteria:

  • Discrimination ability — does the assessor find differences between products: the p-value of the "product" factor in their individual ANOVA for each attribute.
  • Agreement with the panel — do their scores correlate with the panel means; revealed through the "assessor × product" interaction.
  • Repeatability — the spread of their scores for the same sample across replicates (individual MSE).

The "p–MSE" map interactive

The classical panel diagnostic (as in PanelCheck): the X axis shows the p-value of the "product" factor in the assessor's individual ANOVA (discrimination), the Y axis shows the mean square error across replicates (repeatability). The bottom-left corner is the exemplary assessor. Click an assessor's point to get a recommendation.

Click an assessor's point on the chart.

At the whole-panel level, additional tracking includes panel control samples from session to session, control charts of means and — for deep analysis — the MAM model (Mixed Assessor Model), which separates assessors into "core" ones, "scalers" (using the scale differently) and outliers.

Ethics, safety and panel retention

  • Informed consent and the right to refuse any sample and leave the panel at any time without giving reasons and without sanctions; allergy and atopy screening before the first tasting (IFST, 2025 edition).
  • Risk assessment before the test: special categories — novel foods, unapproved ingredients, products past their shelf life in shelf-life tests (microbiological safety is confirmed by a microbiologist, not by the assessor). Products returned from retail are not allowed for tasting.
  • Minimizing deception: if the design requires concealing the test purpose, this is justified and recorded in advance; after the study, assessors are debriefed.
  • Tasting hygiene: room-temperature water and neutral palate cleansers (unsalted crackers, bread) between samples; spitting is mandatory for long series and for products with carry-over effects.
  • Workload limits: as a rule, no more than 6–8 samples per session for taste tests; no more than 1–2 sessions per day; no tasting during a cold and within an hour after eating, coffee or smoking.
  • Motivation: regular feedback on personal performance (Compusense's feedback calendar is strong at this), a clear schedule and recognition of the panel's role in company decisions — otherwise the panel "burns out" within a year.
  • Assessor data is personal data: storage and processing under Federal Law 152-FZ, anonymization in reports (codes instead of names).

Further reading

Test yourself

module 02 quiz

Three questions on panels

1. Quantitative descriptive analysis (QDA) requires a panel of…

QDA is a calibrated-panel method: 8–12 trained assessors, individual scores, replicates, ANOVA. A hundred consumers are needed for hedonic testing, not for profiling.

2. Under ISO 11132, an assessor's repeatability is evaluated through…

MSE across replicates is repeatability; p(product) is discrimination ability; correlation/interaction is agreement. Three different criteria.

3. A candidate completed a series of 15 triangle tests with 8 correct answers. Do they pass screening at α = 0.05?

Check with the calculator above: at n = 15, p₀ = 1/3 the critical number is 9; P(X ≥ 8) ≈ 0.089 > 0.05. "More than a third" is not enough — a statistical margin is needed.

Where to seat the panel and how to serve the samples

Module 03 covers facilities per ISO 8589, blind codes, balanced serving designs and the tasting plan generator.

Module 03 →