Market research method
Penalty analysis for just-about-right (JAR) scales
Penalty analysis links just-about-right (JAR) ratings to overall liking. For each attribute it splits respondents into “too little”, “just about right” and “too much”, then measures how far liking drops on each off-target side and weights that drop by the share of respondents on it. The largest weighted penalties are the attributes to fix first.
How results are checked
Named statistical review pending. Selected automated reference cases are published on the validation page.Run a penalty analysis — free, no signup
Open a file with an overall liking score and one column per JAR attribute (3-, 5- or 7-point), and get mean drops, weighted penalties, Welch t-tests and the 20% action threshold per side. Free and browser-local.
Run a penalty analysis →What a JAR scale measures
A just-about-right question asks whether an attribute is at the right level: for sweetness on a five-point scale, 1 = much too little, 3 = just about right, 5 = much too much. On its own it tells you the direction people want to move. It does not tell you whether being off target matters — some people say “a bit too salty” and still love the product.
Penalty analysis answers that by bringing in the overall liking score (often a nine-point hedonic scale) from the same respondents.
The method
Formula
Mean drop = mean liking(JAR) − mean liking(side) Weighted penalty = mean drop × proportion on that side
where
- JAR
- respondents who answered the middle, just-about-right code
- side
- respondents below the JAR point (too little) or above it (too much), each analysed separately
- proportion
- the side's share of everyone who answered the attribute and the liking question
- test
- Crosstabs adds a Welch two-sample t-test of the JAR group's liking against each side
Worked example
The synthetic snack tracker that the penalty tool loads as sample data has overall liking on a 1–9 scale and five-point JAR questions for sweetness, saltiness and crunch. Mean liking is 5.66 across all 600 respondents.
| Attribute: side | % of respondents | Mean liking | Mean drop | Weighted penalty | Welch p |
|---|---|---|---|---|---|
| Sweetness: too much | 40.8% | 4.77 | 1.65 | 0.675 | < .001 |
| Saltiness: too little | 35.7% | 5.09 | 0.99 | 0.352 | < .001 |
| Sweetness: too little | 21.0% | 6.02 | 0.41 | 0.086 | .001 |
| Crunch: too much | 22.3% | 5.44 | 0.35 | 0.079 | .012 |
| Saltiness: too much | 21.0% | 5.77 | 0.31 | 0.065 | .033 |
| Crunch: too little | 23.3% | 5.57 | 0.22 | 0.052 | .118 |
Synthetic snack-brand tracker · 600 respondents · the same data as “Use sample data”. JAR group mean liking: Sweetness 6.42 (38.2% JAR), Saltiness 6.08 (43.3% JAR), Crunch 5.79 (54.3% JAR).
Sweetness: too much is the clear priority: 40.8% of respondents are on that side and their liking is 1.65 points lower than among those who find it just right, a weighted penalty of 0.67. Next comes saltiness too little (35.7%, drop 0.99).
Every side here is at or above the 20% threshold, so size alone does not separate them — the mean drop does. Crunch too little has a drop that is not significant at 5%, so it is no reason to reformulate. The brief for the product team: make it less sweet first, then a little saltier; crunch costs little in either direction.
Using it well
When to use it
Use it when
- Use: Sensory and product tests where each attribute was asked on a JAR scale alongside overall liking.
- Use: Prioritising reformulation: which attribute, in which direction, costs the most liking.
- Use: Comparing prototypes on the same set of attributes.
Not the right tool when
- Avoid: Attributes asked on a liking or intensity scale rather than a JAR scale — there is no just-right middle to compare against.
- Avoid: Small tests where a side has only a handful of respondents; the drop is then too noisy to act on.
- Avoid: Concluding that fixing an attribute will raise liking by the full penalty: the analysis is correlational, and attributes interact.
How to interpret it
Rule of thumb
Act on sides that are both common and costly. A widely used rule of thumb treats a side as worth acting on when at least 20% of respondents are on it and the mean drop is meaningful (often about one point on a nine-point scale); the weighted penalty combines the two into one ranking.
Reading the penalty plot
Penalty results are usually plotted with the proportion of respondents on the x-axis and the mean drop on the y-axis, one point per attribute side. The top-right quadrant — many respondents, large drop — holds the priorities. Points high on the left are serious for a few people; points low on the right are common complaints that do not hurt liking.
The two sides of one attribute can point in opposite directions. When both are large, the attribute splits the market and a single reformulation will not satisfy everyone; consider variants instead.
Questions
Frequently asked questions
What is a just-about-right (JAR) scale?
A JAR scale asks whether an attribute is at the right level, with a middle point for just about right and points on either side for too little and too much. Five-point versions are the most common; three- and seven-point versions are also used.
How is the penalty calculated in penalty analysis?
For each side of each attribute, the mean drop is the JAR group's mean liking minus that side's mean liking. The weighted penalty multiplies the mean drop by the proportion of respondents on that side.
What is the 20% rule in penalty analysis?
It is a common convention: a side is only considered for action when at least 20% of respondents are on it. Below that, even a large drop affects too few people to justify reformulation.
Should I test the mean drop for significance?
It helps. Crosstabs runs a Welch two-sample t-test of the JAR group's liking against each side, so you can see which drops are larger than sampling noise. The test does not replace judgement about whether a drop is big enough to matter.
Can penalty analysis use weighted data?
Yes. With a weight column the proportions and means are weighted, and the t-test uses Kish effective sample sizes. Without one, every respondent counts once.
Sources and next steps
References & further reading
- [1]Rothman, L., & Parker, M. J. (Eds.). (2009). Just-About-Right (JAR) Scales: Design, Usage, Benefits, and Risks (ASTM Manual MNL63). ASTM International.
- [2]Lawless, H. T., & Heymann, H. (2010). Sensory Evaluation of Food: Principles and Practices (2nd ed.). Springer.
Run a penalty analysis — free, no signup
Open a file with an overall liking score and one column per JAR attribute (3-, 5- or 7-point), and get mean drops, weighted penalties, Welch t-tests and the 20% action threshold per side. Free and browser-local.
Run a penalty analysis →