SPSS command: SPSS WEIGHTED KAPPA
Weighted kappa in your browser
Weighted kappa measures agreement between two raters on an ordered scale, giving partial credit when they disagree by one category instead of three.
Free · No account · Files stay on your device · Updated 18 October 2026
What it tells you
Crosstabs runs the SPSS WEIGHTED KAPPA procedure on your device with linear or quadratic weights, the asymptotic standard error, a z test against chance agreement, a confidence interval and the full rating crosstabulation.
- How much two raters agree beyond chance once near misses count as partial agreement.
- Whether that agreement is significantly better than chance (z test using the null standard error).
- A confidence interval for kappa from the asymptotic standard error.
- Where the disagreements fall, from the rater-by-rater crosstabulation.
When to use it
Use it when
- Two raters score the same items on an ordinal scale: severity grades, essay marks, Likert-type ratings.
- Test-retest agreement of one rater on an ordered scale, with the two occasions as the two variables.
- Reporting inter-rater reliability where quadratic weighted kappa is the expected statistic (it approximates the intraclass correlation).
Use something else when
- Categories have no order (diagnosis A, B, C): use unweighted Cohen's kappa in the Crosstabs statistics panel.
- Three or more raters: use Fleiss' kappa.
- Continuous measurements: use an intraclass correlation from Reliability Analysis or a Bland-Altman style analysis.
How to run it in Crosstabs
- 01
Open your data
Open a file with one row per rated item and one column per rater, then switch to Statistics mode.
- 02
Choose Weighted Kappa
In the Agreement and diagnostics group, pick Weighted Kappa (WEIGHTED KAPPA).
- 03
Choose the two raters
Set Rater 1 and Rater 2 to two different variables. Categories are the sorted union of both raters' values; non-integer numeric ratings are truncated.
- 04
Pick the weights
Set Weighting to Linear (1 − |i − j|/(k − 1)) or Quadratic (1 − (i − j)²/(k − 1)²), and choose the Confidence level (%).
- 05
Run
Declare any weight as a frequency weight (useful when each row is a rating pattern with a count) and press Run.
Worked example
Computed by the Crosstabs engine from the synthetic data below when this page was built — not typed in.
Data
50 synthetic essays scored on a 1–5 scale by two raters. Each rater's score is the essay's true level plus a random shift of −1, 0 or +1 (rater 2 is occasionally a point more lenient), clamped to 1–5. Fixed seed.
| Ratings (weighting) | N | Kappa | Asymptotic Std. Error | Z | Sig. | 95% CI |
|---|---|---|---|---|---|---|
| rater1 – rater2 (Linear) | 50 | 0.773 | 0.047 | 7.69 | <.001 | 0.681 – 0.866 |
| rater1 – rater2 (Quadratic) | 50 | 0.905 | 0.025 | 6.45 | <.001 | 0.856 – 0.954 |
Two runs of the panel, one per Weighting choice. Z uses the standard error under the null hypothesis of chance agreement; the interval uses the asymptotic standard error.
| rater1 \ rater2 | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| 1 | 7 | 2 | 1 | 0 | 0 |
| 2 | 2 | 7 | 1 | 0 | 0 |
| 3 | 0 | 0 | 3 | 4 | 0 |
| 4 | 0 | 0 | 1 | 5 | 5 |
| 5 | 0 | 0 | 0 | 2 | 10 |
How to read it
The raters give exactly the same score to 32 of 50 essays, and 17 of the 18 disagreements are only one point apart, which weighted kappa counts as partial agreement: linear κw = 0.773 (95% CI 0.681 – 0.866) and quadratic κw = 0.905 (95% CI 0.856 – 0.954). Quadratic weights penalise a one-point miss less than linear weights do, so they give the larger value when most disagreements are near misses.
Output tables explained
- Weighted Kappa
- N, kappa, asymptotic standard error, Z, significance and confidence interval. Z uses the standard error under chance agreement; the interval uses the asymptotic standard error.
- Rating Crosstabulation
- Counts of Rater 1 by Rater 2 categories; agreements lie on the diagonal and near misses next to it.
Assumptions and notes
- The same items are rated independently by both raters on the same ordered categories.
- Categories are equally spaced for linear weights; quadratic weights penalise large disagreements much more.
- At least two categories are used; if both raters use a single category, kappa is undefined.
- Cases with a missing rating from either rater are excluded listwise.
How the results are validated
On a 140-case weighted reference table, linear and quadratic weighted kappa, the asymptotic standard error and the null standard error match statsmodels 0.15 cohens_kappa to 1e-12 and z to 1e-10; kappa also matches R irr 0.85 to 1e-12. Validation evidence
Questions
Linear or quadratic weights?
Linear weights treat each step of disagreement equally. Quadratic weights barely penalise one-step misses and heavily penalise large ones; they are standard in many fields because quadratic kappa approximates the intraclass correlation. State your choice in advance.
How do I describe the strength of agreement?
The panel uses the Landis and Koch benchmarks: up to .20 slight, .21–.40 fair, .41–.60 moderate, .61–.80 substantial, above .80 almost perfect. Treat them as rough labels, not thresholds.
Why is kappa low even though raters agree most of the time?
Kappa subtracts the agreement expected by chance. When almost every item falls in one category, chance agreement is already high, so even frequent agreement yields a modest kappa.
Why do the Z test and the confidence interval use different standard errors?
The test asks whether kappa could be 0, so it uses the standard error under that null; the interval describes the estimate itself, so it uses the asymptotic standard error. This is the convention SPSS follows.
Related procedures
- Fleiss' KappaFLEISS MULTIRATER KAPPA
- Crosstabs StatisticsCROSSTABS /STATISTICS
- Reliability AnalysisRELIABILITY
- Bivariate CorrelationsCORRELATIONS
- Free SPSS alternative: full coverage list
Run Weighted Kappa on your own data
Open a .sav, CSV or Excel file. Everything runs in your browser; the file never leaves your device.
SPSS is a trademark of IBM Corp. Crosstabs is not affiliated with or endorsed by IBM.