Statistical methods

Crosstabs reports descriptive tables and common association statistics. This page states the calculation rules, assumptions, warnings, and current editorial-review status.

Last updated

Validation · computed at build48 cases

Largest difference from SciPy, statsmodels and R

5.0 × 10−13

15 of 15 statistic families within tolerance; a report prints three decimals (10⁻³).

  1. Pearson chi‑square + Cramér's V1.4 × 10−14
  2. Fisher's exact (two‑sided)2.8 × 10−17
  3. Yates‑corrected chi‑square8.5 × 10−16
  4. Likelihood‑ratio G‑test1.1 × 10−28
  5. Somers' d0
  6. Phi, V, C, lambda, Goodman–Kruskal tau, U (with ASE and p)2.7 × 10−14
  7. Gamma, tau‑b, tau‑c, Somers' d (with ASE and p)8.9 × 10−15
  8. Pearson r, Spearman, linear‑by‑linear, eta2.8 × 10−14
  9. Cohen's kappa, McNemar, McNemar–Bowker7.1 × 10−15
  10. Odds ratio and relative risks7.1 × 10−14
  11. Fisher's exact (one‑sided)2.1 × 10−15
  12. Fisher–Freeman–Halton exact (R×C)5.7 × 10−14
  13. Raw and standardized residuals0
  14. Cochran–Mantel–Haenszel, Breslow–Day, Tarone, MH odds ratio5.0 × 10−13
  15. Odds ratio, phi, C, lambda, gamma, tau‑b/c, U and adjusted residuals1.1 × 10−16
Inspect every case · public reference manifest

Table construction and notation

Each retained record adds 1 to its cell, and missing values are excluded unless you turn on Include missing. Percentages and expected counts use the standard row, column, total and RiCj/N definitions.

Notation and validity rules

After active filters, each retained record contributes 1 to an ordinary workspace cell. A supported saved survey design may instead execute its declared weight and variance contract. An untyped weight selection returns no table. Missing row or column values are excluded by default; the Include missing setting retains them as an explicit (Missing) category.

  • For observed cell Oij, row total Ri, column total Cj, and grand total N, row percentage is 100Oij/Ri, column percentage is 100Oij/Cj, and total percentage is 100Oij/N.
  • Expected frequency under independence is Eij = RiCj/N.
  • A valid inferential table needs a positive grand total and positive totals for every displayed row and column.

Saved execution settings and column tests

Saved projects compile every setting into one versioned input that is applied exactly or rejected. Column tests are two-sided pooled z-tests, with Bonferroni correction and a minimum base of 30 by default.

Test profiles, weights and digests

Saved projects use Research Execution Contract v1. Row sets, multibanners, table and banner filters, weights, category order, missing-value rules, wave mappings, and test profiles are compiled into one versioned input. A value is applied exactly or the table is rejected; saved choices are not silently ignored.

  • Column-proportion tests compare each pair of testable banner columns with a two-sided pooled z-test. A saved profile controls 90%, 95%, or 99% confidence tiers, Bonferroni or no correction, minimum base, unweighted or weighted effective bases, and display mode.
  • Ad-hoc workspace defaults are 95%/90% confidence, Bonferroni correction across testable column pairs, minimum unweighted respondent base 30, and letter display.
  • An untyped numeric weight cannot produce a table or column tests. A saved project may execute ordinary tests for an explicit non-negative integer frequency weight, and a supported saved survey design executes design-aware inference. Calibration and analytic weights remain unsupported for ordinary inference.
  • Every contract-bearing result includes engine and method versions, per-setting diagnostics, and canonical SHA-256 input/result digests. The digests prove deterministic equality under this contract version; they are not signatures or evidence that a sample is representative.

Pearson, G-test, and residuals

Pearson's χ² and the likelihood-ratio G share df = (r − 1)(c − 1). Adjusted residuals flag cells beyond the chosen two-sided threshold, without a multiple-comparison adjustment.

Formulas and thresholds

Pearson's statistic is χ² = Σ(Oij − Eij)²/Eij, with df = (r − 1)(c − 1). Its p-value is the upper tail of the chi-square distribution. The likelihood-ratio alternative is G = 2Σ Oij log(Oij/Eij) over nonzero observed cells, using the same degrees of freedom and an asymptotic reference distribution.

Adjusted residuals are (Oij − Eij) / √[Eij(1 − Ri/N)(1 − Cj/N)]. Cells beyond the saved profile's strict two-sided threshold receive a directional flag (90%: ±1.645; 95%: ±1.960; 99%: ±2.576). Crosstabs does not adjust these cellwise residual flags for multiple comparisons.

2×2 calculations

2×2 tables add Yates' correction, a two-sided Fisher exact test for unweighted counts within the support limit, and the odds ratio with a large-sample Woolf interval.

Yates, Fisher and odds-ratio rules

Yates' correction uses Σ max(|O − E| − 0.5, 0)²/E. It is displayed for 2×2 tables and is intended for frequency counts. Fisher's exact test is a two-sided Fisher–Irwin, fixed-margin hypergeometric test: it sums feasible tables whose probability is no greater than the observed table. It is available only when all four cells are non-negative safe integers and the fixed-margin support contains at most 100,000 tables; it is hidden for weighted workspace tables.

For [[a,b],[c,d]], the odds ratio is ad/bc. The displayed 95% confidence interval is the large-sample Woolf interval exp(log(OR) ± 1.96√(1/a + 1/b + 1/c + 1/d)), not an exact interval. A zero cell produces the uninformative interval [0, ∞].

Association measures

Cramér's V, phi, the contingency coefficient, lambda, gamma, Kendall's tau, Somers' d and Theil's U are reported. The ordinal measures assume the displayed category order is the intended order.

Definitions and ordinal caveats
  • Cramér's V is the uncorrected V = √[χ²/(N min(r−1,c−1))]. Labels such as small, medium, and large use heuristic Cohen-style thresholds; they are not intrinsic classifications.
  • Phi is the signed 2×2 coefficient. The contingency coefficient is √[χ²/(χ²+N)] and its attainable maximum depends on table dimensions.
  • Goodman–Kruskal lambda is proportional reduction in prediction error. Gamma is (concordant−discordant)/(concordant+discordant).
  • Kendall tau-b and tau-c account for ties and table shape. Somers' d is reported symmetrically and directionally as d(row|column) and d(column|row).
  • Theil's U reports uncertainty reduction in each direction and symmetrically.

Gamma, Kendall tau, and Somers' d are meaningful only when the displayed category sequence represents the intended ordinal order. Manual count calculators preserve entered order. Saved row and banner category order executes exactly; unlisted observed levels follow the deterministic numeric-aware label sort. Check the recorded order before interpreting ordinal measures.

Outcome Studio predictive model

Outcome Studio fits a local, L2-regularized binary logistic model with at most eight drivers and checks it on one 80/20 holdout. It is unweighted, and its coefficients are model summaries, not causal effects.

Model rules and limits

Outcome Studio fits a local binary logistic model for one selected success value versus all other observed values. Numeric drivers are standardized; categorical drivers use reference-level indicator terms. The fit applies an L2 ridge penalty of 0.5 for stability and is bounded to eight selected drivers and 40 encoded terms.

  • Rows missing the outcome or any selected driver are excluded listwise. Successful results show used and excluded counts and the exclusion diagnostic.
  • Validation uses one deterministic, outcome-stratified 80/20 split. AUC, Brier score, accuracy at a 0.50 threshold, and calibration gap are calculated on held-out rows.
  • Outcome Studio is currently unweighted. Any selected workspace weight—including one attached to a supported crosstab survey design—blocks Outcome modeling until a typed, design-aware regression policy exists; no rows are evaluated and no run is persisted.
  • At least 60 complete rows, 25 successes, and 25 other outcomes are required. Predictors with no variance are omitted; categorical predictors above 20 levels and designs above 40 encoded terms are blocked before model allocation.
  • Driver rank is based on the largest absolute standardized coefficient per variable. Odds ratios and p-values are model summaries, not causal effects; regularization and single-holdout selection must be considered when interpreting them.
  • Observed segments are descriptive. Three-group k-means diagnostics run only with at least two numeric drivers and use at most 1,500 deterministic observed rows. Scenario and synthetic-cohort values are model outputs and never become observed evidence.
  • The model is not survival analysis, multilevel modeling, causal inference, true multiclass classification, or complex-survey variance estimation. Repeated holdout, cross-validation, fairness audits, and external validation remain analyst responsibilities.

The public capacity and workload envelope lists the deterministic domain fixtures exercised by the release suite.

Assumptions, weights, and warnings

Inference assumes independent observations and adequate expected counts. Untyped weights fail closed, only supported saved survey designs run design-aware inference, and warnings are diagnostics rather than guarantees.

All assumptions and warnings
  • Inference assumes independent observations, mutually exclusive categories, meaningful sampling, positive margins, and adequate expected counts.
  • Numeric weight candidates are preserved, but an ad-hoc or legacy untyped selection fails closed before results or evidence are returned.
  • Ordinary weighted chi-square, p-values, residuals, effect sizes, and column tests are unavailable until an explicit supported semantic policy authorizes them. A declared frequency weight runs ordinary tests on its whole-number counts.
  • With a declared survey (sampling) weight, column letters and column-mean letters use the effective-base test (approximation, as in SPSS Custom Tables/Q): each banner column's Kish effective base n_eff = (Σw)²/Σw² replaces its sample size in pooled t-tests, with the same correction and minimum-base rules. Chi-square, residuals and effect sizes are not reported; full design-based tests stay in Complex Samples.
  • Saved tab-book tables can bind a validated Taylor-linearized or replicate-weight survey design. Supported runs disclose strata, PSU, FPC or replicate policy, design degrees of freedom, t intervals, design effects, effective bases, and Rao–Scott or adjusted-Wald association. Invalid designs fail closed; independent statistical review is still pending.
  • Warnings identify small samples, expected counts below 5, when more than 20% of expected counts are below 5, blocked or invalid weight contracts, and very wide tables. They are diagnostics, not a guarantee that inference is valid.

The answer-first survey weighting and raking guide distinguishes design, calibration, and frequency weights and explains the quick-workspace fail-closed boundary and supported saved-design path.

Validation evidence

Selected Pearson chi-square, Fisher exact, Cramér's V, and directional Somers' d cases are compared with SciPy through a public reference manifest and automated tests. The CROSSTABS-style measures with asymptotic standard errors and approximate significance, exact and stratified tests are compared with statsmodels, R (DescTools, fisher.test, mantelhaen.test) and a 50-digit delta-method computation, and recomputed again from closed forms in the Python test suite. This evidence is intentionally narrower than a blanket software-equivalence claim and does not replace named review.

Browser engine compared with independent reference values
TestCasesMax |Δ|ReferenceStatus
Pearson chi-square + Cramér's V21.4 × 10−14SciPyWithin tolerance
Fisher's exact (two-sided)42.8 × 10−17SciPyWithin tolerance
Yates-corrected chi-square18.5 × 10−16SciPyWithin tolerance
Likelihood-ratio G-test11.1 × 10−28SciPyWithin tolerance
Somers' d10SciPyWithin tolerance
Phi, V, C, lambda, Goodman–Kruskal tau, U (with ASE and p)62.7 × 10−14R DescTools, SciPy and mpmathWithin tolerance
Gamma, tau-b, tau-c, Somers' d (with ASE and p)58.9 × 10−15R DescTools and mpmathWithin tolerance
Pearson r, Spearman, linear-by-linear, eta42.8 × 10−14SciPy, NumPy and mpmathWithin tolerance
Cohen's kappa, McNemar, McNemar–Bowker37.1 × 10−15statsmodels and SciPyWithin tolerance
Odds ratio and relative risks27.1 × 10−14statsmodelsWithin tolerance
Fisher's exact (one-sided)32.1 × 10−15SciPyWithin tolerance
Fisher–Freeman–Halton exact (R×C)45.7 × 10−14R statsWithin tolerance
Raw and standardized residuals20SciPy and NumPyWithin tolerance
Cochran–Mantel–Haenszel, Breslow–Day, Tarone, MH odds ratio25.0 × 10−13statsmodels and R stats/DescToolsWithin tolerance
Odds ratio, phi, C, lambda, gamma, tau-b/c, U and adjusted residuals81.1 × 10−16Python standard-library math; SciPy 1.18.0 plus Python standard-library math; NumPy; Python standard-library arithmetic; NumPy plus Python standard-library mathWithin tolerance

Inspect the values, tolerance, reproduction contract, and limits on the statistical validation page.

Ownership and review status

This methodology is published and maintained by crosstabs.com. A named statistical owner and independent reviewer have not yet been published, so the site does not claim that an anonymous “methods team” has reviewed it. For high-stakes work, verify the result with a qualified statistician or established statistical package.

Report a suspected error or unclear assumption to support@crosstabs.com. Confirmed corrections will update this page, affected outputs, and the changelog; published review metadata changes only after a real review of the affected revision.

Content updated .

Primary references