Survey tracker operations
How to Detect and Repair Survey Tracker Schema Drift Between Waves
Treat each imported wave as immutable. Fingerprint the reference and candidate schemas; match stable internal IDs before names or labels; classify structural and semantic drift; preview an explicit transformation plan; require human approval; apply it to a copy; validate bases, codes, and distributions; and retain the source, mapping, audit record, and inverse. Compare only variables whose concept, universe, wording, and representation remain defensibly comparable.
Published by crosstabs.com · Named statistical review pending · Last updated
Run this on your own data — free, no signup
Upload a CSV or XLSX. Everything runs in your browser; your file never leaves your device.
Open the workspace →Detect structural drift without confusing it with changed meaning
A schema diff can identify names, types, category vocabularies, display order, and added or removed fields. It cannot establish whether a new question still measures the same concept. DDI's comparison model treats variables, questions, categories, universes, and representations as separate comparison targets; tracker review should do the same.
| Drift class | Detection evidence | Repair decision | Primary risk |
|---|---|---|---|
| Rename | Stable ID, codebook, explicit map | Map old name to new name | A reused label can hide a different concept |
| Type change | String / numeric / multi-select type | Convert only with validation rules | Invalid values can become missing |
| Category recode | Code and label sets | Map every old and new code explicitly | Reversed scales manufacture a trend |
| Category order | Same codes in a new display order | Set an explicit canonical order | Display order can be mistaken for recoding |
| Variable order | Same fields in a different column order | Ignore for meaning; retain for provenance | Position-based mapping links the wrong fields |
| Multilingual label | Stable ID plus reviewed translation | Retain locale-specific labels | Translation similarity is not equivalence |
| Added or removed | Questionnaire version and schema diff | Include, exclude, or block a required field | A missing required variable invalidates a saved table |
| Semantic drift | Wording, universe, routing, recall period | Harmonize with documented judgment or break the series | The schema alone cannot detect changed meaning |
Use stable identifiers, versions, and codebooks as evidence
Start with the questionnaire version, internal question or variable IDs, export settings, and data dictionary. Then compare labels. Qualtrics documents that adding or removing answer choices can reorder recode values, while its internal-ID export option provides non-editable identifiers. A friendly label alone is a weak join key.
Automatic matching can propose a candidate only when the match is unique and type-compatible. Ambiguous matches stay unresolved, and label similarity is not proof of substantive equivalence.
Worked example
Worked example
| Reference wave | Candidate wave | Approved transform | Validation |
|---|---|---|---|
| SAT | Q12_sat | Map variable after ID, wording, universe, and routing review | One unique target; expected base reconciles |
| 1 / 2 / 3 | 2 / 1 / 0 | 2→1, 1→2, 0→3 | No unmapped codes; frequencies reconcile before and after |
| Satisfied / Neutral / Dissatisfied | 满意 / 一般 / 不满意 | Retain both locale labels with the approved code map | Bilingual review confirms equivalent response meanings |
How to interpret it
Rule of thumb
The approved transform makes the candidate wave structurally usable; it does not prove longitudinal comparability. Reconcile row counts, missing rates, category sets, ranges, table bases, and distributions, then record any wording, universe, routing, or recall-period discontinuity beside the result.
Preview, approve, apply, and validate an explicit repair plan
A repair plan should enumerate every map, type conversion, value recode, category-order change, inclusion, and exclusion. Mark which operations are required by saved row sets, banners, filters, weights, or coding variables. Block the wave while a required operation is unresolved.
Apply approved operations to a copy and keep the source wave unchanged. Validate the imported row count, unmapped-value rate, unique values, numeric ranges after conversion, missingness by wave, and the unweighted base for every refreshed table. Investigate distribution jumps before calling them movement.
Make reversal exact, guarded, and non-destructive
Crosstabs' project engine records the source fingerprint, project hash, approved and rejected operations, actor, timestamp, and a pre-repair snapshot. An exact undo is refused if the project changed after repair, because restoring the old snapshot would erase later work. In that case, preserve history and create a new forward repair.
Reversal protects workflow integrity, not statistical meaning. A repair that is mechanically invertible can still be substantively wrong, so the approval evidence and comparison notes must travel with the wave.
Current Crosstabs product boundary
Today's Workspace Waves & refresh panel imports a compatible local replacement as a new wave, suggests safe rename mappings, retains prior waves, blocks unresolved required fields, and compares saved tables across waves. It never overwrites the existing source wave.
The underlying project engine models explicit type conversion, recoding, category order, approval records, audit evidence, and guarded exact reversal. The Workspace does not yet expose the full approval and reversal interface. Use the visible compatible-wave workflow as the shipped UI boundary, not the larger engine contract.
Frequently asked questions
- What counts as survey tracker schema drift?
- Structural drift includes renamed, added, or removed variables; changed types; changed category codes or order; variable reordering; and multilingual label changes. Semantic drift includes changes to question wording, universe, routing, recall period, or concept. A file schema can detect much of the first group but cannot prove the second group is comparable.
- Should similarly named variables be matched automatically?
- No. Stable internal IDs, questionnaire versions, codebooks, and explicit analyst mappings are stronger evidence than labels. Similar labels can support a suggestion, but label similarity is not proof of substantive equivalence and ambiguous suggestions should remain unapproved.
- Should a repair overwrite the old tracker wave?
- No. Retain the source wave unchanged, apply approved transformations to a copy, and store the mapping, input fingerprint, actor, time, operation status, and inverse or prior snapshot. This preserves reproducibility and lets comparisons be audited later.
- When should two waves not be compared?
- Do not treat a variable as longitudinally comparable when its measured concept, target universe, question wording, response representation, or missing-value meaning changed materially and no defensible harmonization exists. Report the discontinuity instead of manufacturing a trend.
- Can a tracker schema repair be undone?
- The Crosstabs project engine stores a pre-repair snapshot and allows exact reversal only while the post-repair project still matches its recorded integrity hash. If later work changed the project, exact undo is refused; create a new forward repair so later edits and the audit history are not silently erased.
References & further reading
Try it on your own data — free, no signup
Upload a CSV or XLSX. Everything runs in your browser; your file never leaves your device.
Open the workspace →