Survey data quality
Survey data cleaner
How results are checked
Named statistical review pending. Selected automated reference cases are published on the validation page.Found in the sample
54of 600 rows
546 interviews kept after cleaning
26 speeders · 38 straight-liners · 4 duplicates
Survey data cleaning flags interviews that should not count: people who finished far faster than everyone else (speeders), gave the same answer to every item in a grid (straight-liners), submitted twice (duplicates), or left most of the questionnaire blank. This tool runs those checks on your file in the browser, shows why each row was flagged, and lets you download a cleaned file — free, with no account, and the file is never uploaded.
Clean a survey file online
Open a CSV, Excel, SPSS, Stata, SAS or JSON file (or the synthetic sample), pick the duration, grid and ID columns, and untick any check you do not want. Every rule is reported separately with its threshold, rows caught by more than one rule are counted, and nothing is removed until you download the cleaned file — the flagged download keeps every row and adds flag_* columns so you can review the decisions in a spreadsheet.
CSV, Excel, SPSS, Stata, SAS, R, Parquet, SQLite or JSON. The file is read in this browser tab and never uploaded.
The four checks
Speeders. The median interview length is a robust benchmark for how long a careful respondent needs. A row is flagged when its duration is below a chosen share of that median (33% by default) or below an absolute floor in seconds, whichever is larger. Rows with a blank or non-numeric duration are never flagged as speeders.
Straight-liners. Across the grid items you select, a row is flagged when every answered item has the same value and at least three items (configurable) were answered. The longest run of identical consecutive answers is shown as a softer secondary signal, because partial straight-lining is common in long grids.
Duplicates. A respondent ID that appears again is flagged on every repeat after the first. Exact copies of an entire row are also flagged, which catches double submissions even when the ID column was regenerated.
Incomplete interviews. A row is flagged when the share of blank cells across all columns is above the threshold (50% by default). Routed-out questions are legitimately blank, so raise the threshold for heavily filtered questionnaires.
The method
Formula
speeder if duration < max(p × median(duration), floor); straight-liner if answered ≥ m and SD(answers) = 0
where
- p
- share of the median below which an interview is too fast (default 0.33)
- floor
- optional absolute minimum duration in seconds
- m
- minimum answered grid items before a row can be judged (default 3)
- missing share
- blank cells ÷ all columns; flagged when it exceeds the threshold (default 0.50)
Worked example
The built-in synthetic tracker has 600 rows (596 generated respondents plus 4 exact duplicate submissions). Running the defaults with duration_sec, the eight brand_grid_* items and respondent_id:
- Median duration is 520 s, so the speeder threshold is 0.33 × 520 = 171.6 s; 26 rows are faster.
- 38 rows gave the same answer to all 8 brand-grid items.
- 4 rows repeat an earlier respondent ID (each is also an exact copy of its first row).
- 0 rows are more than 50% blank.
In total 54 rows are flagged; 14 of them are both speeders and straight-liners, which is why the rule counts (26 + 38 + 4 = 68) add up to more than 54. The cleaned file keeps 546 rows.
Using it well
When to use it
Use it when
- Use: Before tabulating an online panel or self-completion survey, where speeding and straight-lining are most common.
- Use: When a field partner delivers a file that may contain re-entries or test interviews.
- Use: To document a reproducible cleaning rule (thresholds and reasons per row) for a methods appendix or client report.
Not the right tool when
- Avoid: As an automatic deletion step — review the flagged rows, especially respondents caught by only one rule.
- Avoid: On grids where uniform answers are plausible (short grids, factual items, or all-positive brands); a straight line there may be honest.
- Avoid: On interviewer-administered or CATI data where duration reflects the interviewer, not the respondent.
How to interpret it
Rule of thumb
Treat a single flag as a reason to look, and two or more flags as strong evidence of careless responding. Report how many interviews were removed under each rule so readers can judge whether cleaning changed the sample profile.
After cleaning
Load the cleaned CSV into the Crosstabs workspace to build banner tables, or turn numeric codes into readable answers first with the Label Mapper. If your fieldwork file is an SPSS .sav, see how to open a .sav file without SPSS.
Questions
Frequently asked questions
Why compare duration to the median instead of the mean?
Interview lengths are right-skewed: a few people leave the survey open for hours, which drags the mean upward and would flag too many ordinary respondents. The median is not affected by those long tails, so a fixed share of it (commonly a third to a half) is a steadier speeding benchmark.
Is every straight-liner a careless respondent?
No. Someone who genuinely likes every brand can answer 5 to all of them. Straight-lining is more suspicious on long grids, on grids that mix positively and negatively worded items, and when it coincides with speeding. Use the longest identical run and the overlap count to decide, and consider a higher minimum of answered items for short grids.
Which duplicate is kept?
The first occurrence in file order is kept and every later repeat of the same respondent ID (or of an identical full row) is flagged. If your file is sorted differently from the order of submission, sort it by submission time before cleaning so the earliest response is the one kept.
Does the tool delete anything from my file?
No. Your original file is untouched and stays on your device. The cleaned download omits the rows flagged by the rules you left ticked; the flagged download keeps every row and adds flag_speeder, flag_straightliner, flag_duplicate, flag_incomplete and flag_reasons columns.
How are blank cells handled?
Blank cells are ignored by the speeder and straight-liner rules (a blank duration is never flagged, and only answered grid items are compared). They are what the incomplete rule counts, and they are compared exactly in the full-row duplicate check.
Sources and next steps
References & further reading
- [1]Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455.doi.org
- [2]Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19.doi.org
- [3]DeSimone, J. A., Harms, P. D., & DeSimone, A. J. (2015). Best practice recommendations for data screening. Journal of Organizational Behavior, 36(2), 171–181.doi.org
Try it on your own data — free, no signup
Upload a CSV, Excel, SPSS, Stata, SAS, R, Parquet or JSON file. Everything runs in your browser; your file never leaves your device.
Open the workspace →