Survey data quality
How to clean survey data: speeders, straight-liners and duplicates
To clean survey data, set the rules before you look at results, then flag: speeders (interviews shorter than a fraction of the median duration, often a third), straight-liners (the same answer to every item in a rating grid), duplicates (a repeated respondent ID or an exact copy of another row) and incomplete interviews (more than a set share of questions blank). Count each rule, check the overlap, remove the flagged interviews from the analysis file but keep them in an audit file, and weight only after cleaning.
How results are checked
Named statistical review pending. Selected automated reference cases are published on the validation page.1. Write the rules down first
Cleaning rules decided after seeing the results invite cherry-picking. Agree the duration column and threshold, which grids count for straight-lining, the duplicate key and the missing-data limit before fieldwork ends, and apply the same rules to every wave of a tracker.
2. Speeders: compare against the median
Interview length is skewed: a few people leave the survey open for an hour. Compare each duration with the median rather than the mean, and flag those under a fraction of it. A third of the median is a common starting point; for very short questionnaires an absolute floor in seconds can work better. Check the routing first, because a respondent who was screened past most of the questionnaire is legitimately fast.
3. Straight-liners: one answer down a whole grid
A straight-liner gives the same answer to every item of a rating grid. It is a strong signal on long grids that mix positive and negative statements, and a weak one on short grids where identical answers can be sincere. Look at the grids where straight-lining would be implausible and require a minimum number of answered items before judging a row.
4. Duplicates and incomplete interviews
Flag later repeats of a respondent ID and rows that are exact copies of another row, keeping the first occurrence. Panel and IP-based fraud checks catch more, but start with these two. For incomplete interviews, flag rows where the share of blank answers exceeds a limit you set. Routed-out questions are blank by design, so with heavy routing set the limit high enough that legitimate skips do not trip it.
The method
Formula
speeder: duration < max(k × median duration, floor) · straight-liner: all answered grid items equal, answered ≥ m
duplicate: ID seen before, or every column identical to an earlier row · incomplete: blank share > limit
where
- k
- fraction of the median, e.g. 0.33; the floor is an optional absolute minimum in seconds
- m
- minimum answered grid items before a row can be judged (3 by default in the tool)
- limit
- maximum share of blank cells across all columns, e.g. 50%
- kept
- rows that trip no rule; flagged rows are listed with their reasons, not silently deleted
Worked example
Sample data: the synthetic snack-brand tracker that the survey data cleaner loads with “Use sample data” (600 generated interviews, not real people), checked with the tool's four core rules at their default settings.
- Speeders. The median interview lasted 520 seconds. Flagging anything under 33% of the median sets the threshold at 171.6 seconds, which catches 26 interviews (4.3%).
- Straight-liners. On the 8-item brand rating grid, 38 respondents (6.3%) gave the same answer to every item they answered (at least 3 answered items required).
- Duplicates. 4 rows repeat an earlier respondent ID (4) and are exact copies of an earlier row (4); the first copy is kept.
- Incomplete interviews. No row is more than 50% blank, so this rule flags 0.
Adding the flags gives more than the number of people flagged, because rules overlap: 14 respondents trip two or more rules. In total 54 interviews (9.0%) are flagged and 546 remain for analysis. The report to stakeholders should state exactly that: the rules, the count per rule, the overlap and the final base.
Using it well
When to use it
Use it when
- Use: Every online or panel survey, before tabulation, weighting and reporting.
- Use: Trackers, where the same written rules must be applied identically to every wave.
- Use: Before key driver, segmentation or pricing analysis, which careless answers distort more than simple tables.
Not the right tool when
- Avoid: Do not tune thresholds until the results look right; fix the rules first.
- Avoid: Do not judge straight-lining on grids of two or three items or on grids where identical answers are plausible.
- Avoid: Do not remove respondents for holding unpopular or extreme opinions; cleaning targets effort and validity, not views.
How to interpret it
Rule of thumb
A flag is evidence, not proof. Remove interviews that fail a strong rule or several weak ones, keep the audit trail, and report the before and after base so readers can judge how much the cleaning matters.
Next step: check your own file
The free survey data cleaner runs these four checks on a CSV, Excel or SPSS file in your browser, plus optional open-end quality, near-duplicate, IP or device identifier and attention-check rules. It lists every flagged row with its reasons, shows the overlap between rules, and exports a cleaned file and a flagged-rows file. Nothing is uploaded and no account is needed.
Questions
Frequently asked questions
What is a speeder in a survey?
A respondent who finished much faster than anyone could read and answer the questions. The common rule flags interviews shorter than a fixed fraction of the median duration, often a third or 40%; some teams add an absolute floor in seconds. Use the median, not the mean, because a few very long interviews pull the mean up.
Is every straight-liner a bad respondent?
No. Someone who genuinely rates every brand the same can give identical answers, especially on a short grid. Straight-lining is strong evidence of low effort on long grids with mixed or reversed items, and weak evidence on short grids. Many teams remove a respondent only when straight-lining coincides with another flag, such as speeding.
Should I delete flagged respondents or keep them?
Decide the rules before you look at the results, apply them the same way to every wave, and report how many interviews each rule removed. Keep the flagged rows in an audit file rather than deleting them, so the decision can be reviewed or reversed.
Does cleaning change the weights?
Yes. Remove poor-quality interviews first, then weight the cleaned sample. Weighting before cleaning would give the removed respondents' share to the wrong people, and it can leave the cleaned sample off its targets.
How many respondents does a typical cleaning step remove?
It depends on the source, the length of the questionnaire and the rules, so there is no standard rate. What matters is that the rules are written down, applied consistently, and the removed count is reported with the results.
Sources and next steps
References & further reading
- [1]Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455.
- [2]Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19.
Try it on your own data — free, no signup
Upload a CSV, Excel, SPSS, Stata, SAS, R, Parquet or JSON file. Everything runs in your browser; your file never leaves your device.
Open the workspace →