Skip to content
Open navigation
OpenOpen workspace

Stack Overflow Developer Survey 2026

Code testing and reliability tools: Want to work with next year by education

In the Professional degree (JD, MD, Ph.D, Ed.D, etc.) group, 50% selected “pytest”; in the Associate degree (A.A., A.S., etc.) group, 17% did. Of the 38 answers tested, “pytest” differs most by education, a small association: χ²(7, N = 6935) = 186.53, Holm-adjusted p < .001, V = .16. Respondents could select more than one answer, so the percentages add to more than 100%. These are unweighted respondents, not population estimates.

Published by crosstabs.com · Last updated ·
How results are checkedNamed statistical review pending. Selected automated reference cases are published on the validation page.

The table

The table

Question asked: Which tools for code testing, reliability, and maintainability have you done extensive development work in over the past year? Which do you want to work in over the next year? If you worked with the tool and want to continue to do so, please check both boxes in that row. Respondents could select more than one answer. Shown by education, as the percentage of each group's respondents who selected each answer, with the number of respondents beneath. A respondent is counted in every row they selected, so a column adds to more than 100%.

Code testing and reliability tools: Want to work with next year by education: percentage of each group's respondents who selected each answer, and respondent counts
Code testing and reliability tools: Want to work with next yearPrimary/elementary school (A)Secondary school (e.g. American high school, German Realschule or Gymnasium, etc.) (B)Some college/university study without earning a degree (C)Associate degree (A.A., A.S., etc.) (D)Bachelor's degree (B.A., B.S., B.Eng., etc.) (E)Master's degree (M.A., M.S., M.Eng., MBA, etc.) (F)Professional degree (JD, MD, Ph.D, Ed.D, etc.) (G)Other, please specify (H)All
ESLint36.5%G3136.0%FGh16036.6%EFGH31431.4%G6930.6%FG88225.5%G49813.2%5220.0%2029.2%2,026
Playwright16.5%1424.5%G10933.6%ABFG28826.8%G5930.3%FG87225.7%G50213.7%5424.0%2427.7%1,922
pytest25.9%2218.0%8024.7%21217.3%3821.9%63230.1%BDE58850.3%ABCDEFH19826.0%2625.9%1,796
Postman / Newman15.3%1319.8%G8820.7%G17727.3%G6024.2%FG69619.4%G37910.4%4130.0%G3021.4%1,484
Vitest23.5%2026.8%FG11925.1%FG21522.3%g4921.4%G61719.4%G37912.7%5017.0%1721.1%1,466
JUnit15.3%1311.9%5316.6%14216.4%3615.9%45818.7%B36614.5%5719.0%1916.5%1,144
SonarQube / SonarCloud7.1%611.9%5314.9%12815.9%3515.0%43117.8%b34812.2%4819.0%1915.4%1,068
Sentry11.8%1014.6%G6516.5%G14113.2%2914.3%G41314.2%G2787.4%2910.0%1014.1%975
Jest8.2%714.0%G6217.3%FG14817.7%G3915.3%FG44112.0%G2345.6%2211.0%1113.9%964
Datadog9.4%89.7%4312.1%G10413.2%G2912.3%G35410.1%1975.8%239.0%911.1%767
xUnit.net8.2%711.0%4912.4%g10614.5%G3211.3%3259.2%1806.6%2614.0%1410.7%739
Other, please specify20.0%Ef1713.7%6111.3%978.2%189.5%27310.0%19613.2%5220.0%EFd2010.6%734
Testcontainers9.4%88.8%3910.9%9310.0%2210.2%29510.6%2088.4%3313.0%1310.3%711
Selenium10.6%99.9%4411.8%10110.9%2410.8%3108.9%1748.6%3413.0%1310.2%709
CodeRabbit10.6%912.2%FGe548.2%7010.0%227.8%F2265.5%1084.8%1911.0%117.5%519
NUnit7.1%67.2%328.1%697.7%177.6%2206.5%1284.3%1716.0%FGe167.3%505
Cypress8.2%77.2%328.5%EF738.6%195.6%1604.9%964.6%1812.0%f126.0%417
BrowserStack9.4%88.3%FG376.9%597.7%175.0%1444.4%863.0%1215.0%EFG155.5%378
JaCoCo7.1%64.3%195.4%468.6%195.1%1465.7%1114.8%1910.0%105.4%376
Cucumber8.2%75.9%266.4%557.3%164.7%1345.3%1034.6%1812.0%E125.3%371
CodeQL18.8%BCDEFG167.7%F345.7%495.0%115.0%1443.9%766.1%2411.0%F115.3%365
Snyk Code5.9%55.9%265.0%435.9%134.9%1424.1%814.6%189.0%94.9%337
Codecov7.1%65.0%226.0%513.6%84.5%1303.9%777.1%289.0%94.8%331
k64.7%45.4%245.7%498.6%FG194.7%1364.0%782.8%118.0%84.7%329
RSpec5.9%53.6%165.0%434.5%105.0%1443.3%652.8%1110.0%FG104.4%304
Respondents854448572202,8801,9553941006,935

A letter beside a percentage names the columns it is significantly higher than (uppercase 95%, lowercase 90%; pairwise column tests, Bonferroni-corrected). 23,968 of 30,903 respondents have no answer to one of the two questions (not asked or not answered) and are left out. The table shows 25 of the 38 answers: the most selected. The CSV and JSON downloads have every answer.

Analysis

Results

Chi-square test of “pytest”

χ² = 186.53

df = 7, Holm-adjusted p < .001

Effect size

Cramér's V = 0.164

a small association

The strongest answer, APA-style: χ²(7, N = 6935) = 186.53, Holm-adjusted p < .001, V = .16. Each answer is tested on its own, as the respondents who selected it against those who did not, across the 8 groups (N = 6,935). The p-values are Holm-adjusted for the 38 answers tested; 23 of them differ by education at the 0.05 level.

Code testing and reliability tools: Want to work with next year by education: the test of each answer, selected against not selected, by education
AnswerSelectedχ² (df = 7)Adjusted pCramér's V
ESLint29.2%104.48< .0010.123
Playwright27.7%75.30< .0010.104
pytest25.9%186.53< .0010.164
Postman / Newman21.4%57.87< .0010.091
Vitest21.1%38.66< .0010.075
JUnit16.5%16.19.1560.048
SonarQube / SonarCloud15.4%22.00.0410.056
Sentry14.1%20.92.0580.055
Jest13.9%47.46< .0010.083
Datadog11.1%20.82.0580.055
xUnit.net10.7%20.22.0660.054
Other, please specify10.6%31.03.0010.067
Testcontainers10.3%4.10.7680.024
Selenium10.2%9.03.7520.036
CodeRabbit7.5%35.02< .0010.071
NUnit7.3%19.33.0790.053
Cypress6.0%27.13.0060.063
BrowserStack5.5%42.83< .0010.079
JaCoCo5.4%11.32.5000.040
Cucumber5.3%17.16.1310.050
CodeQL5.3%51.80< .0010.086
Snyk Code4.9%7.70.7520.033
Codecov4.8%16.32.1560.049
k64.7%17.74.1190.051
RSpec4.4%19.68.0760.053

How to read it

No single chi-square test covers this table. A respondent appears in every row they selected, so the rows are not separate groups of people, and a test of the whole table would count the same person several times. Each answer is tested on its own instead: did the share who selected it differ between the groups? Testing 38 answers gives 38 chances of a false alarm, so the p-values are Holm-adjusted to keep the chance of any false alarm in this table at 0.05.

For “pytest” the test rejects independence at the 0.05 level (Holm-adjusted p < .001): differences this large between the groups are unlikely if selecting it did not depend on education. Cramér's V of 0.164 is in the small range, which is real but modest: knowing someone's group tells you only a little about their answer.

With 6,935 respondents, even small differences reach statistical significance. Read the percentages and the effect size before the p-value.

Limits

  • The survey has no weights. These tables are unweighted counts of the people who answered, so they describe the respondents, not developers in general.
  • The test treats respondents as a simple random sample. They are people who chose to answer a survey Stack Overflow promoted, not a random sample of developers, so a result holds for these respondents and may not hold beyond them.
  • The adjustment covers the answers of this question by this group only, not the other tables in the library. The headline answer is the strongest of those tested, so its effect size is larger than a typical answer's.
  • A difference between groups is not evidence that group membership causes the answer.

Next steps

Work with this table

Open the test of “pytest” in the calculator

The link fills in the respondents who selected this answer and those who did not, by group. The whole table does not open there: its rows overlap, and the calculator's test needs rows that are separate people.

Open this question in the full file

The workspace opens all 30,903 respondents in your browser with this question selected. The survey has no weights, so the table there is unweighted too. Change the group, add a filter or export from there.

Open in the workspace →

Run this on your own data — free, no signup

Upload a CSV, Excel, SPSS, Stata, SAS, R, Parquet or JSON file. Everything runs in your browser; your file never leaves your device.

Open the workspace →

The same question by other groups

Other questions that differ by education

Source

Stack Overflow. 2026 Developer Survey, https://survey.stackoverflow.co/2026, licensed under the ODbL 1.0. Tabulated by crosstabs.com. Variables CodeTestWantToWorkWith (Code testing and reliability tools: Want to work with next year) and Education. Stack Overflow licenses the survey data under the Open Database License (ODbL) 1.0 and its contents under the Database Contents License (DbCL) 1.0: credit Stack Overflow, keep the licence notice, and share any database made from the data under the ODbL. These tables and their downloads are offered on those terms.

  1. [1]Stack Overflow: 2026 Developer Survey public results (survey_results_public.csv)survey.stackoverflow.co
  2. [2]Stack Overflow: Open Database License (ODbL) 1.0opendatacommons.org