Verified against ChatGPT · 2026-08-08
Code hundreds of open-text survey responses into themes without losing the outlier that matters
Turns a pile of raw open-ended survey answers into a ranked theme list with verbatim quotes as evidence, while explicitly holding back rare-but-important comments from getting averaged away.
The prompt
Ready to copy — highlighted parts are example details you can swap.
You are coding open-text survey responses into themes the way a research analyst would for a stakeholder who will act on the results, not just skimming for a vibe. SURVEY QUESTION ASKED "What, if anything, made it hard to complete your order today?" RAW RESPONSES A list of 140 free-text responses pasted directly from the survey export RESPONDENT COUNT AND CONTEXT 140 responses from customers who completed checkout in the last 30 days WHAT THE RESULTS WILL BE USED FOR Prioritizing the next sprint's checkout fixes with the product team CODING RULES Read through the responses and derive themes from what's actually written — do not start from a preset list of themes you'd expect to see and force-fit responses into it. Each theme needs: a short label, the count and percentage of respondents who raised it, and two to three verbatim quotes (unedited, exact wording) as evidence, so a reader can sanity-check the coding against the actual words used. Rank themes by frequency, but keep a separate section for any response that doesn't fit the top themes and represents a distinct, potentially important signal on its own — a single respondent describing a serious usability blocker or a safety concern should never be discarded just because it's not common; flag it separately with a note on why it might matter disproportionately to its frequency. If a response touches multiple themes, count it under all of them and say so, rather than forcing every response into exactly one bucket. State your confidence in the coding for ambiguous or sarcastic responses where the theme assignment is a judgment call, rather than presenting every categorization with equal certainty. WHAT NOT TO DO Do not paraphrase or clean up quotes when presenting them as evidence — use the respondent's exact words, typos and all, or the evidence loses its verifying power. Do not silently merge a small theme into a larger adjacent one just to make the list tidier. OUTPUT FORMAT 1. Ranked theme table: theme | count | % of respondents | 2-3 verbatim quotes. 2. Notable outlier responses that don't fit the top themes, with a one-line note on why each might matter. 3. Any response coded under multiple themes, listed once with its themes noted. 4. Two-sentence summary suited to Prioritizing the next sprint's checkout fixes with the product team.
Customize
Optional — swap in your own details for the highlighted parts above.
Why this works
A model summarizing open-text survey data by default tends to produce a smooth, averaged narrative that reads well but quietly discards the low-frequency signal, because frequent phrases dominate what gets synthesized into prose — explicitly requiring a separate outlier section reverses that default by giving rare responses a protected place in the output regardless of how the frequency ranking shakes out, which matters because in product and safety-adjacent feedback the single respondent describing a blocking bug is often more actionable than the twenty saying "good overall." Requiring verbatim, unedited quotes as evidence for every theme forces the model to ground its categorization in something checkable rather than asserting a theme exists on its own authority — a reader can immediately verify "price complaints, 22%" against the actual three quotes shown, which a paraphrased summary would make impossible to audit. Instructing the model to derive themes bottom-up from the actual text, instead of starting from an expected list, addresses a specific failure mode where a model primed with domain expectations (checkout surveys "usually" surface price, speed, and trust issues) pattern-matches responses into those familiar buckets even when the actual text is describing something different, silently forcing new signal into old categories. Allowing multi-theme tagging rather than a forced single bucket per response reflects how people actually write open-text feedback — a single sentence complaining about both price and a confusing checkout flow is two data points, and forcing it into one bucket understates both themes' true frequency.
What you get back
Theme: Confusing shipping cost timing — 31 responses (22%) — "i didnt see the shipping fee until the very last step and almost left"; "shipping cost popped up out of nowhere". Outlier flag: One response described being unable to complete checkout on a screen reader — only 1 mention, but represents a distinct accessibility blocker worth flagging regardless of frequency.
Verified against
ChatGPT GPT-5.1 · 2026-08-08
Changelog
- 2026-08-08 — Initial publish, verified against ChatGPT GPT-5.1.
Need this built into your business?
If a prompt isn't enough — what Scult builds, built and maintained for you — that's Scult's day job.
EXPLORE WHAT SCULT BUILDS
