Verified against ChatGPT · 2026-08-11
Score a support ticket transcript against your actual QA rubric instead of a generic politeness checklist
Runs a real ticket transcript through your team's specific QA criteria and produces a scorecard that credits what actually helped the customer, not just tone, so QA feedback targets outcomes rather than surface politeness.
The prompt
Ready to copy — highlighted parts are example details you can swap.
You are performing a QA review of a support interaction against our actual rubric, not a generic customer-service politeness checklist. TRANSCRIPT OR TICKET TEXT Customer: 'My invoice shows the wrong tax rate.' Agent: 'I'm so sorry for the trouble! Let me look into that for you.' [12 minutes later] Agent: 'It looks like your billing address on file is outdated — I've corrected it and the next invoice will reflect the right rate.' QA RUBRIC CRITERIA 1) Accurately diagnosed root cause before acting, 2) Resolved within one contact, 3) Set correct expectations about when the fix takes effect, 4) Used empathetic but not over-apologetic language. AGENT EXPERIENCE LEVEL 3 weeks into the role, first month off training wheels. KNOWN SOFT SPOTS This agent has previously skipped confirming the fix's effective date with customers, leading to repeat tickets asking 'did it work yet?' CALIBRATION NOTE This is a formal monthly QA review that feeds into the agent's performance record, not an informal spot-check. STEP 1 — SCORE EACH CRITERION SEPARATELY Go through each item in the given rubric one at a time and score it based only on what's actually in the transcript — quote the specific line that earns or loses points for each criterion, rather than giving an overall impression score. A criterion about resolving the issue efficiently should be scored on whether the issue was actually resolved and how many back-and-forths it took, not on how pleasant the agent sounded while doing it — tone and effectiveness are different rubric items and should never bleed into each other's score. STEP 2 — WEIGH EXPERIENCE LEVEL WITHOUT LOWERING THE BAR Use the agent's experience level to calibrate what kind of feedback is useful (a newer agent needs more foundational coaching, an experienced one needs more nuanced feedback) but do not adjust the actual score based on experience — a wrong answer given confidently by a tenured agent is still a wrong answer, and grading it more leniently would hide a real problem. STEP 3 — CHECK KNOWN SOFT SPOTS DIRECTLY If a known soft spot was given (a pattern this agent or team has struggled with before), check the transcript against it specifically and state directly whether it shows up here again, rather than only reporting on the rubric criteria in isolation. WHAT NOT TO DO Do not round every score toward the middle to seem balanced — if a criterion was clearly met or clearly missed, say so plainly. Do not manufacture a compliment for every criticism just to soften the review; state what worked because it worked, not to cushion what didn't. OUTPUT FORMAT 1. Score per rubric criterion, each with the quoted transcript line as evidence. 2. Overall assessment in two sentences. 3. Known-soft-spot check, if applicable. 4. The single most useful piece of feedback to give this agent, prioritized over the others.
Customize
Optional — swap in your own details for the highlighted parts above.
Why this works
Requiring a quoted transcript line as evidence for every criterion score is the mechanism that prevents the most common failure of AI-generated QA reviews: a model asked to score a transcript against a rubric will readily produce plausible-sounding scores and justifications that don't actually trace back to anything specific said, because a generically worded justification is easier to generate than one anchored to an exact quote — forcing the citation makes the score falsifiable by whoever reads the review, and it also structurally prevents tone and effectiveness from bleeding into each other, since a real quote about whether the issue got resolved can't double as evidence for a tone score. The explicit instruction to calibrate feedback style by experience level but never the actual score addresses a specific and consequential drift: GPT-5.1, when told an agent is new, will tend to soften not just the coaching tone but the substance of the score itself, which quietly launders a real performance gap into a "they're still learning" framing that would look bad if that gap persists into month six with the score record showing steady high marks. The instruction against manufacturing a compliment for every criticism counters a reflexive pattern in AI feedback generation — a felt need to balance every negative with a positive regardless of whether one is actually earned — which dilutes the review and makes it harder for whoever reads it to identify what actually needs to change versus what was included for tone-softening alone.
What you get back
Criterion 1 (root cause before acting): partially met — agent found the correct cause (outdated billing address) but the transcript shows a 12-minute gap with no interim update to the customer; scored as adequate, not strong. Criterion 3 (expectations about fix timing): not met — agent said 'the next invoice will reflect the right rate' without stating when the next invoice is issued, which matches the known soft spot of not confirming effective dates; likely driver of repeat tickets. Most useful feedback: build a habit of stating a concrete date whenever telling a customer a fix takes effect 'next cycle', since this is the second review cycle this specific gap has shown up.
Verified against
ChatGPT GPT-5.1 · 2026-08-11
Changelog
- 2026-08-11 — Initial publish, verified against ChatGPT GPT-5.1.
Need this built into your business?
If a prompt isn't enough — what Scult builds, built and maintained for you — that's Scult's day job.
EXPLORE WHAT SCULT BUILDS
