Data & BI

Verified against ChatGPT · 2026-08-14

Audit a dashboard for the specific mistakes that make numbers look right until someone checks

Runs a structured QA pass over a dashboard's metric definitions, filters, and calculations, targeting the exact class of silent errors that pass a casual glance but break under a real audit.

ChatGPT (GPT-5.1)4 fillable variables

The prompt

Ready to copy — highlighted parts are example details you can swap.

Run a QA audit on the dashboard described below the way an experienced analyst would before it ships to a wide audience — looking specifically for the errors that look fine at a glance and only surface when someone checks the math.

DASHBOARD DESCRIPTION
A revenue-by-region dashboard joining an orders table to a shipments table, with charts for total revenue, average order value, and on-time delivery rate

METRIC DEFINITIONS AS BUILT
Total revenue = SUM(orders.amount) after joining to shipments on order_id; on-time rate = COUNT(delivered_on_time)/COUNT(all shipment rows)

KNOWN DATA SOURCES FEEDING IT
Orders table logged in UTC from the checkout system; shipments table logged in local warehouse time from a separate logistics vendor system

AUDIENCE AND STAKES
Regional VPs use this weekly to evaluate performance bonuses tied to on-time delivery rate

Check each of the following categories and report a finding or an explicit "checked, no issue found" for each — do not skip a category just because nothing obvious jumped out.

1. DOUBLE-COUNTING — could any metric be summing across a join that fans out rows, inflating a total (e.g., joining orders to a one-to-many items table and then summing an order-level field, multiplying it by item count)?
2. FILTER LEAKAGE — does every chart on the dashboard apply the same date range and segment filters consistently, or could one chart be silently using a different implicit window than the one displayed in the filter control, producing numbers that don't actually correspond to what the filter appears to say?
3. DENOMINATOR MISMATCH — for any rate or percentage metric, does the numerator and denominator come from genuinely the same population and time window, or could a rate be computed from a numerator filtered one way and a denominator filtered another, producing a percentage that isn't actually measuring what it claims to?
4. TIMEZONE/BOUNDARY DRIFT — for any metric bucketed by day or month, is the timezone used for bucketing stated and consistent across all data sources feeding it, given that Orders table logged in UTC from the checkout system; shipments table logged in local warehouse time from a separate logistics vendor system may log timestamps in different timezones?
5. SURVIVORSHIP OR SAMPLE BIAS — does any metric implicitly exclude a group in a way that biases the result (e.g., an "average session length" that only includes completed sessions, silently excluding the abandoned ones that would pull the average down)?

WHAT NOT TO DO
Do not report only the issues found and silently skip the categories where nothing was wrong — an audit needs to show its work on every category checked, not just the ones with findings. Do not treat a suspected issue as confirmed without stating what would need to be checked in the actual data to confirm it.

OUTPUT FORMAT
For each of the 5 categories: verdict (issue found / no issue found / can't determine from what's given) and, for any issue found, exactly what to check in the real data to confirm it and what the fix would look like.
Followed by: overall risk level given Regional VPs use this weekly to evaluate performance bonuses tied to on-time delivery rate, and whether this dashboard should ship as-is, ship with a caveat noted, or be held pending fixes.

Customize

Optional — swap in your own details for the highlighted parts above.

Why this works

The dangerous class of dashboard bug is never the one that produces an obviously wrong number — those get caught immediately — it's the one that produces a plausible-looking number that's subtly wrong for a structural reason like a join fan-out or a mismatched denominator, and a model asked generically to "check this dashboard for errors" has no reliable way to surface those without being pointed at the specific mechanical patterns that cause them, which is why this prompt names five concrete failure categories rather than leaving the audit open-ended. Requiring an explicit verdict for every category, including "no issue found," rather than only reporting problems, matters because a report that only lists findings gives no way to distinguish "this category was checked and is clean" from "this category was never actually examined" — and for an audit specifically, that distinction is the entire deliverable, since the value of a QA pass is knowing what was actually verified, not just what happened to look wrong. The join-fan-out and denominator-mismatch checks target the two most common and most silent classes of dashboard error in practice — a one-to-many join silently multiplying an order-level total by the number of line items, or a percentage computed from a numerator and denominator drawn from subtly different filtered populations — both of which produce numbers that pass every casual sanity check because they're the right order of magnitude and move in believable directions, and neither is visible without someone specifically checking the join and filter logic, not just eyeballing the chart. The instruction to state what to check in the real data to confirm any suspected issue, rather than asserting it as confirmed, respects the actual limit of what's verifiable from a described metric definition versus an actual query result — the model can spot the structural risk pattern from the description given, but confirming it as an actual bug requires running the real numbers, and conflating those two would overstate the audit's certainty.

What you get back

1. Double-counting: Issue found — joining orders to shipments (likely one-to-many if an order ships in multiple packages) before summing orders.amount would inflate total revenue. Check: compare SUM(orders.amount) pre-join vs. post-join for a sample of multi-shipment orders. Fix: sum revenue from the orders table directly, join to shipments only for delivery-rate metrics. 3. Denominator mismatch: Issue found — on-time rate's denominator (all shipment rows) doesn't specify the same date filter as revenue charts. Check: confirm the date range filter applies identically to both the numerator and denominator query. 4. Timezone drift: Issue found — orders logged in UTC, shipments in local warehouse time; daily buckets could misalign by hours near midnight. Fix: standardize both to one timezone before bucketing. Overall risk: high, given this feeds bonus calculations — recommend holding pending fixes to items 1, 3, and 4.

Verified against

ChatGPT GPT-5.1 · 2026-08-14

Changelog

  • 2026-08-14 Initial publish, verified against ChatGPT GPT-5.1.

Need this built into your business?

If a prompt isn't enough — what Scult builds, built and maintained for you — that's Scult's day job.

EXPLORE WHAT SCULT BUILDS
All Data & BI prompts

Check your AI visibility

One URL in, a 0–100 score and the exact fixes out.

RUN THE CHECK

Browse all the tools

15 tools across six categories
13 of them never send your data anywhere

Free · No signup · No trial clock

SEE THE DIRECTORY