Verified against ChatGPT · 2026-08-11
Design a single ad experiment with a real kill criterion so it stops being "let's just try both and see"
Structures one paid-ad experiment with a stated hypothesis, minimum sample size, and an explicit kill criterion decided in advance, instead of an open-ended A/B test that runs until someone eyeballs the numbers and calls it.
The prompt
Ready to copy — highlighted parts are example details you can swap.
Design one ad experiment for the hypothesis below. This is not a general "how to A/B test ads" explainer — it's the specific test plan for this one experiment, with a decision rule set in advance so nobody has to eyeball the numbers later and argue about whether it's done. HYPOTHESIS BEING TESTED We think showing a customer's actual name in the headline via dynamic text will improve click-through. WHAT VARIABLE CHANGES BETWEEN VARIANTS Headline personalization (name insertion) vs. static generic headline — everything else in the ad identical. CURRENT BASELINE METRIC Current CTR on the static headline is 1.2%. DAILY TRAFFIC OR SPEND AVAILABLE FOR THE TEST Around 4,000 impressions per day split across both variants. STATE THE HYPOTHESIS AS A PREDICTION, NOT A QUESTION Rewrite hypothesis, if needed, as a specific falsifiable prediction — "variant B will improve conversion rate by at least X relative to baseline_metric" — rather than an open question like "which one works better," since an open question has no built-in way to ever end the test. ISOLATE THE VARIABLE Confirm that variable_changed is the only meaningful difference between variants; if the described variants actually differ in more than one dimension at once (e.g., both headline and image changed together), say so explicitly and note that a positive result won't tell you which change caused it — recommend splitting into two sequential tests instead if that's the case. SET THE KILL CRITERION BEFORE THE TEST STARTS Using available_traffic_or_spend, estimate roughly how long it would take to reach a sample size large enough to trust a result at this baseline conversion rate, and state two decision rules set now, before any data comes in: the threshold at which the test is declared a clear win worth rolling out, and — just as important — the threshold or time limit at which it gets killed for being inconclusive or clearly not working, so a mediocre result doesn't get to run forever on hope. Name honestly if available_traffic_or_spend is too low to reach a trustworthy sample size in a reasonable window, rather than proceeding as if any test duration is fine. OUTPUT FORMAT 1. Falsifiable hypothesis statement. 2. Variable isolation check — confirmed single-variable, or a flag with a recommended split. 3. Rough sample-size/time estimate given available traffic or spend, with an honest call on whether it's sufficient. 4. Win threshold and kill threshold, both set in advance.
Customize
Optional — swap in your own details for the highlighted parts above.
Why this works
The most common way ad experiments fail isn't a bad hypothesis, it's the absence of a decision rule set before the data starts arriving — without one, a team watching a test in progress tends to keep it running past the point of usefulness whenever the early numbers look mildly promising, and calls it inconclusive and quietly drops it whenever the numbers look mildly disappointing, which means the actual stopping decision is being made by hope rather than by a rule, and GPT-5.1 asked generally "how do I A/B test this" will explain the concept of statistical significance without ever forcing a team to commit to a specific win and kill threshold in advance. Rewriting an open question into a falsifiable numeric prediction matters because "which one works better" genuinely has no defined endpoint — there is always a plausible argument for letting it run one more day — while "variant B beats baseline by at least X" gives the test a condition that can actually be met or missed. The variable-isolation check catches a specific and common experiment design flaw where two creative changes are bundled into one test because it's more convenient to build one variant than two, producing a result that, even if positive, can't be attributed to either change individually — a flaw invisible unless something is explicitly checking for it rather than accepting the described variants at face value. Estimating sample size against the actual available traffic and being willing to say the answer is "this test can't reach a trustworthy result in a reasonable window" is the honesty check most self-directed ad testing skips entirely — teams routinely launch tests against traffic volumes too small to ever produce a statistically meaningful answer, and continuing anyway just produces a confident-sounding conclusion built on noise.
What you get back
Falsifiable hypothesis: dynamic-name-insertion headline will improve CTR to at least 1.5% (a relative lift over the 1.2% baseline), not merely "perform better." Variable isolation: confirmed single-variable — only the headline personalization differs, rest of the ad is identical. Sample estimate: at ~2,000 impressions/day per variant, reaching a trustworthy sample at this baseline CTR will take roughly 2-3 weeks; this is workable but tight, plan for the full window rather than checking early. Win threshold: roll out personalization if CTR clears 1.5% with the full sample reached. Kill threshold: stop the test at day 21 regardless of result if CTR hasn't cleared 1.35%, since a smaller lift than that isn't worth the added complexity of dynamic personalization at scale.
Verified against
ChatGPT GPT-5.1 · 2026-08-11
Changelog
- 2026-08-11 — Initial publish, verified against ChatGPT GPT-5.1.
Need this built into your business?
If a prompt isn't enough — Google Ads management, built and maintained for you — that's Scult's day job.
EXPLORE GOOGLE ADS MANAGEMENT
