data-analysis-standard

---
name: "data-analysis-standard"
description: "Structure a product data analysis, metric deep-dive, funnel analysis, or cohort study. Use when asked to analyse product metrics, investigate a drop in conversion, explain a data change to stakeholders, or find the root cause of a metric movement. Produces a structured analysis with question, root cause, confidence level, and recommended action."
license: "MIT"
---

# Data Analysis Standard Skill

Turn raw numbers into product decisions. Structure every analysis with a clear question, methodology, finding, and recommended action.

## Analysis Framework: The 4-Question Method

Every analysis starts here:
1. **What changed?** (describe the metric and its movement)
2. **Why did it change?** (root cause — segment, funnel step, cohort, channel)
3. **So what?** (business or product impact)
4. **Now what?** (recommended action with confidence level)

Never deliver data without answering all four. A chart with no narrative is not an analysis.

---

## Metric Triage Template

Use when a metric has moved unexpectedly:

```
METRIC: [Name]
MOVEMENT: [X% change over Y period]
BASELINE: [What was normal]

SEGMENTATION CHECK:
- By platform (iOS / Android / Web)?
- By user cohort (new / returning / power users)?
- By acquisition channel?
- By geography?
- By plan/tier?

ROOT CAUSE HYPOTHESIS:
1. [Most likely explanation] — Evidence: [data point]
2. [Alternative explanation] — Evidence: [data point]
3. [Ruling out] — Eliminated because: [reason]

CONCLUSION: [Single sentence answer to "why did this change?"]
CONFIDENCE: [High / Medium / Low] — based on [data available]
```

---

## Funnel Analysis Structure

| Stage | Metric | Current | Benchmark/Target | Drop-off % | Notes |
|---|---|---|---|---|---|
| [Top of funnel] | [Users] | [N] | [N] | — | |
| [Step 2] | [Users] | [N] | [N] | [X%] | |
| [Step 3] | [Users] | [N] | [N] | [X%] | |
| [Conversion] | [Users] | [N] | [N] | [X%] | |

**Biggest drop-off:** [Step X → Step Y] — Hypothesis: [reason]
**Recommended investigation:** [specific query or test]

---

## Cohort Analysis Guidelines

Always define:
- **Cohort definition:** [What groups users — signup week, first action, plan type]
- **Retention metric:** [What counts as retained — login, core action, revenue]
- **Retention window:** [D1, D7, D30, W4, M3, etc.]

Output a cohort retention table and annotate:
- Baseline retention for each cohort
- Cohorts that over/underperform and why (feature launch? campaign? seasonal?)
- Trend direction across cohorts (improving / declining / stable)

---

## Stakeholder Analysis Output Format

### [Analysis Title] — [Date]

**Question being answered:** [Specific question in plain English]
**Time period:** [Date range]
**Data source:** [Where data comes from]

**Finding:**
> [1–2 sentence plain-English summary of what the data shows]

**Key chart / table:** [Include or describe]

**Root cause:** [Best explanation with evidence]

**Confidence level:** [High / Medium / Low] — [reason]

**Recommended action:**
1. [Immediate action — owner, timeline]
2. [Investigation needed — what to check next]
3. [Monitoring — what metric to watch and at what cadence]

**What this analysis does NOT tell us:** [Important caveat — what data is missing or what can't be concluded]

---

## Required Inputs

Ask the user for these if not provided:
- **Metric or question** being investigated
- **Time period** (what changed, from when to when)
- **Data available** (which segments, sources, or queries you have access to)
- **Business context** (what decision this analysis informs)
- **Audience** (who will read this — exec / team / data team)

## Deeper Materials

This skill ships with support files — use them when they are available:

- **`references/analysis-integrity.md`** — Analysis Integrity: the Checks Between Query and Conclusion. Apply it while producing the output; it carries the calibration and judgment calls the method summary above compresses.
- **`templates/analysis-writeup.md`** — a fill-in version of the deliverable with the quality gates inline. Offer it when the user wants to work the document themselves rather than have it generated.

## Scoring Rubric (0–40)

Score any output of this skill before handing it over; 32+ is ship-quality.

| Dimension | 0 | 5 | 10 |
|---|---|---|---|
| Four-question completeness | Describes what changed and stops | Covers what/why but "so what / now what" are thin | All four answered with proportionate depth; the "now what" is decision-ready |
| Evidence behind the root cause | Root cause asserted from intuition | One supporting data point, alternatives unexamined | Root cause tested against at least one rival explanation, with the discriminating evidence shown |
| Uncertainty honesty | Reads as certain; no confidence statement | Confidence stated but not justified | Confidence level justified, and "what the data cannot tell us" names the real blind spots, not token ones |
| Actionability | Findings with no action | Action named but ownerless or dateless | Recommended action has an owner, a timeline, and a stated expected effect worth checking later |

## Quality Checks

- [ ] Analysis answers all 4 questions: what changed, why, so what, now what
- [ ] Root cause has evidence (not just hypothesis)
- [ ] Confidence level is stated and justified
- [ ] What the data cannot tell us is explicitly named
- [ ] Recommended action includes an owner and timeline

## Anti-Patterns

- [ ] Do not present correlations as causation — always state the distinction explicitly
- [ ] Do not report a metric movement without stating the time window and comparison baseline
- [ ] Do not skip the "so what" — raw observations without recommended actions are incomplete analysis
- [ ] Do not overstate confidence — label hypotheses clearly and note what data would be needed to confirm them
- [ ] Do not ignore segment breakdowns — aggregate metrics can mask opposing trends in sub-segments

## Guidelines

- Always state what the data *cannot* tell you — never oversell confidence
- Correlations are not causation — flag this every time
- If the user has no baseline, recommend establishing one before drawing conclusions
- Recommend the simplest chart for each finding: bar for comparison, line for trends, scatter for correlation, table for detailed breakdowns
- Always specify the time window — "conversion dropped" is meaningless without "from X to Y over Z period"