customize

---
name: customize
description: Adapt this C/C++ ASAN vulnerability pipeline to a different vulnerability class, target shape, language, or detection mechanism. Use when the user wants to port, migrate, retarget, customize, or fork the pipeline for something other than C/C++ memory-safety bugs — web apps, smart contracts, deserialization, ML systems, or any other domain.
license: Apache-2.0
---

# Customize the vuln-pipeline

This pipeline ships as an opinionated C/C++ + AddressSanitizer demo. Its real shape is more general: **an agent crafts an input, runs a target in a sandbox, a detector fires, a second agent verifies, a third agent analyzes exploitability.** Every noun in that sentence can be swapped. Your job is to interview the user, figure out which nouns they want to swap, and rewrite the relevant files.

The existing C/C++ code is the worked example. You don't need a playbook for each domain — read what's there, understand what's generic vs. ASAN-specific, and adapt.

## STEP 1 — Read the pipeline (do this BEFORE asking anything)

Skim these files so your questions are grounded:

- `README.md` — pipeline overview (recon → find → grade → judge → report)
- `harness/cli.py` — orchestration; shows how stages wire together and what lands on disk
- `harness/find.py`, `harness/grade.py`, `harness/report.py` — the three container-agent loops; mostly generic plumbing
- `harness/prompts/find_prompt.py`, `harness/prompts/grade_prompt.py` — **the C/C++-specific parts**; bug taxonomy, quality tiers, grading rubric
- `harness/prompts/report_prompt.py`, `harness/prompts/report_grader_prompt.py` — **also C/C++-specific**; exploitability sections (primitive, heap layout, escalation path) and the rubric that scores them
- `harness/prompts/judge_prompt.py` — triage prompt; keys on ASAN excerpts and memory-safety crash classes
- `harness/prompts/system_prompt.py` — authorization block; hard-codes "C/C++ target" and "sanitizer output"
- `harness/asan.py` — stack-trace parser for dedup/judge signatures; ASAN-specific regex
- `harness/artifacts.py` — `CrashArtifact`, `GraderVerdict`, `JudgeVerdict`, `ReportVerdict` data contracts
- `harness/config.py`, `targets/drlibs/config.yaml` — target config schema
- `targets/README.md` — how a target directory is structured (Dockerfile + config.yaml + entry wrapper)

You don't need `agent.py`, `docker_ops.py`, `recon.py`, `judge.py`, or `novelty.py` in detail — they're generic plumbing (judge/novelty domain-specificity lives in the prompts and the asan parser, not the flow). One caution: part of `harness/` doubles as shared infrastructure — `dnr_harness/` (the detection-and-response pipeline) imports `agent.py`, `sandbox.py`, `docker_ops.py`, `agent_image.py`'s `build`/`ensure_base`, `auth.py`, `cli.py`'s `resolve_target_dir`/`terminate_subprocesses`, and `prompts/system_prompt.py`'s `load_engagement_context`. A domain port should touch only the domain-specific files (the prompts, `asan.py`, `artifacts.py`, target Dockerfiles); treat any edit to the shared files as a change to both pipelines.

## STEP 2 — Interview the user

Use **AskUserQuestion** to gather requirements. Start with broad context, then narrow to technical specifics based on what they say.

### Round 1 — Context (always ask these first, together)

Two open-ended questions to understand who you're talking to and what they're after. Expect most answers to come via **Other** as free text — the options are there to prompt thinking, not to constrain.

**Question A — Operating context**
- header: `Context`
- question: `What's your operating environment? Who will run this pipeline and why?`
- options: a few archetypes as inspiration — e.g. "Pentesting firm — client engagements, need reportable findings", "Internal appsec team — scan our own services in CI", "Smart-contract auditor — pre-deployment reviews", "Security researcher — hunting novel bug classes". These tell you what output format, grading rigor, and workflow integration matter.

**Question B — Goal**
- header: `Goal`
- question: `Describe in your own words what you want this pipeline to find. What kind of target, what kind of bugs?`
- options: 2–3 concrete examples (e.g. "Web vulnerabilities like SQLi/XSS in HTTP services", "Reentrancy and access-control bugs in Solidity contracts", "Deserialization RCE in Java microservices").

The context answer calibrates your follow-ups: a pentesting firm probably cares about CVSS scoring and SARIF output; a researcher may want differential testing and novel detection signals; an internal team likely wants CI integration and low false-positive rates.

### Round 2 — Technical follow-ups (adaptive — derive from round-1 answers)

Parse their round-1 answers against the **axes of variation** below. For each axis left ambiguous, ask a targeted follow-up. Batch up to 4 questions per AskUserQuestion call. Common follow-ups:

- **Detection signal** — "How will the pipeline know it found something?" (crash, exception, canary file appears, DNS callback, differential mismatch, invariant violation)
- **PoC shape** — "What does a proof-of-concept look like?" (single file, HTTP request sequence, transaction list, test-pipeline code)
- **Isolation** — "Where does the target run?" (Docker, VM, testnet, remote sandbox, or no execution — static-only)
- **Grading criteria** — "What makes a finding high-quality vs. low-quality in this domain?"
- **Exploitability analysis** — "What sections should a report contain?" The C/C++ report has primitive · reachability · heap layout · escalation path · constraints. A web-vuln report might want injection vector · auth bypass · data exposure · chaining potential. Ask what they need, or whether they want the report stage at all.
- **Novelty/upstream check** — "Should the pipeline check if a finding is already fixed upstream?" The C/C++ version shallow-clones the target's GitHub and checks `git log <commit>..HEAD -- <crash_file>`. Only applies if targets have a canonical upstream and a sensible "crashing file" to key on — many domains won't.
- **Scope** — "Replace the C/C++ support entirely, or keep it alongside the new domain via a profile system?"

Keep going until you can fill in every row of the architecture map in STEP 3. If an answer is vague, ask a narrower follow-up rather than guessing.

## Background — axes of variation (context for formulating follow-ups)

These are the dimensions along which customers might want to deviate from the C/C++ demo. Use this list to spot gaps in the user's description and generate follow-up questions — do **not** present it as a menu.

**Vulnerability class:** memory safety · web/API (SQLi, XSS, SSRF, XXE, path traversal, IDOR) · deserialization RCE · logic/race (TOCTOU, privilege escalation) · crypto (weak RNG, timing, nonce reuse) · DoS (ReDoS, hash flooding) · smart contracts (reentrancy, access control, front-running) · ML/AI (prompt injection, jailbreaks, data extraction) · protocol parsing

**Target shape:** CLI binary + file · HTTP service · library via test harness · network daemon · smart contract · browser extension · mobile app

**Detection mechanism:** crash/abort · uncaught exception · sanitizer hooks (Jazzer/Atheris) · outcome-based (canary file, DNS callback, shell spawn) · differential testing · invariant violation · taint tracking

**Input modality:** single file · HTTP request chain · multi-file archive · stdin stream · args + env + config combo · transaction sequence

**Isolation boundary:** Docker container · full VM · remote sandbox · local testnet · none (static analysis)

**Dedup signature:** (crash_type, top_frame) · (vuln_type, endpoint, param) · (function, state_transition) · (component, precondition)

**Report structure:** primitive/heap/escalation (memory safety) · vector/auth/exposure (web) · invariant/path/impact (contracts) · or drop the report stage entirely if find+grade is the deliverable

**Output format:** result.json + poc.bin · SARIF · Nuclei template · prose report

**Patch verification signal:** ASAN-clean exit · uncaught-exception-free · sanitizer hook silent (Jazzer/Atheris) · canary file untouched · invariant assertion holds · differential output matches reference. This is what `_t1_passes()` in `patch_grade.py` encodes — "the bug is gone" for the new domain.

## Background — architecture map (what changes vs. what stays)

| File | C/C++-specific? | What it does |
|---|---|---|
| `harness/prompts/find_prompt.py` | **Yes — rewrite** | Bug taxonomy, quality tiers, ASAN output format, exit-code examples |
| `harness/prompts/grade_prompt.py` | **Yes — rewrite** | 5-criterion rubric assumes ASAN traces and Unix signal exit codes |
| `harness/prompts/report_prompt.py` | **Yes — rewrite** | Exploitability sections: primitive, heap layout, escalation path — memory-safety-specific |
| `harness/prompts/report_grader_prompt.py` | **Yes — rewrite** | Scores the above sections; rubric is tied to the section set |
| `harness/prompts/judge_prompt.py` | **Yes — rewrite** | Triage keys on ASAN excerpts and crash-class taxonomy |
| `harness/prompts/patch_prompt.py` | **Yes — rewrite** | Asks for `git diff -- '*.c' '*.h'`, assumes ASAN trace, `memcpy`-style root-cause guidance |
| `harness/prompts/system_prompt.py` | **Yes — rewrite** | Authorization block says "C/C++ target", "sanitizer output" |
| `harness/asan.py` | **Yes — rewrite** | Regex for `#N 0xHEX in func /path:line` frames; feeds dedup, judge, novelty |
| `targets/README.md` + Dockerfile template | **Yes — rewrite** | `gcc -fsanitize=address`, `entry.c` wrapper pattern |
| `harness/patch_grade.py` | Light edit | `_t1_passes()` checks `AddressSanitizer:` substring; rest of the verification ladder is generic |
| `harness/report.py` | Light edit | `_SECTIONS` tuple and token lists need to match the new report structure; flow is generic |
| `harness/novelty.py` | Light edit | `crash_file_from_frame()` is ASAN-specific; git-log logic is generic. Drop entirely if no upstream. |
| `harness/config.py` | Light edit | May need new fields (`profile`, `run_command` instead of `binary_path`); `attack_surface` likely stays |
| `harness/artifacts.py` | Light edit | `crash_type`/`exit_code` semantics may shift; `ReportVerdict.section_scores` keys must match new sections |
| `harness/dedup.py` | Light edit | Signature function needs the new parser; grouping logic is generic |
| `harness/prompts/recon_prompt.py` | Light edit | Mostly language-agnostic; scrub C idioms |
| `harness/cli.py` | **Unchanged** | Orchestration is domain-neutral |
| `harness/agent.py` | **Unchanged** | Agent runner is generic |
| `harness/docker_ops.py` | **Unchanged** | Container plumbing is generic (may need changes if isolation ≠ Docker) |
| `harness/find.py`, `grade.py`, `recon.py`, `judge.py`, `patch.py` | **Unchanged** | Flow is generic; only injected prompts change |

## STEP 3 — Present a plan and get confirmation

Before editing anything, summarize back to the user:

1. **What you understood** — restate their goal in one sentence
2. **What will change** — list each file you'll edit with a one-line rationale
3. **What stays** — reassure them the orchestration core is untouched; call
   out explicitly if the plan does reach into the shared files listed above
   (they also serve `dnr_harness/`) and why
4. **Open questions** — anything you're still unsure about

Wait for explicit approval. If they adjust the plan, incorporate and re-confirm.

## STEP 4 — Execute

Edit the files per the approved plan. Work through them in dependency order: prompts and parser first (they're standalone), then config/artifacts, then the target template, then README. Commit incrementally if the user wants checkpoints.

## STEP 5 — Validate

1. Add a canary target under `targets/<domain>-canary/` with 2–3 planted bugs of the new class
2. Run: `bin/vp-sandboxed run <domain>-canary --model <model-id> --runs 3 --parallel --stream --max-turns 50` (use Claude Opus unless the user specifies a different model). Run `./scripts/setup_sandbox.sh` once first if the sandbox isn't already set up.
3. Confirm all planted bugs are found and graded PASS
4. Confirm judge triage worked: `cat results/<domain>-canary/<ts>/reports/judge_log.jsonl` — expect one NEW per distinct bug, DUP_SKIP for repeats
5. Confirm reports landed: `ls results/<domain>-canary/<ts>/reports/bug_*/report.json` and spot-check section scores
6. Run `vuln-pipeline dedup results/<domain>-canary/` and confirm signatures group correctly