knowledge-base

---
name: "knowledge-base"
description: "Project-specific prompt optimization knowledge management. Use when storing or retrieving learned patterns from comparisons. Provides schema, extraction criteria, capacity management, and retention scoring."
license: "MIT"
---

# Knowledge Base Skill

## Storage Location

```
{project_root}/.claude/.rashomon/prompt-knowledge.yaml
```

## Schema

```yaml
patterns:
  - name: "Pattern name"
    what_to_look_for: |
      When this pattern applies
    improvement: |
      How to improve when detected
    learned_from: "Date and context"
    source: "comparison ID, report path, or user feedback"
    source_fingerprint: "stable hash or revision identifying the evidence"
    validity_scope: "project area, version range, or conditions where this applies"
    last_verified: "ISO-8601 timestamp"
    invalidated_when: "observable condition that requires revalidation"
    confidence: 0.0-1.0
    times_applied: 0

anti_patterns:
  - name: "Anti-pattern name"
    what_to_look_for: |
      What to avoid
    why_bad: |
      Why problematic in this project
    learned_from: "Date and context"
    source: "comparison ID, report path, or user feedback"
    source_fingerprint: "stable hash or revision identifying the evidence"
    validity_scope: "project area, version range, or conditions where this applies"
    last_verified: "ISO-8601 timestamp"
    invalidated_when: "observable condition that requires revalidation"
    confidence: 0.0-1.0
    times_applied: 0

metadata:
  last_updated: "ISO-8601 timestamp"
  total_comparisons: 0
  patterns_count: 0
  anti_patterns_count: 0
  max_entries: 20
```

## Extraction Criteria

### Save as Improvement Pattern

**ALL conditions must be true**:
- Optimized prompt showed **structural improvement** (not variance)
- Improvement is **project-specific** (not explained by BP-001~008)
- Pattern is likely to **recur** in this project

**Confidence Assignment**:
| Evidence | Confidence |
|----------|------------|
| Multiple comparisons confirmed | 0.8+ |
| Single comparison, clear effect | 0.5-0.7 |
| Effect present but uncertain | 0.3-0.5 |

**Minimum threshold**: 0.3 (entries below this are skipped)

### Save as Anti-Pattern

**ALL conditions must be true**:
- Original had problem **specific to this project**
- Problem is **project-specific** (beyond standard patterns BP-001~008)
- Problem **likely to recur**

### Extraction Scope

Save only entries that are:
- Project-specific (beyond standard best practices BP-001~008)
- Likely to recur in this project
- Showing clear effect (structural improvement, confidence ≥ 0.3)

## Capacity Management

**Maximum**: 20 entries (patterns + anti_patterns combined)

**Retention Score**: `confidence * (1 + log(times_applied + 1))`

This formula:
- Prioritizes high-confidence entries
- Rewards frequently-used patterns
- Applies no direct age penalty; validity is evaluated separately

Age alone does not reduce retention. Before scoring, revalidate an entry when its `invalidated_when` condition is observed or its source fingerprint no longer matches. An entry with unresolved validity is excluded from retrieval until verified.

**Eviction Process**:
1. Calculate retention scores for all entries
2. Calculate score for new candidate
3. If new > lowest existing: remove lowest, add new
4. Otherwise: skip new entry

## Operations

### Retrieval

At start of prompt analysis:
1. Read `.claude/.rashomon/prompt-knowledge.yaml` (if exists)
2. Exclude entries whose validity condition is triggered or whose source fingerprint is stale
3. For each valid entry, check `what_to_look_for` against current prompt
4. Return relevant entries with relevance scores and provenance

Retrieval is read-only. It records proposed entry IDs in the prompt-analysis result; it does not increment counters or write the knowledge file.

### Storage

After a comparison and user feedback confirm how an entry affected execution:
1. Evaluate against extraction criteria
2. Generate candidate entries
3. Check for duplicates
4. Increment `times_applied` for each valid entry whose use is confirmed by the report
5. Revalidate source fingerprints and validity conditions
6. Apply capacity management
7. Write updated knowledge base
8. Update metadata

## Example Entry

```yaml
patterns:
  - name: "TypeScript interface reference"
    what_to_look_for: |
      Code generation prompts creating TypeScript types without referencing existing type definitions in src/types/
    improvement: |
      Add: "Reference existing types in src/types/ to maintain consistency and avoid duplicate type definitions"
    learned_from: "2026-01-14: Comparison showed better type reuse"
    source: "comparison: cmp-20260114-types"
    source_fingerprint: "git:abc123:src/types"
    validity_scope: "TypeScript generation under src/"
    last_verified: "2026-01-14T12:00:00Z"
    invalidated_when: "src/types is removed or its public type policy changes"
    confidence: 0.7
    times_applied: 3
```

## Feedback-Based Adjustments

When comparison results require knowledge base updates:

**Confidence Adjustments**:
- User confirms improvement: +0.1 (cap at 0.95)
- Pattern led to worse result: -0.2
- Remove entry if confidence < 0.2 after decrease

**Entry Management**:
- Add new entries from user insight (initial confidence: 0.5)
- Remove entries that fall below confidence threshold