omnigraph-best-practices

---
name: omnigraph-best-practices
description: Operate a locally or remotely deployed Omnigraph graph database. Use this skill whenever you see Omnigraph CLI commands (omnigraph init/query/mutate/load/ingest/schema/lint/embed/branch/commit), .pg schema files, .gq query files, RustFS S3 URIs (s3://omnigraph-local/...), remote bearer-authed graph endpoints, 504 errors against a graph, work inside a folder containing omnigraph.yaml or cluster.yaml, or cluster commands (omnigraph cluster validate/plan/apply/approve, omnigraph-server --cluster). Covers cluster-mode declarative deployments (cluster.yaml, plan/apply loop, approval-gated deletes), local RustFS setup, project layout, schema authoring and evolution (plan before apply), query linting, data changes (mutate vs load vs ingest, --mode merge vs overwrite), branches for data review, embeddings, aliases for automation, HTTP server operation, Cedar policy, remote graph operations (504 verification ritual, ingest vs load tradeoffs, version drift), and common gotchas. Especially important BEFORE running schema apply (plan first), any load (pick --mode carefully), any .gq/.pg edit (lint afterward), or any write to a remote graph (verify via commit list afterward). Apply this skill aggressively when the user mentions Omnigraph, graph migrations, remote graph deploys, 504 errors, or graph database development.
license: MIT
---

# Operating Omnigraph Locally

This skill captures the operational rules for working with a locally deployed Omnigraph (RustFS-backed or remote S3). Follow them when authoring schema, writing queries, loading data, evolving schema, or automating graph operations.

## The Seven Rules

1. **Lint before commit** — `omnigraph lint --schema schema.pg --query queries/foo.gq` validates both sides against each other. No running repo required. (`omnigraph query lint` still works as a deprecated alias.)
2. **Plan before apply** — never run `schema apply` without a successful `schema plan` first. Apply is destructive; plan is free. (Cluster mode has the same rule with different verbs: `cluster plan` before `cluster apply` — the plan embeds the engine's real migration steps.)
3. **Branches are for data; apply is for schema** — review data ingests on a feature branch then merge. Schema changes go straight to `main` (single-graph: `omnigraph schema apply`; cluster mode: edit the `.pg` and run `cluster apply` — there is no direct `schema apply` in cluster deployments).
4. **Pick the right write command** — `mutate` for edits (typechecked, parameterized), `load --mode merge` for bulk upsert on local repos, `ingest` for remote, `load --mode overwrite` only for clean slates.
5. **Parameterize everything** — never string-interpolate values into `.gq` bodies or `--params`. Declare `$var: Type` and pass via `--params`.
6. **Expose agent operations as aliases** — not raw CLI invocations. Aliases decouple the operation name from the query implementation.
7. **Verify after every remote write** — compare `commit list --branch main` head before and after. The CLI's exit code is not authoritative on remote graphs; proxies can drop the response while the write commits server-side. See `references/remote-ops.md` for the verification ritual and how to recover from 504s.

## Five Ontology Design Criteria (Gruber 1993)

Omnigraph schemas are ontologies. The canonical design criteria from Gruber's *Toward Principles for the Design of Ontologies Used for Knowledge Sharing* (Int. J. Human-Computer Studies 43:907–928) apply directly when authoring `.pg` files.

1. **Clarity** — definitions should communicate intended meaning unambiguously and be independent of social or computational context. In Omnigraph: precise type names, narrow enums over `String`, `@check`/`@range` for stated invariants. A reviewer should understand the domain from the schema alone.
2. **Coherence** — inferences sanctioned by the schema must be consistent with the domain modeled. Gruber's trap: defining quantity as a `(magnitude, unit)` pair makes `6 feet ≠ 2 yards` even though they describe the same length. In Omnigraph: watch for `@card`, `@unique`, and edge directionality that let the schema distinguish things the domain treats as equal.
3. **Extendibility** — the schema should support specialization without revising existing definitions. In Omnigraph: prefer interfaces for shared shape, leave enums open where the domain genuinely admits more, model identifiers via mapping functions rather than baking units/formats into the entity.
4. **Minimal encoding bias** — representation choices made for notation or implementation convenience leak into the model. In Omnigraph: don't type dates as `String` because the source API returns strings; separate conceptual entities (a publication date, a person) from their surface encoding (a year integer, a name string) when both matter.
5. **Minimal ontological commitment** — make as few claims about the world as the use case requires. In Omnigraph: don't add required properties, closed enums, or `@card(1..1)` "in case"; tighten later via `schema plan`/`apply` when a real constraint emerges. Weaker schemas leave consumers room to specialize.

The criteria trade off against each other — Clarity wants tight definitions while Minimal Commitment wants weak ones. Gruber's resolution: *having decided a distinction is worth making, give it the tightest possible definition*. Decide what to model conservatively; once modeled, constrain precisely.

## Schema Authoring Principles

Twelve practical rules for `.pg` authoring — full text and examples in [`docs/omni-schema.md`](../../docs/omni-schema.md). In short: schema-is-the-contract · explicit identity via `@key` · model meaning not tables · strong intentional types · deliberate optionality · shared shape in interfaces · schema-level constraints (`@unique`/`@index`/`@range`/`@check`/`@card`) · search as a schema decision · edge semantics matter · reviewable schemas · intentional migrations (`@rename_from`) · domain clarity over ORM habits.

Design flow: entities → stable keys → relationships worth their own edge → enum candidates → uniqueness/bounds/cardinality → search needs → shared shape into interfaces → evolution plan.

## Provenance Is Structural (Multi-Agent Source of Truth)

When Omnigraph serves as canonical truth across multiple agents, every assertion must answer *who said it, when, based on what evidence*. This is the runtime guarantee Gruber's criteria don't cover — his agents shared vocabulary; ours additionally must share attribution. Provenance belongs in the schema, not in logs.

Without structural provenance, agents cannot reconcile contradictory assertions, retract facts when a source is discredited, replay graph state at a past timestamp, or distinguish high-evidence facts from speculation.

**In Omnigraph:** model provenance as a `Claim`-style interface (or a separate `Claim` node linked to each sourced fact) with required fields — `asserted_by: Actor`, `asserted_at: DateTime`, `evidence_source: Source`, optionally `confidence: F64`. Don't stash provenance into a free-text `source: String` or a `metadata: JSON` dump — structured provenance is queryable, indexable, and migratable; free-form is none of these.

## Local Setup

### Bootstrap a local RustFS + Omnigraph in one command

**Requires Docker.** RustFS runs in a container — install [Docker](https://docs.docker.com/get-docker/) and verify before running the bootstrap:

```bash
docker version >/dev/null 2>&1 || { echo "Install Docker first: https://docs.docker.com/get-docker/"; exit 1; }
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/local-rustfs-bootstrap.sh | bash
```

Defaults: RustFS S3 on `127.0.0.1:9000`, console on `:9001`, `omnigraph-server` on `:8080`, bucket `omnigraph-local`. Override with `BUCKET=foo PREFIX=repos/bar BIND=127.0.0.1:8080 curl ...`.

**Heads up — port :8080 is in use after bootstrap.** The bootstrap auto-starts an `omnigraph-server` against its own demo repo. If you start a second server (e.g. for a different cookbook) without stopping the first, you'll get a silent port collision. Either stop the bootstrap server or start yours with `--bind 127.0.0.1:8090`.

Bootstrap also installs `omnigraph` and `omnigraph-server` binaries under `<workdir>/.omnigraph-rustfs-demo/bin/` — **not on PATH by default**. Add it or invoke binaries by absolute path.

### AWS env vars (for `init`, `load`, and the server)

`init` and `load` write S3-backed storage directly, and `omnigraph-server` reads from it. Both need AWS credentials pointed at RustFS. Keep them in `.env.omni` (git-ignored):

```bash
AWS_ACCESS_KEY_ID=rustfsadmin
AWS_SECRET_ACCESS_KEY=rustfsadmin
AWS_REGION=us-east-1
AWS_ENDPOINT_URL=http://127.0.0.1:9000
AWS_ENDPOINT_URL_S3=http://127.0.0.1:9000
AWS_ALLOW_HTTP=true
AWS_S3_FORCE_PATH_STYLE=true
```

Source before running CLI commands:

```bash
set -a && source .env.omni && set +a
```

Or reference `auth.env_file: .env.omni` in `omnigraph.yaml` for automatic loading.

### Validate the setup

```bash
curl http://127.0.0.1:8080/healthz
omnigraph snapshot s3://omnigraph-local/repos/<your-repo> --json
```

## Project Layout

### Two deployment models — pick one per project

- **Cluster mode** (omnigraph >= 0.7.0, recommended for new projects): a
  `cluster.yaml` declares the deployment (graphs, schemas, stored queries,
  policies); `omnigraph cluster apply` converges it and
  `omnigraph-server --cluster .` serves it. `omnigraph.yaml` shrinks to
  per-operator ergonomics (aliases, CLI defaults, `cli.actor`). See
  `references/cluster.md`.
- **Single-graph mode** (classic): `omnigraph.yaml` is the operational
  backbone below. Fully supported; required for S3-hosted graphs today.

A server boots from one source or the other — never a merge of both.

### `omnigraph.yaml` is the operational backbone (single-graph mode)

Run CLI commands from the folder containing `omnigraph.yaml` — relative paths for `queries/`, `schema.pg`, and `.env.omni` resolve from there.

```yaml
graphs:
  local_s3:
    uri: s3://omnigraph-local/repos/spike-intel
  local_server:
    uri: http://127.0.0.1:8080
    bearer_token_env: OMNIGRAPH_BEARER_TOKEN

server:
  graph: local_s3
  bind: 127.0.0.1:8080

cli:
  graph: local_s3
  branch: main
  output_format: jsonl      # use `table` for human reading; `jsonl` for automation

auth:
  env_file: .env.omni

query:
  roots: [queries, .]

aliases:
  signal:
    command: query        # `read` still accepted (deprecated)
    query: signals.gq
    name: get_signal
    args: [slug]
    format: kv
```

Key naming: the config field is `graphs:` (not `targets:` — that's the old schema). `cli.graph` and `server.graph` replaced `cli.target` / `server.target`. The CLI flag is still `--target <name>` though.

### What to commit

**Commit:** `schema.pg`, `queries/*.gq`, `omnigraph.yaml`, `seed.md`, `seed.jsonl`, per-cookbook `README.md` and `CLAUDE.md`.

**Ignore:** `.env.omni` (credentials), `.claude/` (local agent state), `*.omni/` (local repo artifacts), `.omnigraph-rustfs-demo/` (bootstrap state).

### Give agents a `CLAUDE.md`

A per-cookbook `CLAUDE.md` tells coding agents where files live and what conventions matter. Without it, agents re-discover the same things every session.

## Common Gotchas

These are the traps most likely to bite. Scan this table before debugging any parse or runtime error.

| Trap | Symptom | Fix |
|------|---------|-----|
| `#` comments in `.pg` | `parse error: expected schema_file` | Use `//` |
| Standalone `enum Foo { ... }` block | `parse error: expected EOI or schema_decl` | Inline: `kind: enum(a, b)` |
| `[Category]` (list of enum) | compile error | Use `[String]`; lists must contain scalars |
| `@embed(text)` without quotes | `unexpected constraint_name` | `@embed("text")` |
| `@unique(src)` on edge without body block | parse error | `@card(1..1) { @unique(src) }` |
| `load --mode merge` after `@embed` source change | stale embeddings | `omnigraph embed --reembed_all` or `load --mode overwrite` |
| `schema apply` with feature branches open | rejected | Merge or delete branches first |
| `nearest(...)` / `bm25(...)` / `rrf(...)` without `limit` | compile error | Add `limit N` |
| Adding non-nullable property without backfill | unsupported migration | Make optional → backfill → tighten in follow-up apply |
| Config uses `targets:` / `target:` | `graph 'X' not found in omnigraph.yaml` | Rename to `graphs:` / `graph:` |
| `omnigraph init --json` | `unexpected argument --json` | `init` doesn't support `--json`; drop the flag |
| `omnigraph init` on an already-initialized URI | `AlreadyInitialized` error (v0.6.0+) | `--force` to re-init (skips the schema preflight; does **not** purge data) |
| `schema apply` dropping a property/type | soft-dropped or rejected (no data loss) | add `--allow-data-loss` to actually drop the column |
| Committing `.env.omni` | credential leak | Add `.env*` to `.gitignore` |
| Non-parameterized query values | typecheck surprise, injection risk | Declare `$param: Type` and pass via `--params` |
| Missing required field in `insert` | `T12: insert for 'X' must provide non-nullable property 'Y'` | Accept the param in the mutation signature |
| Long-lived feature branches | merge conflicts, schema apply blocked | Merge promptly; delete when done |
| `mutation { ... }` wrapper in `.gq` | `parse error: expected query_file` at line 1 | Use `query <name>(...) { insert T { ... } }`; there is no top-level `mutation` keyword |
| `--config` placed before subcommand | `unexpected argument --config` | Put `--config` **after** the subcommand (e.g. `omnigraph schema show --config X`) |
| Reading a large schema via stdout-capped tool | Truncated, garbled, or duplicated output | `omnigraph schema show > /tmp/schema.pg` first; then read the file with offset/limit |
| `omnigraph load` against a remote URI | `load is only supported against local repo URIs in this milestone` | Use `ingest` for remote graphs |
| Blind retry after 504 | Duplicate Signal/Decision/Claim (append-only types lack `@key` dedup) | `commit list --branch main --json` first; head advanced means it landed; only retry if unchanged |
| `sync_branch()` mentioned in version-drift error | Searching for nonexistent CLI command | Server-internal directive in error text; retry once, fall back to `ingest` if persistent |
| Stale `ingest/<name>` branches at `main`'s head | 504-orphaned empty branches; eventually block writes | List branches, find ones at `main`'s `graph_commit_id`, `omnigraph branch delete --config X <name>` |
| Top-level `policy:`/`queries:` with a **named** graph (`server.graph`/`--target`) | server refuses to boot with migration guidance (v0.6.1) | Nest under `graphs.<name>.policy` / `graphs.<name>.queries`. Top-level is valid **only** for an anonymous bare-URI single-graph server |
| `omnigraph optimize` against a table with a `Blob` property | table is **skipped**, not failed (Lance blob-v2 compaction bug) | Expected — `--json` reports it under `skipped`; non-blob tables still compact |

## Edge Traversal Casing

Schema declares edges in **PascalCase** (`FormsPattern`), but queries traverse them in **lowerCamelCase**:

```gq
match {
    $s: Signal
    $s formsPattern $p       // edge FormsPattern: Signal -> Pattern
}
```

## Deep Dives

- `references/cluster.md` — cluster-mode declarative deployments: cluster.yaml, the validate/import/plan/apply loop, approval-gated deletes, `--cluster` serving, the two-file contract, recovery

For anything beyond the basics, load the relevant reference file. Each is self-contained — load only what you need.

| Reference | When to load |
|-----------|--------------|
| [`references/schema.md`](references/schema.md) | Editing `.pg` files, running `schema plan`/`apply`, renaming types, backfilling required fields |
| [`references/queries.md`](references/queries.md) | Writing or linting `.gq` files, search functions, aggregations, multi-hop patterns |
| [`references/data.md`](references/data.md) | Choosing between `mutate`, `load`, and `ingest`; branch review workflow; destructive ops |
| [`references/remote-ops.md`](references/remote-ops.md) | Operating against a remote/CloudFront-fronted graph: 504 verification ritual, version drift, ingest fingerprints, append-only retry safety |
| [`references/search.md`](references/search.md) | Embeddings, `@embed`, vector/text ranking, scope-then-rank pattern |
| [`references/aliases.md`](references/aliases.md) | Defining aliases for agents, structured output, JSON args |
| [`references/stored-queries.md`](references/stored-queries.md) | Server-side stored-query registry (v0.6.1): `queries:` config, `omnigraph queries validate/list`, `GET /queries` + `POST /queries/{name}`, `invoke_query` Cedar gating, MCP exposure |
| [`references/server-policy.md`](references/server-policy.md) | Starting the HTTP server, routes, bearer auth, Cedar policy gating, multi-graph mode |
| [`references/commands.md`](references/commands.md) | `snapshot`, `export`, `commit list/show`, config resolution order |