code_of_conquest_dnd/docs/superpowers/specs/2026-07-11-content-canon-build-tool-design.md

Content & Canon Build Tool — Design

• Date: 2026-07-11
• Status: Design approved; not yet planned/implemented.
• Implements: the build tool whose contract was fixed by 2026-07-11-content-canon-schema-design.md (the locked schema — envelope, two categories, NPC layer, gate logic, role-based split, canon home, promotion/validation). That schema is locked; this doc does not re-litigate it — it specifies the compiler that realizes it.
• Specimen: candidate-town.md (Duncarrow) at repo root — the bible the tool parses and, as its acceptance test, promotes.

---

Purpose

Build the compiler that turns the authored Markdown bible (content/lore/*.md, prose + fenced yaml blocks) into validated runtime JSON, split by role into a client tree and a server-only tree, with validation as a hard gate. Promote Duncarrow as the tool's first real use and end-to-end proof.

Charter alignment: §2 (code owns state, AI owns text — this is a build-time transform, no runtime state), §4 (the server/ split keeps spoilers + personas off the client), §6 (the gate skeleton feeds available_moves), §16 (/content layout, "prompts/content are source, reviewed and versioned"), §18 (branch off dev).

---

Locked decisions (from the brainstorm)

#	Decision	Choice
L1	Language / runtime	Python, standalone package under tools/ — reuses the repo's Python + jsonschema/pytest stack; no Godot dependency to run a build; not bundled into the api proxy image.
L2	Output layout	Per-category dirs that fit the existing readers. Minimal, additive loader change; no per-town bundle rewrite.
L3	When it runs	Manual command + CI --check. Built JSON is committed; CI fails on stale-or-invalid.
L4	start_disposition → integer	Band midpoint. Refinement (L4a): only the client needs band integers at runtime, so the midpoint mapping lives client-side; the build stays number-free and emits the rung name.
L5	Promote Duncarrow this cycle	Yes — the tool's first real use and acceptance test.

Refinements settled in the brainstorm (explicitly approved)

• L4a — the build needs zero disposition numbers. The only runtime consumer of band integers is the client (it owns disposition state and computes available_moves). The build validates gate names only. So band numbers + the rung→midpoint mapping live in one client GDScript const (the palette.gd pattern); the build emits start_disposition as the rung name and passes it through. Legal rung names are read from a structured rungs: [...] field on rule.disposition-ladder in the bible — the bible owns names/order (D1), code owns numbers, and there is no cross-language table to keep in sync.
• L5a — the validator rejects knows → canon-entity. Spec category A: canon entities never appear in a knows list. The specimen's Mayor and Harn carry knows: town.duncarrow; the validator enforces the rule and would reject them. Promotion drops those links (the town is public/tier-0, spoken freely, and needs no gated knows-link). This is intended — the validator catching real authoring drift.

---

Scope boundary

In scope (this cycle):

• The tools/content_build/ compiler: parse → resolve/validate → emit → split.
• The validation gate (all rules below), abort-on-failure with source-line context.
• A --check mode for CI (stale-or-invalid → nonzero exit).
• Promoting Duncarrow: candidate-town.md → content/lore/duncarrow.md, reformat blocks to valid YAML, drop the knows → canon-entity links, add rungs: to the ladder, flip entries candidate → canon, add content/lore/canon-roadmap.md.
• Committing the built content/world/** + content/server/** artifacts.
• A minimal, additive content_db.gd change so the client loader tolerates the new canon/ and topics/ dirs (existing dirs + legacy fenn.json untouched).

Out of scope (next cycle / deferred):

• Runtime consumption — the client computing gated available_moves from the new npc skeletons, the api reading personas from content/server/, and voicing Mera Fenn end-to-end. Promotion proves the compiler, not the runtime.
• Legacy Fenn migration — content/world/npcs/fenn.json's old persona+knowledge+capabilities shape stays as-is; it is superseded when Fenn gets his own bible (flagged deferred in the schema doc).
• Godot export packaging — deliberately including world/ and excluding server/ from the client export. /content already sits outside res://, so server/ is out of the export by default; the packaging step is the pre-existing debt, unblocked by this clean split but not built here.

---

Architecture

tools/content_build/ — a Python package, run as python -m content_build.

tools/content_build/
  __init__.py
  __main__.py    # CLI entry: build [--check] [--lore DIR] [--world DIR] [--server DIR]
  parse.py       # Markdown → list of raw entries (dict + source line)
  model.py       # Entry / NpcLayer / KnowsLink dataclasses; envelope + category rules
  resolve.py     # id map, cross-ref resolution, the validation gate
  rungs.py       # reads ladder `rungs:`; legal-gate set = rungs + {never}
  emit.py        # route fields per D4; serialize the two JSON trees
  errors.py      # BuildError carrying (entry id, source file:line, message)
  tests/
    fixtures/    # tiny valid + deliberately-broken bibles
    test_parse.py
    test_resolve.py       # one test per gate rule (each broken fixture)
    test_emit.py          # split routing + secrecy safety-check
    test_duncarrow.py     # the real end-to-end round-trip

Dependencies: PyYAML (block parsing). Added to a dev requirements file, not api/requirements.txt — the proxy runtime never imports the build tool.

Each module has one job and a narrow interface:

• parse — parse_bible(path) -> list[RawEntry]. Knows Markdown fences and YAML; knows nothing about the schema's meaning.
• model — the dataclasses + envelope/category predicates. Pure data + rules, no I/O.
• resolve — resolve(entries) -> World | raises BuildError. The gate. Pure function over parsed entries; no file writes.
• rungs — the one place that knows the legal gate vocabulary, sourced from the ladder entry.
• emit — emit(world, world_dir, server_dir). Routing + serialization only.
• main — orchestration + --check diffing + exit codes.

---

Pipeline

1. Parse

For each content/lore/*.md: extract every ```yaml-fenced block, yaml.safe_load each into a dict, attach source file:line. Prose between blocks is ignored. One block = one entry.

The promoted bible must contain valid YAML — the specimen's current - fact: x gate: y one-liners are not valid YAML and become {fact: x, gate: y} (or the two-line block form) during promotion (spec D3: "formalized as real YAML so parsing is trivial").

2. Resolve & validate — the gate

Build the id → entry map, then enforce every rule below. Any failure aborts the build, emits nothing, and prints the offending id + source file:line.

• No id collision — two entries with the same id → fail.
• id/type agreement — the id prefix must be a legal namespace from the known set {town, place, region, faction, person, rule, rumor, fact, secret, npc}, and type must be consistent with it: prefix npc → type is person (the interactive layer vs the person.* world-record); otherwise the prefix equals type. A non-namespace prefix (e.g. the specimen's shrine.) → fail; such ids are renamed to their type namespace at promotion (shrine.the-white-antlers → place.the-white-antlers).
• related resolves — every id in every related list is a real entry defined in the built world (strict, per schema §Build). The specimen references region.the-tallow-reach and faction.elves without defining them; promotion adds minimal canon stubs for both so the graph stays closed (there is no forward-ref-to-pending concept — the closed graph is the invariant).
• NPC-layer shape — npc.* entries carry knows and start_disposition; person.* records carry neither.
• knows targets — every knows[].fact resolves to a knowledge entry (rumor/fact/secret). A canon-entity id in a knows list → fail (L5a).
• Gate legality — every knows[].gate ∈ rungs (read from rule.disposition-ladder) ∪ {never}.
• Ladder coverage — exactly one rule.disposition-ladder entry exists and carries a non-empty rungs: list.
• Secrecy safety-check — after routing (stage 4), no secrecy ≥ 3 body may appear in any client (content/world/**) artifact → fail. Defense-in-depth behind the structural routing, not the router itself (schema D4).
• start_disposition rung legality — the rung is one of rungs.

Contradiction-with-existing-canon is author-asserted, not auto-checked — it is resolved by the human at the candidate → canon promotion (schema §Build).

3. Emit

Only status: canon entries build into artifacts. candidate entries are dev-visible (they parse and validate) but are not emitted — with an all-candidate bible the artifacts are empty, which is why L5 promotes Duncarrow to canon this cycle.

4. Split (routing per schema D4)

Client — content/world/ (bundled in the export; also readable by the server):

Dir	Contents
npcs/<slug>.json	npc skeleton: {id, type:"person", start_disposition:"<rung>", related, knows:[{fact_id, gate}]} — no persona/body
canon/<slug>.json	each public canon entity (town/place/region/faction/person/rule) whole, including its public body and (for the ladder) rungs
topics/<slug>.json	knowledge-entry skeleton: {id, type, related} — no body

Server — content/server/ (server-only; excluded from the Godot export):

Dir	Contents
npcs/<slug>.json	{id, persona:<body>, disposition_notes}
topics/<slug>.json	{id, body, secrecy} — the knowledge bodies /npc/speak voices

• Canon-entity body is public (category A, secrecy:0) → ships in world/canon/.
• Knowledge-entry body and NPC persona/disposition_notes → content/server/ only.
• The server process reads both trees (it has full filesystem access); only the client export excludes content/server/.

<slug> = the id with its <type>./npc. prefix stripped (npc.mera-fenn → mera-fenn, town.duncarrow → duncarrow).

---

Client loader change (minimal, additive)

client/scripts/content/content_db.gd gains two dirs:

• load_from() also loads world/canon/ and world/topics/ into generic id-keyed dictionaries (canon, topics), mirroring the existing _load_dir pattern.
• Existing locations/npcs/quests/items loading is untouched; the legacy fenn.json still loads (it has an id, which is all _load_dir requires).
• Add canon(id) / topic(id) / has_canon / has_topic accessors.

No api change this cycle — runtime consumption of the new shapes is next cycle (scope boundary).

---

CLI & CI

• python -m content_build — rebuild content/world/** + content/server/** from content/lore/*.md.
• python -m content_build --check — build to memory, diff against the committed JSON, exit nonzero if the tree is stale (would-differ) or validation fails. This is the CI gate.
• Dir overrides (--lore/--world/--server) exist for tests to run against fixtures in a temp dir.

Built JSON is committed (schema/prompt "content is source, reviewed" culture); --check keeps the committed artifacts honest.

---

Testing

• Parse — a fixture bible with prose + multiple yaml blocks parses to the expected entry list with source lines.
• Resolve — one deliberately-broken fixture per gate rule (id collision, bad id/type, dangling related, person.* with knows, knows → canon-entity, illegal gate, missing ladder, secrecy ≥ 3 body routed to client), each asserting the specific BuildError + that nothing is emitted.
• Emit — a valid fixture routes every field to the correct tree; the secrecy safety-check passes for a legal body and fails for an illegal one.
• Duncarrow round-trip (test_duncarrow.py) — the real acceptance test: the promoted content/lore/duncarrow.md builds green, and the emitted client/server JSON matches expectations (npc skeletons carry no persona; the secret.crell-* body lands only in content/server/topics/; world/canon/ holds the public entities incl. the ladder rungs).

---

Promotion checklist (Duncarrow — the acceptance run)

1. git mv candidate-town.md content/lore/duncarrow.md.
2. Reformat every fenced block to a ```yaml fence with valid YAML (fix the knows one-liners to {fact, gate}).
3. Drop the knows: town.duncarrow links from Mayor and Harn (L5a).
4. Rename shrine.the-white-antlers → place.the-white-antlers and update every related reference to it (id/type-agreement rule).
5. Add canon stubs for region.the-tallow-reach and faction.elves so all related ids resolve (closed-graph rule).
6. Drop secret.crell-runs-slave-trade from the public person.mayor-oswin-crell record's related — a public record must not point at a secret; that graph edge lives on the secret entry (server-side). The npc.* layer still holds the secret at gate: never, which is the actual bounded-move test.
7. Add rungs: [hostile, cold, neutral, warm, trusted] to rule.disposition-ladder.
8. Add content/lore/canon-roadmap.md (the authored-vs-pending index, schema D5).
9. Run the tool; fix validation errors; flip each entry candidate → canon as it passes.
10. Commit the promoted bible + built content/world/** + content/server/**.
11. --check is green in CI.

---

Open / deferred (carried, not decided here)

• Band numbers — tunable placeholder in the client const; settle by playtest (schema D1). The build never sees them (L4a).
• Runtime consumption / legacy Fenn migration / export packaging — the scope boundary above; each is its own later cycle.
• make/just wrapper — a convenience target for the build command can be added when a task runner lands; not required for this cycle.
• ids are self-describing (spoiler property, schema-level). Per D4 the gate skeleton ships each NPC's knows:[{fact_id, gate}] to the client, so a descriptive slug like secret.crell-runs-slave-trade reaches the client binary even when held at gate: never (the body never does). This is a property of the locked schema, not the build tool — opaque/hashed topic ids are a possible future hardening if datamining becomes a concern. Noted, not reopened.