code_of_conquest_dnd/docs/superpowers/specs/2026-07-09-narrator-ollama-pipeline-design.md

Narrator / Ollama Pipeline (server) — Design

Date: 2026-07-09 Status: approved (design); implementation plan to follow Charter refs: §2 (code owns state / AI owns text), §3 (tone/voice), §4 (the guarding proxy, model routing as config, logging from day one), §5 (the DM is not one prompt; Narrator = the good model), §10 (seed + full prompt logged), §11 (canon log), §12 (output contracts, tags, retry policy), §13 (degraded DM), §14 (cost/latency), §16 (prompts are source code) Binds to: the canon log contract (docs/canon-log.md, docs/schemas/canon-log.schema.json) and the existing proxy skeleton (api/app/main.py — role endpoints + valid_turn + unified 422). Consumed by the later client HTTP loop.

Problem

The proxy validates every posted canon log but every role endpoint still return {"detail": "not implemented"} — no model is wired, no prompt is authored, nothing is logged. Until a real model sits behind /dm/narrate, the POC's one question — does bounded AI dialogue feel alive? (charter §17) — cannot be tested at all.

This plan makes /dm/narrate real: assemble the Narrator prompt + a canon-log digest, call Ollama, return the prose, log the call, and signal failure honestly — building the reusable server-side model-call pipeline that the other four roles later drop into.

Scope

In scope — the shared pipeline + the Narrator as its first, proving role:

• ollama_client.py — httpx call to Ollama /api/chat (non-streaming), single retry, typed ModelError.
• routing.py — role → {model, options} config, env-driven (§4/§5).
• prompts.py — load a role's .md system prompt (cached) + render_digest(canon_log).
• call_log.py — structured JSON-lines logging of every call (§10).
• narrate.py — the Narrator service: render → route → call → log → return.
• The authored narrator.md system-prompt body (source code, §16).
• /dm/narrate wired to the service; main.py handler stays thin.
• Unit tests (mocked Ollama boundary) + one gated live smoke test.

Out of scope — later plans (recorded in docs/roadmap.md):

• Streaming (§14) — non-streaming request/response now; the Ollama client is shaped so a streaming mode is additive, not a rewrite.
• The client HTTP layer, degraded-DM fallback display, and [FACT] harvest (the client already owns TagExtractor).
• The other four roles (Adjudicator / Improviser / NPC / Banter).
• The Replicate provider; auth / metering.

Decisions (locked in brainstorming)

1. Shared pipeline + Narrator first — build the reusable machinery once; the Narrator proves it; the other roles become thin drop-ins.
2. Test: mocked boundary + gated live smoke — unit tests use httpx MockTransport (no new dependency); a @pytest.mark.live smoke test hits the real reachable Ollama and is skipped by default.
3. Non-streaming now, streaming later — request/response; streaming is a named later phase landed with the client HTTP layer that can consume it.
4. Typed error on model failure — after one retry, /dm/narrate returns a non-200 (502) in the {"detail": {...}} family; the client owns the degraded-DM fallback on any non-200 (already specified in docs/canon-log.md). No server-side fallback prose.
5. Rendered digest into the prompt — narrator.md is the system prompt; code renders the canon log into a curated, labeled digest in the user message; no raw JSON, no numbers.
6. Tag handling stays client-side — the server returns raw prose (tags intact) and logs it; the client's TagExtractor strips for display and harvests [FACT] into its own canon log. §2 (client owns state) + DRY (one extractor).

Architecture — layered modules, thin handler

POST /dm/narrate  (main.py handler — thin)
  └─ valid_turn (EXISTING) → 422 envelope on a bad canon log
  └─ narrate.run(canon_log):
       system  = prompts.system_prompt("narrator")        # narrator.md body (the IP)
       user    = prompts.render_digest(canon_log)          # curated labeled digest (§11 context)
       cfg     = routing.for_role("narrator")              # model + options (+ per-call seed) + think flag
       text    = ollama_client.chat(cfg.model, [system, user], cfg.options, think=cfg.think)  # 1 retry, else ModelError
       call_log.record(role, model, options, seed, messages, canon_log, text|error, latency_ms, ok)
       return text
  └─ 200 {"prose": text}   |   except ModelError → 502 {"detail": {"model_error": "<reason>"}}

ollama_client / routing / prompts / call_log are the shared pipeline; narrate.py is the first role service. Adding a role later is a new service module + prompt + output parsing — the four shared modules are reused untouched. Mocks enter the system at the ollama_client boundary.

The prompt — source code (§16)

narrator.md system-prompt body (authored in this plan)

The file already carries role/voice header; this plan writes the instruction body. It must encode:

• Task: describe the current scene from the supplied context, second person, present tense. Scenes, outcomes, transitions only.
• Voice (§3): dry; gritty, not grim; plays the world straight; never winks; nothing is "hilariously" anything; profanity earned, not decorative.
• Hard constraints (enforced by wording — output is prose, not a schema):
  • Never invent a proper noun (person/place/thing) without emitting [FACT: <the fact>] so the client can harvest it (§11).
  • Never contradict an Established fact in the supplied context.
  • Never state a number, stat, HP, or Luck value; never decide or narrate the player's choices or actions (code owns those — describe the world, not the protagonist's will).
  • Output prose only, plus zero or more [FACT:] tags. No JSON, no headers, no meta-talk.
  • Brevity — a few sentences (§14, small-model discipline).

render_digest(canon_log) -> str — a curated projection

Turns the posted log into a labeled block for the user message:

Location: the Greywater docks
Recent events:
- Arrived at Greywater by barge before dawn
- Brannoc recognised the harbourmaster and went quiet
Established facts:
- the eastern bridge out of Greywater is washed out
- the harbourmaster is named Oda Fenn
Companions present: Brannoc Thane, Cadwyn Vell
Active quest: The Missing Ledger — Find who took Fenn's ledger
Fortune: Fortune spits on you
Player: Aldric, a sellsword

Describe the scene as it is now.

The digest is a projection, not a dump. It surfaces narrative context and deliberately omits raw integers: companion disposition values render as names only, luck_descriptor renders as flavor ("Fortune: …", never the number, §7), schema_version/ids are dropped. This generalizes the §7 rule — the Narrator gets nouns and situation, never anything to do arithmetic on. Deterministic and unit-testable (assert the rendered string).

Model routing (§4/§5)

# routing.py
@dataclass(frozen=True)
class RoleConfig:
    model: str
    options: dict            # Ollama generation options, includes a per-call seed at call time
    think: bool | None = None  # Ollama top-level `think` flag; None omits it

ROLES = {
    "narrator": RoleConfig(
        model=env("OLLAMA_NARRATOR_MODEL", "qwen3.5:latest"),   # §5 "the good model" — swappable per deploy
        options={"temperature": 0.8, "top_p": 0.9, "num_predict": 300},   # voice; capped length (§14)
        think=False,   # qwen3.x thinking OFF — no <think> preamble in narrator prose
    ),
}
def for_role(role: str) -> RoleConfig: ...

• Base URL from OLLAMA_BASE_URL (already in .env.example).
• Model and options are config, not law — swapping the Narrator model is an env change, exactly §4's "a deploy, not a client patch." qwen3.5:latest is the default (local models available: qwen3.5, llama3.1, llama3.2).
• think: False (qwen3.x thinking disabled): qwen3.5 is a thinking model; without this it emits a <think>…</think> preamble that would pollute the narrator prose. ollama_client passes think as a top-level /api/chat field (not inside options) when RoleConfig.think is not None. Verified in the 2026-07-09 probe: think:false yields an empty thinking field and no leakage, and is gracefully accepted by non-thinking models too (llama3.1/3.2 return normally) — so it is safe across an env model swap.
• Per-call seed (§10): the server sets an explicit options.seed per call (a random int, captured in the log) so any call is replayable against a candidate model — the "free eval infrastructure" of §4. Without setting it, Ollama's own random seed is unrecoverable.
• Provider abstraction (Ollama → Replicate) is deferred (YAGNI). routing.py is the seam it slots into later; today, Ollama only.

Model probe (2026-07-09)

Warm, seeded, narrator-shaped call (system + digest, num_predict:300) against the reachable Ollama:

model	warm latency	tok/s	notes
qwen3.5 (think:false)	~2.0s	~107	richest prose; only model to emit [FACT:]; no <think> leak
llama3.1	~1.2s	~149	solid 2nd-person prose
llama3.2	~0.5s	~295	fastest, coherent, shorter

Decision: qwen3.5 is the default (§5 "the good model") — ~2s non-streaming is acceptable for the POC and streaming hides it later. llama3.1/llama3.2 remain one-env-var swaps if latency needs to drop; the speed data is on record for that tuning call.

Failure, retry, error contract (§12/§13)

ollama_client.chat() owns the retry; callers see success-or-ModelError.

Retry (exactly one, no backoff — §12 "never retry more than once, latency compounds") triggers on:

• transport error (connection refused / DNS),
• timeout (OLLAMA_TIMEOUT_SECONDS, default ~30s),
• non-2xx from Ollama,
• empty / whitespace-only content (the one "bad output" case; no JSON/tag parse to fail for prose).

Flow: attempt → on any of the above, retry once → still failing → raise ModelError(reason).

Handler mapping:

200  {"prose": text}                                   # success only
502  {"detail": {"model_error": "<short reason>"}}      # ModelError — same {"detail":{...}} family as the 422

The client contract is already satisfied — docs/canon-log.md says the degraded-DM fallback fires on any non-200, so the client treats 422 and 502 identically. The model_error string is for logs/dev, never shown to the player. No authored fallback prose on the server (decision 4).

Call logging (§10) — the eval infrastructure

call_log.py writes one structured JSON object per line, every call, success and failure — charter §10's "log the seed and the full prompt with every call" and §4's "replay yesterday's real traffic against a candidate model."

{
  "ts": "2026-07-09T…Z",
  "role": "narrator",
  "model": "llama3.1:latest",
  "options": { "temperature": 0.8, "top_p": 0.9, "num_predict": 300, "seed": 481923 },
  "messages": [ {"role":"system","content":"…narrator.md…"}, {"role":"user","content":"…digest…"} ],
  "canon_log": { "…the posted log…": true },
  "ok": true,
  "response": "You stand on the rot-black wharf… [FACT: …]",
  "latency_ms": 1840
}

On failure: "ok": false, "error": "<reason>", no response.

• Everything needed to replay is in the line — canon_log + messages + model + seed.
• Sink: JSON-lines to stdout by default (12-factor; captured by Docker/fly platform logs), redirectable to a file via CALL_LOG_PATH for local replay. The writer takes an injectable sink so tests capture and assert the record.
• No secrets to redact — the client never sends keys (§4), the canon log carries no PII, and the prompt is exactly what we want logged.

Testing

Unit — httpx MockTransport, deterministic, no new dependency:

• ollama_client: success parses content; retry (fail-once → recovers; fail-twice → ModelError); timeout → ModelError; non-2xx → retry → ModelError; empty content → retry → ModelError. Asserts the outgoing payload (model, messages, stream:false, options incl. seed, and top-level think present only when set).
• prompts: render_digest on canon_log_valid.json — correct labeled block, dispositions omitted, luck as descriptor-not-number; system_prompt("narrator") loads the file.
• routing: for_role returns the configured model + options; env override respected.
• call_log: writes a parseable JSON line with every field to the injected sink; failure-record shape.
• endpoint (TestClient, Ollama boundary mocked): valid log → 200 {"prose": …}; ModelError → 502 envelope; invalid log → still 422 (existing behavior intact).

Live smoke — @pytest.mark.live, skipped by default (registered marker; run with -m live or an env flag): hits the real reachable Ollama, asserts 200 + non-empty prose of sane length (no tag assertion — nondeterministic). Run during implementation to prove the wire.

Phases (milestones)

• Phase 1 — Pipeline foundation: ollama_client, routing, call_log, prompts (loader + digest), each mock/unit-tested. Endpoint untouched.
• Phase 2 — Narrator online: author narrator.md body → narrate service → wire /dm/narrate → endpoint tests (mock) + gated live smoke green. "Prove the loop" on the server side.

Storage & config

api/app/
  ollama_client.py  routing.py  prompts.py  call_log.py  narrate.py
  main.py           (modified — /dm/narrate wired)
api/prompts/
  narrator.md       (body authored)
api/tests/
  test_ollama_client.py  test_routing.py  test_prompts.py
  test_call_log.py       test_narrate_endpoint.py  test_live_smoke.py (gated)

New env (add to .env.example): OLLAMA_NARRATOR_MODEL, OLLAMA_TIMEOUT_SECONDS, CALL_LOG_PATH (optional). OLLAMA_BASE_URL already present.

Consequences / follow-ups

• The client HTTP plan consumes {"prose": …} / the 502 envelope, runs TagExtractor, harvests [FACT], and displays the degraded-DM fallback on non-200.
• The streaming plan adds a streaming mode to ollama_client + an SSE/chunked response, landed with the client stream consumer.
• The other four roles reuse ollama_client/routing/prompts/call_log, adding their own service + prompt + output parsing (the Adjudicator adds strict-JSON parsing + the §12 one-retry-with-error-appended path).
• Replicate provider slots into routing.py; auth/metering are middleware, retrofit later (§4).

Open questions

None blocking. The Narrator model default and generation options (temperature/length) are tuning values, seeded as env-overridable defaults. Prompt wording is authored and reviewed in Phase 2.