From 959abc79ebb91f0b88916256bc263b2cb0bda97e Mon Sep 17 00:00:00 2001 From: Phillip Tarrant Date: Thu, 9 Jul 2026 15:58:03 -0500 Subject: [PATCH] docs(spec): Narrator/Ollama pipeline (server) design MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Shared server-side model-call pipeline (Ollama client, role->model routing, prompt loader + canon-log digest, §10 call logging, retry->typed 502) wired through /dm/narrate as the first role. Non-streaming; server returns raw prose, client extracts tags. Two phases: pipeline foundation, Narrator online. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...6-07-09-narrator-ollama-pipeline-design.md | 205 ++++++++++++++++++ 1 file changed, 205 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-09-narrator-ollama-pipeline-design.md diff --git a/docs/superpowers/specs/2026-07-09-narrator-ollama-pipeline-design.md b/docs/superpowers/specs/2026-07-09-narrator-ollama-pipeline-design.md new file mode 100644 index 0000000..7868b20 --- /dev/null +++ b/docs/superpowers/specs/2026-07-09-narrator-ollama-pipeline-design.md @@ -0,0 +1,205 @@ +# Narrator / Ollama Pipeline (server) — Design + +**Date:** 2026-07-09 +**Status:** approved (design); implementation plan to follow +**Charter refs:** §2 (code owns state / AI owns text), §3 (tone/voice), §4 (the guarding proxy, model routing as config, logging from day one), §5 (the DM is not one prompt; Narrator = the good model), §10 (seed + full prompt logged), §11 (canon log), §12 (output contracts, tags, retry policy), §13 (degraded DM), §14 (cost/latency), §16 (prompts are source code) +**Binds to:** the canon log contract (`docs/canon-log.md`, `docs/schemas/canon-log.schema.json`) and the existing proxy skeleton (`api/app/main.py` — role endpoints + `valid_turn` + unified 422). Consumed by the later client HTTP loop. + +## Problem + +The proxy validates every posted canon log but every role endpoint still `return {"detail": "not implemented"}` — no model is wired, no prompt is authored, nothing is logged. Until a real model sits behind `/dm/narrate`, the POC's one question — *does bounded AI dialogue feel alive?* (charter §17) — cannot be tested at all. + +This plan makes `/dm/narrate` real: assemble the Narrator prompt + a canon-log digest, call Ollama, return the prose, log the call, and signal failure honestly — building the reusable server-side model-call pipeline that the other four roles later drop into. + +## Scope + +**In scope — the shared pipeline + the Narrator as its first, proving role:** + +- `ollama_client.py` — httpx call to Ollama `/api/chat` (non-streaming), single retry, typed `ModelError`. +- `routing.py` — role → `{model, options}` config, env-driven (§4/§5). +- `prompts.py` — load a role's `.md` system prompt (cached) + `render_digest(canon_log)`. +- `call_log.py` — structured JSON-lines logging of every call (§10). +- `narrate.py` — the Narrator service: render → route → call → log → return. +- The authored `narrator.md` system-prompt body (source code, §16). +- `/dm/narrate` wired to the service; `main.py` handler stays thin. +- Unit tests (mocked Ollama boundary) + one gated live smoke test. + +**Out of scope — later plans (recorded in `docs/roadmap.md`):** + +- Streaming (§14) — non-streaming request/response now; the Ollama client is shaped so a streaming mode is additive, not a rewrite. +- The client HTTP layer, degraded-DM fallback display, and `[FACT]` harvest (the client already owns `TagExtractor`). +- The other four roles (Adjudicator / Improviser / NPC / Banter). +- The Replicate provider; auth / metering. + +## Decisions (locked in brainstorming) + +1. **Shared pipeline + Narrator first** — build the reusable machinery once; the Narrator proves it; the other roles become thin drop-ins. +2. **Test: mocked boundary + gated live smoke** — unit tests use httpx `MockTransport` (no new dependency); a `@pytest.mark.live` smoke test hits the real reachable Ollama and is skipped by default. +3. **Non-streaming now, streaming later** — request/response; streaming is a named later phase landed with the client HTTP layer that can consume it. +4. **Typed error on model failure** — after one retry, `/dm/narrate` returns a non-200 (`502`) in the `{"detail": {...}}` family; the client owns the degraded-DM fallback on any non-200 (already specified in `docs/canon-log.md`). No server-side fallback prose. +5. **Rendered digest into the prompt** — `narrator.md` is the system prompt; code renders the canon log into a curated, labeled digest in the user message; no raw JSON, no numbers. +6. **Tag handling stays client-side** — the server returns raw prose (tags intact) and logs it; the client's `TagExtractor` strips for display and harvests `[FACT]` into its own canon log. §2 (client owns state) + DRY (one extractor). + +## Architecture — layered modules, thin handler + +``` +POST /dm/narrate (main.py handler — thin) + └─ valid_turn (EXISTING) → 422 envelope on a bad canon log + └─ narrate.run(canon_log): + system = prompts.system_prompt("narrator") # narrator.md body (the IP) + user = prompts.render_digest(canon_log) # curated labeled digest (§11 context) + cfg = routing.for_role("narrator") # model + options (+ per-call seed) + text = ollama_client.chat(cfg.model, [system, user], cfg.options) # 1 retry, else ModelError + call_log.record(role, model, options, seed, messages, canon_log, text|error, latency_ms, ok) + return text + └─ 200 {"prose": text} | except ModelError → 502 {"detail": {"model_error": ""}} +``` + +`ollama_client` / `routing` / `prompts` / `call_log` are the shared pipeline; `narrate.py` is the first role service. Adding a role later is a new service module + prompt + output parsing — the four shared modules are reused untouched. Mocks enter the system at the `ollama_client` boundary. + +## The prompt — source code (§16) + +### `narrator.md` system-prompt body (authored in this plan) + +The file already carries role/voice header; this plan writes the instruction body. It must encode: + +- **Task:** describe the *current* scene from the supplied context, second person, present tense. Scenes, outcomes, transitions only. +- **Voice (§3):** dry; gritty, not grim; plays the world straight; never winks; nothing is "hilariously" anything; profanity earned, not decorative. +- **Hard constraints** (enforced by wording — output is prose, not a schema): + - Never invent a proper noun (person/place/thing) without emitting `[FACT: ]` so the client can harvest it (§11). + - Never contradict an Established fact in the supplied context. + - Never state a number, stat, HP, or Luck value; never decide or narrate the **player's** choices or actions (code owns those — describe the world, not the protagonist's will). + - Output prose only, plus zero or more `[FACT:]` tags. No JSON, no headers, no meta-talk. + - Brevity — a few sentences (§14, small-model discipline). + +### `render_digest(canon_log) -> str` — a curated projection + +Turns the posted log into a labeled block for the user message: + +``` +Location: the Greywater docks +Recent events: +- Arrived at Greywater by barge before dawn +- Brannoc recognised the harbourmaster and went quiet +Established facts: +- the eastern bridge out of Greywater is washed out +- the harbourmaster is named Oda Fenn +Companions present: Brannoc Thane, Cadwyn Vell +Active quest: The Missing Ledger — Find who took Fenn's ledger +Fortune: Fortune spits on you +Player: Aldric, a sellsword + +Describe the scene as it is now. +``` + +**The digest is a projection, not a dump.** It surfaces narrative context and deliberately omits raw integers: companion `disposition` values render as names only, `luck_descriptor` renders as flavor ("Fortune: …", never the number, §7), `schema_version`/ids are dropped. This generalizes the §7 rule — the Narrator gets nouns and situation, never anything to do arithmetic on. Deterministic and unit-testable (assert the rendered string). + +## Model routing (§4/§5) + +```python +# routing.py +@dataclass(frozen=True) +class RoleConfig: + model: str + options: dict # Ollama generation options, includes a per-call seed at call time + +ROLES = { + "narrator": RoleConfig( + model=env("OLLAMA_NARRATOR_MODEL", "llama3.1:latest"), # §5 "the good model" — swappable per deploy + options={"temperature": 0.8, "top_p": 0.9, "num_predict": 300}, # voice; capped length (§14) + ), +} +def for_role(role: str) -> RoleConfig: ... +``` + +- Base URL from `OLLAMA_BASE_URL` (already in `.env.example`). +- Model and options are **config, not law** — swapping the Narrator model is an env change, exactly §4's "a deploy, not a client patch." `llama3.1:latest` is a starting default (local models available: `llama3.1`, `llama3.2`, `qwen3.5`). +- **Per-call seed (§10):** the server sets an explicit `options.seed` per call (a random int, captured in the log) so any call is replayable against a candidate model — the "free eval infrastructure" of §4. Without setting it, Ollama's own random seed is unrecoverable. +- **Provider abstraction (Ollama → Replicate) is deferred** (YAGNI). `routing.py` is the seam it slots into later; today, Ollama only. + +## Failure, retry, error contract (§12/§13) + +`ollama_client.chat()` owns the retry; callers see success-or-`ModelError`. + +Retry (exactly one, no backoff — §12 "never retry more than once, latency compounds") triggers on: +- transport error (connection refused / DNS), +- timeout (`OLLAMA_TIMEOUT_SECONDS`, default ~30s), +- non-2xx from Ollama, +- empty / whitespace-only content (the one "bad output" case; no JSON/tag parse to fail for prose). + +Flow: attempt → on any of the above, retry once → still failing → raise `ModelError(reason)`. + +Handler mapping: +``` +200 {"prose": text} # success only +502 {"detail": {"model_error": ""}} # ModelError — same {"detail":{...}} family as the 422 +``` + +The client contract is already satisfied — `docs/canon-log.md` says the degraded-DM fallback fires on **any non-200**, so the client treats 422 and 502 identically. The `model_error` string is for logs/dev, never shown to the player. No authored fallback prose on the server (decision 4). + +## Call logging (§10) — the eval infrastructure + +`call_log.py` writes one structured JSON object per line, every call, success and failure — charter §10's "log the seed and the full prompt with every call" and §4's "replay yesterday's real traffic against a candidate model." + +```json +{ + "ts": "2026-07-09T…Z", + "role": "narrator", + "model": "llama3.1:latest", + "options": { "temperature": 0.8, "top_p": 0.9, "num_predict": 300, "seed": 481923 }, + "messages": [ {"role":"system","content":"…narrator.md…"}, {"role":"user","content":"…digest…"} ], + "canon_log": { "…the posted log…": true }, + "ok": true, + "response": "You stand on the rot-black wharf… [FACT: …]", + "latency_ms": 1840 +} +``` + +On failure: `"ok": false, "error": ""`, no `response`. + +- Everything needed to replay is in the line — `canon_log` + `messages` + `model` + `seed`. +- **Sink:** JSON-lines to **stdout by default** (12-factor; captured by Docker/fly platform logs), redirectable to a file via `CALL_LOG_PATH` for local replay. The writer takes an injectable sink so tests capture and assert the record. +- No secrets to redact — the client never sends keys (§4), the canon log carries no PII, and the prompt is exactly what we want logged. + +## Testing + +**Unit — httpx `MockTransport`, deterministic, no new dependency:** + +- `ollama_client`: success parses content; retry (fail-once → recovers; fail-twice → `ModelError`); timeout → `ModelError`; non-2xx → retry → `ModelError`; empty content → retry → `ModelError`. Asserts the outgoing payload (model, `messages`, `stream:false`, options incl. `seed`). +- `prompts`: `render_digest` on `canon_log_valid.json` — correct labeled block, dispositions omitted, luck as descriptor-not-number; `system_prompt("narrator")` loads the file. +- `routing`: `for_role` returns the configured model + options; env override respected. +- `call_log`: writes a parseable JSON line with every field to the injected sink; failure-record shape. +- endpoint (`TestClient`, Ollama boundary mocked): valid log → 200 `{"prose": …}`; `ModelError` → 502 envelope; invalid log → still 422 (existing behavior intact). + +**Live smoke — `@pytest.mark.live`, skipped by default** (registered marker; run with `-m live` or an env flag): hits the real reachable Ollama, asserts 200 + non-empty prose of sane length (no tag assertion — nondeterministic). Run during implementation to prove the wire. + +## Phases (milestones) + +- **Phase 1 — Pipeline foundation:** `ollama_client`, `routing`, `call_log`, `prompts` (loader + digest), each mock/unit-tested. Endpoint untouched. +- **Phase 2 — Narrator online:** author `narrator.md` body → `narrate` service → wire `/dm/narrate` → endpoint tests (mock) + gated live smoke green. "Prove the loop" on the server side. + +## Storage & config + +``` +api/app/ + ollama_client.py routing.py prompts.py call_log.py narrate.py + main.py (modified — /dm/narrate wired) +api/prompts/ + narrator.md (body authored) +api/tests/ + test_ollama_client.py test_routing.py test_prompts.py + test_call_log.py test_narrate_endpoint.py test_live_smoke.py (gated) +``` + +New env (add to `.env.example`): `OLLAMA_NARRATOR_MODEL`, `OLLAMA_TIMEOUT_SECONDS`, `CALL_LOG_PATH` (optional). `OLLAMA_BASE_URL` already present. + +## Consequences / follow-ups + +- The client HTTP plan consumes `{"prose": …}` / the 502 envelope, runs `TagExtractor`, harvests `[FACT]`, and displays the degraded-DM fallback on non-200. +- The streaming plan adds a streaming mode to `ollama_client` + an SSE/chunked response, landed with the client stream consumer. +- The other four roles reuse `ollama_client`/`routing`/`prompts`/`call_log`, adding their own service + prompt + output parsing (the Adjudicator adds strict-JSON parsing + the §12 one-retry-with-error-appended path). +- Replicate provider slots into `routing.py`; auth/metering are middleware, retrofit later (§4). + +## Open questions + +None blocking. The Narrator model default and generation options (temperature/length) are tuning values, seeded as env-overridable defaults. Prompt wording is authored and reviewed in Phase 2.