docs(spec): Narrator/Ollama pipeline (server) design
Shared server-side model-call pipeline (Ollama client, role->model routing, prompt loader + canon-log digest, §10 call logging, retry->typed 502) wired through /dm/narrate as the first role. Non-streaming; server returns raw prose, client extracts tags. Two phases: pipeline foundation, Narrator online. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,205 @@
|
||||
# Narrator / Ollama Pipeline (server) — Design
|
||||
|
||||
**Date:** 2026-07-09
|
||||
**Status:** approved (design); implementation plan to follow
|
||||
**Charter refs:** §2 (code owns state / AI owns text), §3 (tone/voice), §4 (the guarding proxy, model routing as config, logging from day one), §5 (the DM is not one prompt; Narrator = the good model), §10 (seed + full prompt logged), §11 (canon log), §12 (output contracts, tags, retry policy), §13 (degraded DM), §14 (cost/latency), §16 (prompts are source code)
|
||||
**Binds to:** the canon log contract (`docs/canon-log.md`, `docs/schemas/canon-log.schema.json`) and the existing proxy skeleton (`api/app/main.py` — role endpoints + `valid_turn` + unified 422). Consumed by the later client HTTP loop.
|
||||
|
||||
## Problem
|
||||
|
||||
The proxy validates every posted canon log but every role endpoint still `return {"detail": "not implemented"}` — no model is wired, no prompt is authored, nothing is logged. Until a real model sits behind `/dm/narrate`, the POC's one question — *does bounded AI dialogue feel alive?* (charter §17) — cannot be tested at all.
|
||||
|
||||
This plan makes `/dm/narrate` real: assemble the Narrator prompt + a canon-log digest, call Ollama, return the prose, log the call, and signal failure honestly — building the reusable server-side model-call pipeline that the other four roles later drop into.
|
||||
|
||||
## Scope
|
||||
|
||||
**In scope — the shared pipeline + the Narrator as its first, proving role:**
|
||||
|
||||
- `ollama_client.py` — httpx call to Ollama `/api/chat` (non-streaming), single retry, typed `ModelError`.
|
||||
- `routing.py` — role → `{model, options}` config, env-driven (§4/§5).
|
||||
- `prompts.py` — load a role's `.md` system prompt (cached) + `render_digest(canon_log)`.
|
||||
- `call_log.py` — structured JSON-lines logging of every call (§10).
|
||||
- `narrate.py` — the Narrator service: render → route → call → log → return.
|
||||
- The authored `narrator.md` system-prompt body (source code, §16).
|
||||
- `/dm/narrate` wired to the service; `main.py` handler stays thin.
|
||||
- Unit tests (mocked Ollama boundary) + one gated live smoke test.
|
||||
|
||||
**Out of scope — later plans (recorded in `docs/roadmap.md`):**
|
||||
|
||||
- Streaming (§14) — non-streaming request/response now; the Ollama client is shaped so a streaming mode is additive, not a rewrite.
|
||||
- The client HTTP layer, degraded-DM fallback display, and `[FACT]` harvest (the client already owns `TagExtractor`).
|
||||
- The other four roles (Adjudicator / Improviser / NPC / Banter).
|
||||
- The Replicate provider; auth / metering.
|
||||
|
||||
## Decisions (locked in brainstorming)
|
||||
|
||||
1. **Shared pipeline + Narrator first** — build the reusable machinery once; the Narrator proves it; the other roles become thin drop-ins.
|
||||
2. **Test: mocked boundary + gated live smoke** — unit tests use httpx `MockTransport` (no new dependency); a `@pytest.mark.live` smoke test hits the real reachable Ollama and is skipped by default.
|
||||
3. **Non-streaming now, streaming later** — request/response; streaming is a named later phase landed with the client HTTP layer that can consume it.
|
||||
4. **Typed error on model failure** — after one retry, `/dm/narrate` returns a non-200 (`502`) in the `{"detail": {...}}` family; the client owns the degraded-DM fallback on any non-200 (already specified in `docs/canon-log.md`). No server-side fallback prose.
|
||||
5. **Rendered digest into the prompt** — `narrator.md` is the system prompt; code renders the canon log into a curated, labeled digest in the user message; no raw JSON, no numbers.
|
||||
6. **Tag handling stays client-side** — the server returns raw prose (tags intact) and logs it; the client's `TagExtractor` strips for display and harvests `[FACT]` into its own canon log. §2 (client owns state) + DRY (one extractor).
|
||||
|
||||
## Architecture — layered modules, thin handler
|
||||
|
||||
```
|
||||
POST /dm/narrate (main.py handler — thin)
|
||||
└─ valid_turn (EXISTING) → 422 envelope on a bad canon log
|
||||
└─ narrate.run(canon_log):
|
||||
system = prompts.system_prompt("narrator") # narrator.md body (the IP)
|
||||
user = prompts.render_digest(canon_log) # curated labeled digest (§11 context)
|
||||
cfg = routing.for_role("narrator") # model + options (+ per-call seed)
|
||||
text = ollama_client.chat(cfg.model, [system, user], cfg.options) # 1 retry, else ModelError
|
||||
call_log.record(role, model, options, seed, messages, canon_log, text|error, latency_ms, ok)
|
||||
return text
|
||||
└─ 200 {"prose": text} | except ModelError → 502 {"detail": {"model_error": "<reason>"}}
|
||||
```
|
||||
|
||||
`ollama_client` / `routing` / `prompts` / `call_log` are the shared pipeline; `narrate.py` is the first role service. Adding a role later is a new service module + prompt + output parsing — the four shared modules are reused untouched. Mocks enter the system at the `ollama_client` boundary.
|
||||
|
||||
## The prompt — source code (§16)
|
||||
|
||||
### `narrator.md` system-prompt body (authored in this plan)
|
||||
|
||||
The file already carries role/voice header; this plan writes the instruction body. It must encode:
|
||||
|
||||
- **Task:** describe the *current* scene from the supplied context, second person, present tense. Scenes, outcomes, transitions only.
|
||||
- **Voice (§3):** dry; gritty, not grim; plays the world straight; never winks; nothing is "hilariously" anything; profanity earned, not decorative.
|
||||
- **Hard constraints** (enforced by wording — output is prose, not a schema):
|
||||
- Never invent a proper noun (person/place/thing) without emitting `[FACT: <the fact>]` so the client can harvest it (§11).
|
||||
- Never contradict an Established fact in the supplied context.
|
||||
- Never state a number, stat, HP, or Luck value; never decide or narrate the **player's** choices or actions (code owns those — describe the world, not the protagonist's will).
|
||||
- Output prose only, plus zero or more `[FACT:]` tags. No JSON, no headers, no meta-talk.
|
||||
- Brevity — a few sentences (§14, small-model discipline).
|
||||
|
||||
### `render_digest(canon_log) -> str` — a curated projection
|
||||
|
||||
Turns the posted log into a labeled block for the user message:
|
||||
|
||||
```
|
||||
Location: the Greywater docks
|
||||
Recent events:
|
||||
- Arrived at Greywater by barge before dawn
|
||||
- Brannoc recognised the harbourmaster and went quiet
|
||||
Established facts:
|
||||
- the eastern bridge out of Greywater is washed out
|
||||
- the harbourmaster is named Oda Fenn
|
||||
Companions present: Brannoc Thane, Cadwyn Vell
|
||||
Active quest: The Missing Ledger — Find who took Fenn's ledger
|
||||
Fortune: Fortune spits on you
|
||||
Player: Aldric, a sellsword
|
||||
|
||||
Describe the scene as it is now.
|
||||
```
|
||||
|
||||
**The digest is a projection, not a dump.** It surfaces narrative context and deliberately omits raw integers: companion `disposition` values render as names only, `luck_descriptor` renders as flavor ("Fortune: …", never the number, §7), `schema_version`/ids are dropped. This generalizes the §7 rule — the Narrator gets nouns and situation, never anything to do arithmetic on. Deterministic and unit-testable (assert the rendered string).
|
||||
|
||||
## Model routing (§4/§5)
|
||||
|
||||
```python
|
||||
# routing.py
|
||||
@dataclass(frozen=True)
|
||||
class RoleConfig:
|
||||
model: str
|
||||
options: dict # Ollama generation options, includes a per-call seed at call time
|
||||
|
||||
ROLES = {
|
||||
"narrator": RoleConfig(
|
||||
model=env("OLLAMA_NARRATOR_MODEL", "llama3.1:latest"), # §5 "the good model" — swappable per deploy
|
||||
options={"temperature": 0.8, "top_p": 0.9, "num_predict": 300}, # voice; capped length (§14)
|
||||
),
|
||||
}
|
||||
def for_role(role: str) -> RoleConfig: ...
|
||||
```
|
||||
|
||||
- Base URL from `OLLAMA_BASE_URL` (already in `.env.example`).
|
||||
- Model and options are **config, not law** — swapping the Narrator model is an env change, exactly §4's "a deploy, not a client patch." `llama3.1:latest` is a starting default (local models available: `llama3.1`, `llama3.2`, `qwen3.5`).
|
||||
- **Per-call seed (§10):** the server sets an explicit `options.seed` per call (a random int, captured in the log) so any call is replayable against a candidate model — the "free eval infrastructure" of §4. Without setting it, Ollama's own random seed is unrecoverable.
|
||||
- **Provider abstraction (Ollama → Replicate) is deferred** (YAGNI). `routing.py` is the seam it slots into later; today, Ollama only.
|
||||
|
||||
## Failure, retry, error contract (§12/§13)
|
||||
|
||||
`ollama_client.chat()` owns the retry; callers see success-or-`ModelError`.
|
||||
|
||||
Retry (exactly one, no backoff — §12 "never retry more than once, latency compounds") triggers on:
|
||||
- transport error (connection refused / DNS),
|
||||
- timeout (`OLLAMA_TIMEOUT_SECONDS`, default ~30s),
|
||||
- non-2xx from Ollama,
|
||||
- empty / whitespace-only content (the one "bad output" case; no JSON/tag parse to fail for prose).
|
||||
|
||||
Flow: attempt → on any of the above, retry once → still failing → raise `ModelError(reason)`.
|
||||
|
||||
Handler mapping:
|
||||
```
|
||||
200 {"prose": text} # success only
|
||||
502 {"detail": {"model_error": "<short reason>"}} # ModelError — same {"detail":{...}} family as the 422
|
||||
```
|
||||
|
||||
The client contract is already satisfied — `docs/canon-log.md` says the degraded-DM fallback fires on **any non-200**, so the client treats 422 and 502 identically. The `model_error` string is for logs/dev, never shown to the player. No authored fallback prose on the server (decision 4).
|
||||
|
||||
## Call logging (§10) — the eval infrastructure
|
||||
|
||||
`call_log.py` writes one structured JSON object per line, every call, success and failure — charter §10's "log the seed and the full prompt with every call" and §4's "replay yesterday's real traffic against a candidate model."
|
||||
|
||||
```json
|
||||
{
|
||||
"ts": "2026-07-09T…Z",
|
||||
"role": "narrator",
|
||||
"model": "llama3.1:latest",
|
||||
"options": { "temperature": 0.8, "top_p": 0.9, "num_predict": 300, "seed": 481923 },
|
||||
"messages": [ {"role":"system","content":"…narrator.md…"}, {"role":"user","content":"…digest…"} ],
|
||||
"canon_log": { "…the posted log…": true },
|
||||
"ok": true,
|
||||
"response": "You stand on the rot-black wharf… [FACT: …]",
|
||||
"latency_ms": 1840
|
||||
}
|
||||
```
|
||||
|
||||
On failure: `"ok": false, "error": "<reason>"`, no `response`.
|
||||
|
||||
- Everything needed to replay is in the line — `canon_log` + `messages` + `model` + `seed`.
|
||||
- **Sink:** JSON-lines to **stdout by default** (12-factor; captured by Docker/fly platform logs), redirectable to a file via `CALL_LOG_PATH` for local replay. The writer takes an injectable sink so tests capture and assert the record.
|
||||
- No secrets to redact — the client never sends keys (§4), the canon log carries no PII, and the prompt is exactly what we want logged.
|
||||
|
||||
## Testing
|
||||
|
||||
**Unit — httpx `MockTransport`, deterministic, no new dependency:**
|
||||
|
||||
- `ollama_client`: success parses content; retry (fail-once → recovers; fail-twice → `ModelError`); timeout → `ModelError`; non-2xx → retry → `ModelError`; empty content → retry → `ModelError`. Asserts the outgoing payload (model, `messages`, `stream:false`, options incl. `seed`).
|
||||
- `prompts`: `render_digest` on `canon_log_valid.json` — correct labeled block, dispositions omitted, luck as descriptor-not-number; `system_prompt("narrator")` loads the file.
|
||||
- `routing`: `for_role` returns the configured model + options; env override respected.
|
||||
- `call_log`: writes a parseable JSON line with every field to the injected sink; failure-record shape.
|
||||
- endpoint (`TestClient`, Ollama boundary mocked): valid log → 200 `{"prose": …}`; `ModelError` → 502 envelope; invalid log → still 422 (existing behavior intact).
|
||||
|
||||
**Live smoke — `@pytest.mark.live`, skipped by default** (registered marker; run with `-m live` or an env flag): hits the real reachable Ollama, asserts 200 + non-empty prose of sane length (no tag assertion — nondeterministic). Run during implementation to prove the wire.
|
||||
|
||||
## Phases (milestones)
|
||||
|
||||
- **Phase 1 — Pipeline foundation:** `ollama_client`, `routing`, `call_log`, `prompts` (loader + digest), each mock/unit-tested. Endpoint untouched.
|
||||
- **Phase 2 — Narrator online:** author `narrator.md` body → `narrate` service → wire `/dm/narrate` → endpoint tests (mock) + gated live smoke green. "Prove the loop" on the server side.
|
||||
|
||||
## Storage & config
|
||||
|
||||
```
|
||||
api/app/
|
||||
ollama_client.py routing.py prompts.py call_log.py narrate.py
|
||||
main.py (modified — /dm/narrate wired)
|
||||
api/prompts/
|
||||
narrator.md (body authored)
|
||||
api/tests/
|
||||
test_ollama_client.py test_routing.py test_prompts.py
|
||||
test_call_log.py test_narrate_endpoint.py test_live_smoke.py (gated)
|
||||
```
|
||||
|
||||
New env (add to `.env.example`): `OLLAMA_NARRATOR_MODEL`, `OLLAMA_TIMEOUT_SECONDS`, `CALL_LOG_PATH` (optional). `OLLAMA_BASE_URL` already present.
|
||||
|
||||
## Consequences / follow-ups
|
||||
|
||||
- The client HTTP plan consumes `{"prose": …}` / the 502 envelope, runs `TagExtractor`, harvests `[FACT]`, and displays the degraded-DM fallback on non-200.
|
||||
- The streaming plan adds a streaming mode to `ollama_client` + an SSE/chunked response, landed with the client stream consumer.
|
||||
- The other four roles reuse `ollama_client`/`routing`/`prompts`/`call_log`, adding their own service + prompt + output parsing (the Adjudicator adds strict-JSON parsing + the §12 one-retry-with-error-appended path).
|
||||
- Replicate provider slots into `routing.py`; auth/metering are middleware, retrofit later (§4).
|
||||
|
||||
## Open questions
|
||||
|
||||
None blocking. The Narrator model default and generation options (temperature/length) are tuning values, seeded as env-overridable defaults. Prompt wording is authored and reviewed in Phase 2.
|
||||
Reference in New Issue
Block a user