# Narrator / Ollama Pipeline (server) — Design

**Date:** 2026-07-09
**Status:** approved (design); implementation plan to follow
**Charter refs:** §2 (code owns state / AI owns text), §3 (tone/voice), §4 (the guarding proxy, model routing as config, logging from day one), §5 (the DM is not one prompt; Narrator = the good model), §10 (seed + full prompt logged), §11 (canon log), §12 (output contracts, tags, retry policy), §13 (degraded DM), §14 (cost/latency), §16 (prompts are source code)
**Binds to:** the canon log contract (`docs/canon-log.md`, `docs/schemas/canon-log.schema.json`) and the existing proxy skeleton (`api/app/main.py` — role endpoints + `valid_turn` + unified 422). Consumed by the later client HTTP loop.

## Problem

The proxy validates every posted canon log but every role endpoint still `return {"detail": "not implemented"}` — no model is wired, no prompt is authored, nothing is logged. Until a real model sits behind `/dm/narrate`, the POC's one question — *does bounded AI dialogue feel alive?* (charter §17) — cannot be tested at all.

This plan makes `/dm/narrate` real: assemble the Narrator prompt + a canon-log digest, call Ollama, return the prose, log the call, and signal failure honestly — building the reusable server-side model-call pipeline that the other four roles later drop into.

## Scope

**In scope — the shared pipeline + the Narrator as its first, proving role:**

- `ollama_client.py` — httpx call to Ollama `/api/chat` (non-streaming), single retry, typed `ModelError`.
- `routing.py` — role → `{model, options}` config, env-driven (§4/§5).
- `prompts.py` — load a role's `.md` system prompt (cached) + `render_digest(canon_log)`.
- `call_log.py` — structured JSON-lines logging of every call (§10).
- `narrate.py` — the Narrator service: render → route → call → log → return.
- The authored `narrator.md` system-prompt body (source code, §16).
- `/dm/narrate` wired to the service; `main.py` handler stays thin.
- Unit tests (mocked Ollama boundary) + one gated live smoke test.

**Out of scope — later plans (recorded in `docs/roadmap.md`):**

- Streaming (§14) — non-streaming request/response now; the Ollama client is shaped so a streaming mode is additive, not a rewrite.
- The client HTTP layer, degraded-DM fallback display, and `[FACT]` harvest (the client already owns `TagExtractor`).
- The other four roles (Adjudicator / Improviser / NPC / Banter).
- The Replicate provider; auth / metering.

## Decisions (locked in brainstorming)

1. **Shared pipeline + Narrator first** — build the reusable machinery once; the Narrator proves it; the other roles become thin drop-ins.
2. **Test: mocked boundary + gated live smoke** — unit tests use httpx `MockTransport` (no new dependency); a `@pytest.mark.live` smoke test hits the real reachable Ollama and is skipped by default.
3. **Non-streaming now, streaming later** — request/response; streaming is a named later phase landed with the client HTTP layer that can consume it.
4. **Typed error on model failure** — after one retry, `/dm/narrate` returns a non-200 (`502`) in the `{"detail": {...}}` family; the client owns the degraded-DM fallback on any non-200 (already specified in `docs/canon-log.md`). No server-side fallback prose.
5. **Rendered digest into the prompt** — `narrator.md` is the system prompt; code renders the canon log into a curated, labeled digest in the user message; no raw JSON, no numbers.
6. **Tag handling stays client-side** — the server returns raw prose (tags intact) and logs it; the client's `TagExtractor` strips for display and harvests `[FACT]` into its own canon log. §2 (client owns state) + DRY (one extractor).

## Architecture — layered modules, thin handler

```
POST /dm/narrate  (main.py handler — thin)
  └─ valid_turn (EXISTING) → 422 envelope on a bad canon log
  └─ narrate.run(canon_log):
       system  = prompts.system_prompt("narrator")        # narrator.md body (the IP)
       user    = prompts.render_digest(canon_log)          # curated labeled digest (§11 context)
       cfg     = routing.for_role("narrator")              # model + options (+ per-call seed) + think flag
       text    = ollama_client.chat(cfg.model, [system, user], cfg.options, think=cfg.think)  # 1 retry, else ModelError
       call_log.record(role, model, options, seed, messages, canon_log, text|error, latency_ms, ok)
       return text
  └─ 200 {"prose": text}   |   except ModelError → 502 {"detail": {"model_error": "<reason>"}}
```

`ollama_client` / `routing` / `prompts` / `call_log` are the shared pipeline; `narrate.py` is the first role service. Adding a role later is a new service module + prompt + output parsing — the four shared modules are reused untouched. Mocks enter the system at the `ollama_client` boundary.

## The prompt — source code (§16)

### `narrator.md` system-prompt body (authored in this plan)

The file already carries role/voice header; this plan writes the instruction body. It must encode:

- **Task:** describe the *current* scene from the supplied context, second person, present tense. Scenes, outcomes, transitions only.
- **Voice (§3):** dry; gritty, not grim; plays the world straight; never winks; nothing is "hilariously" anything; profanity earned, not decorative.
- **Hard constraints** (enforced by wording — output is prose, not a schema):
  - Never invent a proper noun (person/place/thing) without emitting `[FACT: <the fact>]` so the client can harvest it (§11).
  - Never contradict an Established fact in the supplied context.
  - Never state a number, stat, HP, or Luck value; never decide or narrate the **player's** choices or actions (code owns those — describe the world, not the protagonist's will).
  - Output prose only, plus zero or more `[FACT:]` tags. No JSON, no headers, no meta-talk.
  - Brevity — a few sentences (§14, small-model discipline).

### `render_digest(canon_log) -> str` — a curated projection

Turns the posted log into a labeled block for the user message:

```
Location: the Greywater docks
Recent events:
- Arrived at Greywater by barge before dawn
- Brannoc recognised the harbourmaster and went quiet
Established facts:
- the eastern bridge out of Greywater is washed out
- the harbourmaster is named Oda Fenn
Companions present: Brannoc Thane, Cadwyn Vell
Active quest: The Missing Ledger — Find who took Fenn's ledger
Fortune: Fortune spits on you
Player: Aldric, a sellsword

Describe the scene as it is now.
```

**The digest is a projection, not a dump.** It surfaces narrative context and deliberately omits raw integers: companion `disposition` values render as names only, `luck_descriptor` renders as flavor ("Fortune: …", never the number, §7), `schema_version`/ids are dropped. This generalizes the §7 rule — the Narrator gets nouns and situation, never anything to do arithmetic on. Deterministic and unit-testable (assert the rendered string).

## Model routing (§4/§5)

```python
# routing.py
@dataclass(frozen=True)
class RoleConfig:
    model: str
    options: dict            # Ollama generation options, includes a per-call seed at call time
    think: bool | None = None  # Ollama top-level `think` flag; None omits it

ROLES = {
    "narrator": RoleConfig(
        model=env("OLLAMA_NARRATOR_MODEL", "qwen3.5:latest"),   # §5 "the good model" — swappable per deploy
        options={"temperature": 0.8, "top_p": 0.9, "num_predict": 300},   # voice; capped length (§14)
        think=False,   # qwen3.x thinking OFF — no <think> preamble in narrator prose
    ),
}
def for_role(role: str) -> RoleConfig: ...
```

- Base URL from `OLLAMA_BASE_URL` (already in `.env.example`).
- Model and options are **config, not law** — swapping the Narrator model is an env change, exactly §4's "a deploy, not a client patch." `qwen3.5:latest` is the default (local models available: `qwen3.5`, `llama3.1`, `llama3.2`).
- **`think: False` (qwen3.x thinking disabled):** qwen3.5 is a thinking model; without this it emits a `<think>…</think>` preamble that would pollute the narrator prose. `ollama_client` passes `think` as a **top-level** `/api/chat` field (not inside `options`) when `RoleConfig.think is not None`. Verified in the 2026-07-09 probe: `think:false` yields an empty `thinking` field and no leakage, and is **gracefully accepted by non-thinking models too** (llama3.1/3.2 return normally) — so it is safe across an env model swap.
- **Per-call seed (§10):** the server sets an explicit `options.seed` per call (a random int, captured in the log) so any call is replayable against a candidate model — the "free eval infrastructure" of §4. Without setting it, Ollama's own random seed is unrecoverable.
- **Provider abstraction (Ollama → Replicate) is deferred** (YAGNI). `routing.py` is the seam it slots into later; today, Ollama only.

### Model probe (2026-07-09)

Warm, seeded, narrator-shaped call (system + digest, `num_predict:300`) against the reachable Ollama:

| model | warm latency | tok/s | notes |
|---|---|---|---|
| **qwen3.5** (`think:false`) | ~2.0s | ~107 | richest prose; only model to emit `[FACT:]`; no `<think>` leak |
| llama3.1 | ~1.2s | ~149 | solid 2nd-person prose |
| llama3.2 | ~0.5s | ~295 | fastest, coherent, shorter |

Decision: **qwen3.5 is the default** (§5 "the good model") — ~2s non-streaming is acceptable for the POC and streaming hides it later. llama3.1/llama3.2 remain one-env-var swaps if latency needs to drop; the speed data is on record for that tuning call.

## Failure, retry, error contract (§12/§13)

`ollama_client.chat()` owns the retry; callers see success-or-`ModelError`.

Retry (exactly one, no backoff — §12 "never retry more than once, latency compounds") triggers on:
- transport error (connection refused / DNS),
- timeout (`OLLAMA_TIMEOUT_SECONDS`, default ~30s),
- non-2xx from Ollama,
- empty / whitespace-only content (the one "bad output" case; no JSON/tag parse to fail for prose).

Flow: attempt → on any of the above, retry once → still failing → raise `ModelError(reason)`.

Handler mapping:
```
200  {"prose": text}                                   # success only
502  {"detail": {"model_error": "<short reason>"}}      # ModelError — same {"detail":{...}} family as the 422
```

The client contract is already satisfied — `docs/canon-log.md` says the degraded-DM fallback fires on **any non-200**, so the client treats 422 and 502 identically. The `model_error` string is for logs/dev, never shown to the player. No authored fallback prose on the server (decision 4).

## Call logging (§10) — the eval infrastructure

`call_log.py` writes one structured JSON object per line, every call, success and failure — charter §10's "log the seed and the full prompt with every call" and §4's "replay yesterday's real traffic against a candidate model."

```json
{
  "ts": "2026-07-09T…Z",
  "role": "narrator",
  "model": "llama3.1:latest",
  "options": { "temperature": 0.8, "top_p": 0.9, "num_predict": 300, "seed": 481923 },
  "messages": [ {"role":"system","content":"…narrator.md…"}, {"role":"user","content":"…digest…"} ],
  "canon_log": { "…the posted log…": true },
  "ok": true,
  "response": "You stand on the rot-black wharf… [FACT: …]",
  "latency_ms": 1840
}
```

On failure: `"ok": false, "error": "<reason>"`, no `response`.

- Everything needed to replay is in the line — `canon_log` + `messages` + `model` + `seed`.
- **Sink:** JSON-lines to **stdout by default** (12-factor; captured by Docker/fly platform logs), redirectable to a file via `CALL_LOG_PATH` for local replay. The writer takes an injectable sink so tests capture and assert the record.
- No secrets to redact — the client never sends keys (§4), the canon log carries no PII, and the prompt is exactly what we want logged.

## Testing

**Unit — httpx `MockTransport`, deterministic, no new dependency:**

- `ollama_client`: success parses content; retry (fail-once → recovers; fail-twice → `ModelError`); timeout → `ModelError`; non-2xx → retry → `ModelError`; empty content → retry → `ModelError`. Asserts the outgoing payload (model, `messages`, `stream:false`, options incl. `seed`, and top-level `think` present only when set).
- `prompts`: `render_digest` on `canon_log_valid.json` — correct labeled block, dispositions omitted, luck as descriptor-not-number; `system_prompt("narrator")` loads the file.
- `routing`: `for_role` returns the configured model + options; env override respected.
- `call_log`: writes a parseable JSON line with every field to the injected sink; failure-record shape.
- endpoint (`TestClient`, Ollama boundary mocked): valid log → 200 `{"prose": …}`; `ModelError` → 502 envelope; invalid log → still 422 (existing behavior intact).

**Live smoke — `@pytest.mark.live`, skipped by default** (registered marker; run with `-m live` or an env flag): hits the real reachable Ollama, asserts 200 + non-empty prose of sane length (no tag assertion — nondeterministic). Run during implementation to prove the wire.

## Phases (milestones)

- **Phase 1 — Pipeline foundation:** `ollama_client`, `routing`, `call_log`, `prompts` (loader + digest), each mock/unit-tested. Endpoint untouched.
- **Phase 2 — Narrator online:** author `narrator.md` body → `narrate` service → wire `/dm/narrate` → endpoint tests (mock) + gated live smoke green. "Prove the loop" on the server side.

## Storage & config

```
api/app/
  ollama_client.py  routing.py  prompts.py  call_log.py  narrate.py
  main.py           (modified — /dm/narrate wired)
api/prompts/
  narrator.md       (body authored)
api/tests/
  test_ollama_client.py  test_routing.py  test_prompts.py
  test_call_log.py       test_narrate_endpoint.py  test_live_smoke.py (gated)
```

New env (add to `.env.example`): `OLLAMA_NARRATOR_MODEL`, `OLLAMA_TIMEOUT_SECONDS`, `CALL_LOG_PATH` (optional). `OLLAMA_BASE_URL` already present.

## Consequences / follow-ups

- The client HTTP plan consumes `{"prose": …}` / the 502 envelope, runs `TagExtractor`, harvests `[FACT]`, and displays the degraded-DM fallback on non-200.
- The streaming plan adds a streaming mode to `ollama_client` + an SSE/chunked response, landed with the client stream consumer.
- The other four roles reuse `ollama_client`/`routing`/`prompts`/`call_log`, adding their own service + prompt + output parsing (the Adjudicator adds strict-JSON parsing + the §12 one-retry-with-error-appended path).
- Replicate provider slots into `routing.py`; auth/metering are middleware, retrofit later (§4).

## Open questions

None blocking. The Narrator model default and generation options (temperature/length) are tuning values, seeded as env-overridable defaults. Prompt wording is authored and reviewed in Phase 2.
