Add per-topic structured fact-cache pattern, validator, and docs

Introduce a machine-readable layer on top of the markdown corpus so AI/scripts
can query a topic's facts without re-reading whole sources (anti-RAG stays for
synthesis/quotes).

- md/demonology/demons.json: fact-cache, 33 entities attested in 2+ sources,
  each with rank/domain/signs/origins + provenance (sources, citations).
- md/demonology/demons.schema.json: JSON Schema for the dataset.
- md/demonology/INDEX.md: topic front-door (query JSON -> synthesis -> source).
- validate.py: generic schema + house-rule validator (source_count, cross_refs,
  unique ids); discovers <name>.schema.json/<name>.json pairs across all topics.
- docs/data-convention.md: the reusable, topic-agnostic pattern + how to add it
  to a new topic.
- CLAUDE.md: pointer so the convention is picked up every session.
- requirements.txt: add jsonschema (used by validate.py).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-28 19:56:54 -05:00
parent 8120079d99
commit 2ea33cff61
7 changed files with 1295 additions and 0 deletions

View File

@@ -49,6 +49,24 @@ When asked to synthesize a topic:
`md/<topic>/<slug>-synthesis.md`, unless told otherwise.
4. Syntheses are part of the record — they get committed.
## Structured data (fact-caches)
A topic may carry a machine-readable **fact-cache** alongside its markdown so you
can answer attribute questions without re-reading the corpus. Convention, per
topic folder `md/<topic>/`:
- `<name>.json` — facts, one record per entity, each with `sources` +
`citations` back to the source files.
- `<name>.schema.json` — JSON Schema; validates the sibling `<name>.json`.
- `INDEX.md` — read this **first**: query the `.json` for facts → the
`*-synthesis.md` for narrative → a source `.md` only for verbatim quotes.
Validate after any edit: `./.venv/bin/python validate.py <topic>` (or no arg for
the whole repo). Edit the **JSON first**, validate, then update prose.
Full pattern + how to add it to a new topic: **`docs/data-convention.md`**.
Worked example: `md/demonology/`.
## Git conventions
- Treat this repo as code: **everything goes in git** except what `.gitignore`