Add per-topic structured fact-cache pattern, validator, and docs
Introduce a machine-readable layer on top of the markdown corpus so AI/scripts can query a topic's facts without re-reading whole sources (anti-RAG stays for synthesis/quotes). - md/demonology/demons.json: fact-cache, 33 entities attested in 2+ sources, each with rank/domain/signs/origins + provenance (sources, citations). - md/demonology/demons.schema.json: JSON Schema for the dataset. - md/demonology/INDEX.md: topic front-door (query JSON -> synthesis -> source). - validate.py: generic schema + house-rule validator (source_count, cross_refs, unique ids); discovers <name>.schema.json/<name>.json pairs across all topics. - docs/data-convention.md: the reusable, topic-agnostic pattern + how to add it to a new topic. - CLAUDE.md: pointer so the convention is picked up every session. - requirements.txt: add jsonschema (used by validate.py). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
73
md/demonology/INDEX.md
Normal file
73
md/demonology/INDEX.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Demonology — Topic Index
|
||||
|
||||
**Read this file first.** It tells an AI agent what exists for this topic and
|
||||
which artifact to use, so you don't re-read the full source corpus (~3.4 MB)
|
||||
unless you actually need a direct quote.
|
||||
|
||||
## Decision order for an agent
|
||||
|
||||
1. **Need a fact, attribute, rank, or to filter/sort entities?**
|
||||
→ Query `demons.json`. Do **not** read the sources. It is the fact-cache.
|
||||
2. **Need the narrative, the argument, or how the sources disagree?**
|
||||
→ Read `demon-hierarchy-synthesis.md` (prose).
|
||||
3. **Need a verbatim quote or a detail not in the data?**
|
||||
→ Open the one source `.md` named in the entity's `citations`, at the cited
|
||||
page/line. Read that file whole (repo convention), not a snippet.
|
||||
|
||||
## Artifacts
|
||||
|
||||
| File | Type | Use it for |
|
||||
|------|------|-----------|
|
||||
| `demons.json` | structured data (33 entities) | Querying attributes: rank, legions, domain, signs, sin, source_count, cross_refs. The machine source of truth for facts. |
|
||||
| `demons.schema.json` | JSON Schema | Field definitions, allowed values, validation. Read to learn the data shape. |
|
||||
| `demon-hierarchy-synthesis.md` | prose synthesis | The ranked hierarchy as narrative + the recurrence map + where sources disagree. |
|
||||
| *(6 source `.md` files)* | converted sources | Direct quotes / details beyond the dataset. Listed under `sources` in `demons.json`. |
|
||||
|
||||
## What's in `demons.json`
|
||||
|
||||
- **Scope:** only entities attested in **2+ of the 6 sources** (33 entities).
|
||||
Single-source names (most of the Goetia 72, the world-folklore long tail) are
|
||||
intentionally excluded — see the synthesis for why.
|
||||
- **Tiers:** `supreme-prince` (11), `directional-king` (4), `goetia-officer` (2),
|
||||
`watcher` (7), `other-recurring` (9).
|
||||
- **Source codes:** `FG` Field Guide · `DM` Dark Mirrors · `PU` Personal/Impersonal
|
||||
(Barth/Unger) · `SP` Spiritism · `DJ` Daemonologie · `EN` Encyclopedia. Full
|
||||
descriptions in `demons.json` → `sources`.
|
||||
- **Aliases are merged** to a primary name; `cross_refs` link conflated/related
|
||||
entities (e.g. `satan` ↔ `lucifer` ↔ `satanael`; `asmodeus` ↔ `aeshma`/`ahriman`).
|
||||
|
||||
## Example queries (jq)
|
||||
|
||||
```bash
|
||||
cd md/demonology
|
||||
# All Kings attested in 2+ sources
|
||||
jq -r '.demons[] | select(.rank != null and (.rank|test("King"))) | .name' demons.json
|
||||
# The seven Princes of Hell by deadly sin
|
||||
jq '.demons[] | select(.deadly_sin) | {name, deadly_sin}' demons.json
|
||||
# Everything naming the Watcher myth (Dark Mirrors)
|
||||
jq '.demons[] | select(.sources|index("DM")) | .name' demons.json
|
||||
# Most-corroborated entities first
|
||||
jq '[.demons[] | {name, source_count}] | sort_by(-.source_count)' demons.json
|
||||
```
|
||||
|
||||
## Keeping it in sync
|
||||
|
||||
`demons.json` is the source of truth for **facts**; the synthesis is the source of
|
||||
truth for **narrative**. If you add/correct an entity, update `demons.json` first,
|
||||
re-validate against the schema, then reflect it in the synthesis prose — not the
|
||||
other way around.
|
||||
|
||||
```bash
|
||||
# validate after editing (schema + house rules: source_count, cross_refs, unique ids)
|
||||
./.venv/bin/python validate.py demonology # this topic
|
||||
./.venv/bin/python validate.py # every topic in the repo
|
||||
```
|
||||
|
||||
The validator is generic: any topic that drops a `<name>.schema.json` next to its
|
||||
`<name>.json` is picked up automatically. Exit code is non-zero on failure, so it
|
||||
doubles as a git pre-commit hook or CI step.
|
||||
|
||||
To extend scope (e.g. add the full Goetia 72 or the folklore long tail), re-run
|
||||
the per-source extraction, set each record's `sources`/`source_count`
|
||||
accordingly, and relax the schema's `minItems: 2` on `sources` if you drop the
|
||||
2+ rule.
|
||||
Reference in New Issue
Block a user