ptarrant
2ea33cff61
Introduce a machine-readable layer on top of the markdown corpus so AI/scripts can query a topic's facts without re-reading whole sources (anti-RAG stays for synthesis/quotes). - md/demonology/demons.json: fact-cache, 33 entities attested in 2+ sources, each with rank/domain/signs/origins + provenance (sources, citations). - md/demonology/demons.schema.json: JSON Schema for the dataset. - md/demonology/INDEX.md: topic front-door (query JSON -> synthesis -> source). - validate.py: generic schema + house-rule validator (source_count, cross_refs, unique ids); discovers <name>.schema.json/<name>.json pairs across all topics. - docs/data-convention.md: the reusable, topic-agnostic pattern + how to add it to a new topic. - CLAUDE.md: pointer so the convention is picked up every session. - requirements.txt: add jsonschema (used by validate.py). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
16 lines
750 B
Plaintext
16 lines
750 B
Plaintext
# Primary converter for clean text PDFs -> LLM-oriented markdown.
|
|
# Pulls pymupdf + tabulate transitively; no separate pin needed.
|
|
# NOTE: pinned to the lightweight 0.3.x line on purpose. The 1.27.x releases
|
|
# bundle an ML layout/OCR pipeline (onnxruntime + Tesseract) that fails on plain
|
|
# text PDFs without a tessdata install and pulls heavy deps we don't want here.
|
|
pymupdf4llm==0.3.4
|
|
|
|
# Validates topic datasets (md/<topic>/<name>.json) against their JSON Schemas.
|
|
# Used by validate.py; lightweight, pure-Python.
|
|
jsonschema>=4.0
|
|
|
|
# Fallbacks (install only if needed, see README):
|
|
# markitdown # DOCX/PPTX/XLSX/HTML -> md
|
|
# marker-pdf # heavier, ML/GPU, OCRs scanned PDFs
|
|
# docling # heavier, ML/GPU, messy layouts/tables
|