Add PDF→markdown batch converter and research-library workflow

convert.py walks pdfs/ (recursing topic subfolders), mirrors a .md tree
into md/ via pymupdf4llm, idempotent on mtime. Detects no-text-layer PDFs
(needs-ocr.txt) and falls back to plain per-page text when pymupdf4llm's
layout pass returns near-empty despite a real text layer.

Pin pymupdf4llm==0.3.4 (lightweight line; 1.27.x bundles an ML/OCR
pipeline that fails on plain text PDFs). PDFs gitignored (copyrighted,
large) — only generated markdown is committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

README.md

@@ -0,0 +1,77 @@
+# Research library: PDF → markdown for Claude-assisted synthesis
+
+Local, git-tracked workflow. Drop text PDFs into topic folders, convert them to
+markdown, then have Claude Code **read whole files** to cross-reference and
+synthesize across a topic — instead of chunked RAG, which gave shallow results.
+
+## Layout
+
+```
+demonology/
+  pdfs/<topic>/*.pdf     # source PDFs (gitignored — kept local, not committed)
+  md/<topic>/*.md        # converted markdown — the files Claude reads
+  convert.py             # batch converter
+  requirements.txt       # pins pymupdf4llm
+  needs-ocr.txt          # generated: PDFs with no text layer (gitignored)
+  README.md
+```
+
+Group PDFs into topic subfolders under `pdfs/` (e.g. `pdfs/angelology/`). The
+converter mirrors that structure into `md/`. A flat `pdfs/` (no subfolders) works
+too — it just produces a flat `md/`. Currently all PDFs sit directly in `pdfs/`.
+
+> **PDFs are gitignored.** They are large and copyrighted, so only the generated
+> markdown is committed. Keep your PDFs backed up outside git. To version the
+> PDFs too, remove `pdfs/` from `.gitignore` (consider git-lfs first).
+
+## Setup
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r requirements.txt
+```
+
+## Convert
+
+```bash
+source .venv/bin/activate
+python convert.py                 # pdfs/ -> md/, idempotent
+python convert.py --force         # reconvert everything
+python convert.py --src other --out other-md
+```
+
+Behavior:
+- **Recurses** `pdfs/` and mirrors the folder structure into `md/`.
+- **Idempotent**: skips a PDF whose `.md` exists and is newer than the PDF.
+- **Scan detection**: PDFs with ~no extractable text are logged to
+  `needs-ocr.txt` and left unconverted (no empty markdown) — see Fallbacks.
+- **Plain-text fallback**: on some PDFs pymupdf4llm's layout pass emits almost
+  nothing despite a real text layer. When its output is implausibly small versus
+  the raw extractable text, `convert.py` falls back to plain per-page text
+  (same `-----` page separators, marked `[plain-text fallback]` in the log).
+  Structure (headings/tables) is lost but the text is not.
+- Prints a summary: converted / skipped / flagged-for-OCR (/ failed).
+
+## Using it with Claude Code
+
+Per topic, ask things like:
+
+> "Read everything under `md/demonology/` and cross-reference the documents to
+> produce <specific outcome>, then save the result as a markdown file in that
+> folder."
+
+The markdown keeps headings, lists, tables, and page boundaries (`-----`
+separators) so Claude can cite locations while reading entire files.
+
+## Fallbacks
+
+`convert.py` uses **pymupdf4llm** (fast, no ML deps, best for clean text PDFs).
+If a PDF lands in `needs-ocr.txt`, or converts poorly (garbled tables/layout),
+use a heavier tool on just that file:
+
+- **scanned / no text layer** → `marker-pdf` or `docling` (OCR + layout).
+- **DOCX/PPTX/XLSX/HTML** sources → `markitdown`.
+
+Install on demand (see commented lines in `requirements.txt`), convert the
+problem file, and drop the result into the matching `md/<topic>/` path.