Add CLAUDE.md with operating conventions for Claude Code

Session guide: venv usage, convert.py workflow, whole-file synthesis
convention (read all md/<topic>/, save syntheses back in-folder), git
habits, and the pymupdf4llm pin + plain-text fallback gotchas.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-26 15:36:26 -05:00
parent 04010d5ee2
commit 5f0a542ab2

72
CLAUDE.md Normal file
View File

@@ -0,0 +1,72 @@
# CLAUDE.md
Operating guide for Claude Code in this repo. See `README.md` for the full
project description.
## What this is
An AI-assisted research center. Source material lives by topic, gets converted to
markdown, and **you read whole markdown files** to cross-reference and synthesize
across a topic. This is deliberately *not* chunked RAG — read entire files.
## Repo map
- `pdfs/<topic>/*.pdf` — source PDFs. **Gitignored** (large, often copyrighted). Local only.
- `md/<topic>/*.md` — converted markdown **and** syntheses you write. This is what you read.
- `convert.py` — batch PDF→markdown converter.
- `requirements.txt` — pins `pymupdf4llm==0.3.4` (lightweight line; see Gotchas).
- `needs-ocr.txt` — generated list of no-text-layer PDFs. Gitignored.
- `.venv/` — virtualenv. Gitignored.
## Environment
Always use the venv. It is already built.
```bash
source .venv/bin/activate # or call ./.venv/bin/python directly
```
If a package is missing, `pip install -r requirements.txt`.
## Converting PDFs
```bash
./.venv/bin/python convert.py # pdfs/ -> md/, idempotent (skips current files)
./.venv/bin/python convert.py --force # reconvert everything
```
- Idempotent on mtime — safe to rerun; only new/changed PDFs convert.
- Adding a topic: `mkdir pdfs/<topic>`, drop PDFs in, run `convert.py`.
- After converting, check the summary line and `needs-ocr.txt` for anything flagged.
## Doing synthesis (the main job)
When asked to synthesize a topic:
1. Read **every** file under `md/<topic>/` — full files, not snippets.
2. Cross-reference across documents; cite using page boundaries (`-----`
separators mark PDF pages) and the source name in each file's header.
3. Save the result as a new markdown file **in that topic folder**, e.g.
`md/<topic>/<slug>-synthesis.md`, unless told otherwise.
4. Syntheses are part of the record — they get committed.
## Git conventions
- Treat this repo as code: **everything goes in git** except what `.gitignore`
excludes (PDFs, `.venv/`, `.claude/`, `needs-ocr.txt`).
- Commit converted markdown and syntheses. Do **not** commit source PDFs.
- Commit when a unit of work is done (a topic converted, a synthesis written).
Don't push unless asked.
## Gotchas
- **pymupdf4llm is pinned to `0.3.4` on purpose.** The `1.27.x` releases bundle
an ML layout/OCR pipeline (onnxruntime + Tesseract) that fails on plain text
PDFs without a tessdata install. Don't "upgrade" it without testing a full
conversion pass.
- **Plain-text fallback:** on some PDFs pymupdf4llm emits near-empty markdown
despite a real text layer. `convert.py` detects this and falls back to plain
per-page text (logged `[plain-text fallback]`). Those files lose
heading/table structure but keep all text — still fine to read.
- **For poor conversions or scans:** use `marker-pdf`/`docling` (OCR + layout) or
`markitdown` (DOCX/PPTX/XLSX/HTML) on the single problem file, then drop the
result into the matching `md/<topic>/` path. See README → Fallbacks.