diff --git a/README.md b/README.md index 7ec01f0..7fbcc6f 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,54 @@ -# Research library: PDF → markdown for Claude-assisted synthesis +# My AI-Assisted Research Center -Local, git-tracked workflow. Drop text PDFs into topic folders, convert them to -markdown, then have Claude Code **read whole files** to cross-reference and -synthesize across a topic — instead of chunked RAG, which gave shallow results. +A local, **git-backed** research workspace where I collect source material by +topic, convert it to clean markdown, and have **Claude Code read whole files** to +cross-reference and synthesize across a subject — instead of chunked RAG, which +gave shallow results. + +Everything that matters is versioned in git: the converted markdown, the +conversion tooling, and any syntheses Claude produces. Source PDFs stay local +(see below). The repo is the durable record; Claude is the analyst working over +it. + +**Topics:** TBD and growing. The primary one today is **demonology**. The layout +is topic-per-folder so the same workflow scales to any subject — add a folder, +drop in sources, convert, ask. + +## How it works + +1. **Collect** — drop text PDFs into `pdfs//`. +2. **Convert** — `python convert.py` mirrors them into `md//` as markdown + (headings, lists, tables, and `-----` page boundaries preserved). +3. **Synthesize** — point Claude Code at `md//` and ask it to read the + whole topic and produce something. It saves the result back into the folder. +4. **Commit** — markdown sources and syntheses go into git. ## Layout ``` -demonology/ +research-center/ pdfs//*.pdf # source PDFs (gitignored — kept local, not committed) - md//*.md # converted markdown — the files Claude reads - convert.py # batch converter - requirements.txt # pins pymupdf4llm + md//*.md # converted markdown + saved syntheses — what Claude reads + convert.py # batch PDF→markdown converter + requirements.txt # pins the converter (pymupdf4llm) needs-ocr.txt # generated: PDFs with no text layer (gitignored) README.md ``` -Group PDFs into topic subfolders under `pdfs/` (e.g. `pdfs/angelology/`). The -converter mirrors that structure into `md/`. A flat `pdfs/` (no subfolders) works -too — it just produces a flat `md/`. Current topics: `demonology/`. Add more by -creating a new `pdfs//` and dropping PDFs in. +Group sources into topic subfolders under `pdfs/` (e.g. `pdfs/demonology/`, +`pdfs/alchemy/`). The converter mirrors that structure into `md/`. A flat +`pdfs/` (no subfolders) works too — it just produces a flat `md/`. -> **PDFs are gitignored.** They are large and copyrighted, so only the generated -> markdown is committed. Keep your PDFs backed up outside git. To version the -> PDFs too, remove `pdfs/` from `.gitignore` (consider git-lfs first). +**Add a new topic:** + +```bash +mkdir pdfs/ # drop PDFs in +python convert.py # converts only the new files into md// +``` + +> **PDFs are gitignored.** They are large and often copyrighted, so only the +> generated markdown is committed. Keep your PDFs backed up outside git. To +> version the PDFs too, remove `pdfs/` from `.gitignore` (consider git-lfs first). ## Setup @@ -63,7 +88,9 @@ Per topic, ask things like: > folder." The markdown keeps headings, lists, tables, and page boundaries (`-----` -separators) so Claude can cite locations while reading entire files. +separators) so Claude can cite locations while reading entire files. Syntheses +Claude writes land alongside the sources in `md//` and get committed too — +so the research center accumulates both raw material and worked analysis. ## Fallbacks