Add PDF→markdown batch converter and research-library workflow

convert.py walks pdfs/ (recursing topic subfolders), mirrors a .md tree
into md/ via pymupdf4llm, idempotent on mtime. Detects no-text-layer PDFs
(needs-ocr.txt) and falls back to plain per-page text when pymupdf4llm's
layout pass returns near-empty despite a real text layer.

Pin pymupdf4llm==0.3.4 (lightweight line; 1.27.x bundles an ML/OCR
pipeline that fails on plain text PDFs). PDFs gitignored (copyrighted,
large) — only generated markdown is committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

first_prompt.md

@@ -0,0 +1,62 @@
+# Project: PDF research library → markdown for Claude-assisted synthesis
+
+  ## Goal
+  I'm building a local, git-tracked research workflow. I have folders of **text PDFs**
+  (real text layer, not scans) grouped by topic. I want to convert them to markdown so
+  that you (Claude Code) can read the actual files and answer cross-reference / synthesis
+  questions across a topic — instead of using a RAG tool with a small local model, which
+  gave shallow results.
+
+  Treat this repo as code. Everything goes in git.
+
+  ## Tooling decision (already made — don't re-litigate unless something breaks)
+  - **Primary converter: `pymupdf4llm`** — pip install, no ML deps, fast, LLM-oriented
+    markdown (keeps headings/lists/tables, can mark page boundaries). Best fit for clean
+    text PDFs.
+  - Fallbacks if needed:
+    - `markitdown` (Microsoft) — if I also need DOCX/PPTX/XLSX/HTML → md.
+    - `marker` or `docling` — heavier, ML/GPU, **also OCR scanned PDFs** and handle messy
+      layouts/tables better. Use only for PDFs that pymupdf4llm handles poorly or that
+      turn out to be scans (no text layer).
+
+  ## What I want you to build
+  1. A small **batch converter** (`convert.py` or similar):
+     - Input: a source dir of PDFs (recursing into topic subfolders).
+     - Output: mirrored `.md` files under an output dir, preserving the topic folder
+  structure.
+     - **Idempotent**: skip a PDF if its `.md` already exists and is newer than the PDF.
+     - **Detect no-text-layer PDFs** (pymupdf4llm yields little/no text) and log them to a
+       `needs-ocr.txt` list instead of writing empty markdown — those need the marker/OCR
+  path.
+     - Print a summary: converted / skipped / flagged-for-ocr counts.
+  2. A `requirements.txt` pinning `pymupdf4llm` (and a venv setup note in the README).
+  3. A short `README.md` documenting the workflow and folder layout.
+
+  ## Suggested repo layout (adjust to what you find)
+  ```
+  research-library/
+    pdfs/<topic>/*.pdf        # source (consider git-lfs or .gitignore if large)
+    md/<topic>/*.md           # converted markdown — the stuff you'll read
+    convert.py
+    requirements.txt
+    needs-ocr.txt             # generated
+    README.md
+  ```
+  Recommend whether to commit the source PDFs (size?), git-lfs them, or gitignore them and
+  commit only the markdown. Ask me if it's a judgment call on size.
+
+  ## How I'll use it afterward
+  Per topic, I'll ask you things like: "Read everything under `md/<topic>/` and
+  cross-reference the documents to produce <specific outcome>, then save the result as a
+  markdown file in that folder." So optimize the markdown for *you* reading whole files,
+  not for chunked retrieval.
+
+  ## First steps for you
+  1. Look at the current folder/repo state and tell me what's here.
+  2. Confirm/adjust the layout above.
+  3. Write `convert.py`, `requirements.txt`, `README.md`.
+  4. Run a conversion pass and report the summary + anything flagged in `needs-ocr.txt`.
+
+  That'll get the desktop session productive immediately. When you're ready to actually do
+  synthesis, just point it at md/<topic>/ and ask — that's the part jarvis was failing at, and
+  it's exactly what reading-whole-files is good for.