Add PDF→markdown batch converter and research-library workflow

convert.py walks pdfs/ (recursing topic subfolders), mirrors a .md tree
into md/ via pymupdf4llm, idempotent on mtime. Detects no-text-layer PDFs
(needs-ocr.txt) and falls back to plain per-page text when pymupdf4llm's
layout pass returns near-empty despite a real text layer.

Pin pymupdf4llm==0.3.4 (lightweight line; 1.27.x bundles an ML/OCR
pipeline that fails on plain text PDFs). PDFs gitignored (copyrighted,
large) — only generated markdown is committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-26 15:24:22 -05:00
commit 057c96e10b
11 changed files with 83616 additions and 0 deletions

13
.gitignore vendored Normal file
View File

@@ -0,0 +1,13 @@
# Source PDFs kept local only — copyrighted, large. Commit markdown, not PDFs.
pdfs/
# Generated OCR worklist (regenerated each run)
needs-ocr.txt
# Claude Code session/harness state
.claude/
# Python
.venv/
__pycache__/
*.pyc