Add PDF→markdown batch converter and research-library workflow
convert.py walks pdfs/ (recursing topic subfolders), mirrors a .md tree into md/ via pymupdf4llm, idempotent on mtime. Detects no-text-layer PDFs (needs-ocr.txt) and falls back to plain per-page text when pymupdf4llm's layout pass returns near-empty despite a real text layer. Pin pymupdf4llm==0.3.4 (lightweight line; 1.27.x bundles an ML/OCR pipeline that fails on plain text PDFs). PDFs gitignored (copyrighted, large) — only generated markdown is committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
13
.gitignore
vendored
Normal file
13
.gitignore
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
# Source PDFs kept local only — copyrighted, large. Commit markdown, not PDFs.
|
||||
pdfs/
|
||||
|
||||
# Generated OCR worklist (regenerated each run)
|
||||
needs-ocr.txt
|
||||
|
||||
# Claude Code session/harness state
|
||||
.claude/
|
||||
|
||||
# Python
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
Reference in New Issue
Block a user