ptarrant
057c96e10b
convert.py walks pdfs/ (recursing topic subfolders), mirrors a .md tree into md/ via pymupdf4llm, idempotent on mtime. Detects no-text-layer PDFs (needs-ocr.txt) and falls back to plain per-page text when pymupdf4llm's layout pass returns near-empty despite a real text layer. Pin pymupdf4llm==0.3.4 (lightweight line; 1.27.x bundles an ML/OCR pipeline that fails on plain text PDFs). PDFs gitignored (copyrighted, large) — only generated markdown is committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
12 lines
604 B
Plaintext
12 lines
604 B
Plaintext
# Primary converter for clean text PDFs -> LLM-oriented markdown.
|
|
# Pulls pymupdf + tabulate transitively; no separate pin needed.
|
|
# NOTE: pinned to the lightweight 0.3.x line on purpose. The 1.27.x releases
|
|
# bundle an ML layout/OCR pipeline (onnxruntime + Tesseract) that fails on plain
|
|
# text PDFs without a tessdata install and pulls heavy deps we don't want here.
|
|
pymupdf4llm==0.3.4
|
|
|
|
# Fallbacks (install only if needed, see README):
|
|
# markitdown # DOCX/PPTX/XLSX/HTML -> md
|
|
# marker-pdf # heavier, ML/GPU, OCRs scanned PDFs
|
|
# docling # heavier, ML/GPU, messy layouts/tables
|