From d5f10e4bfbc999764668882ddd2d5d5f8e6c4223 Mon Sep 17 00:00:00 2001 From: Phillip Tarrant Date: Tue, 21 Oct 2025 23:14:21 -0500 Subject: [PATCH] updating plans and save points --- punchlist.md | 297 --------------------------------------------------- 1 file changed, 297 deletions(-) delete mode 100644 punchlist.md diff --git a/punchlist.md b/punchlist.md deleted file mode 100644 index 0995dc5..0000000 --- a/punchlist.md +++ /dev/null @@ -1,297 +0,0 @@ -# Phase 0 – Baseline + Decisions (Quick) - -**Goals** - -* Confirm we’ll keep one Jinja template with palette includes. -* Choose PDF engine flags and file layout. - -**Decisions** - -* PDF engine: `wkhtmltopdf` via `subprocess` (wrapper optional). Decide if we want pdfkit -* Page size: **Letter** (or A4 if you prefer). -* Output layout: `/data/output///report.pdf`. -* ZIP per run: `/data/output/reports_.zip`. - -**Deliverables** - -* Short `DECISIONS.md` capturing above. -* “Current state” run that still only outputs HTML (for baseline timings). - -**Acceptance** - -* We can state in one sentence where PDFs will land and how they’re named. - ---- - -# Phase 1 – HTML → PDF Renderer (Additive) - -**Goals** - -* Convert your already-rendered HTML into a PDF file per scan config. - -**Tasks** - -* Create `report_renderer.py` with: - - * `render_html(context) -> str` (calls existing Jinja template) - * `html_to_pdf(html, out_path, *, page="Letter", margin_mm=12, dpi=96) -> Path` -* Ensure local assets (e.g., logo) resolve without network. -* Add `run_repo_scan()` call: after HTML render, write PDF. - -**Deliverables** - -* One PDF per config alongside existing HTML. - -**Acceptance** - -* PDFs open cleanly (no missing fonts/boxes). -* Render time acceptable (<1–2s per average report on your host). - ---- - -# Phase 2 – Bundle + Manifest - -**Goals** - -* Zip all PDFs for a run and write an audit manifest. - -**Tasks** - -* Create `report_bundler.py` with: - - * `collect_pdfs(run_dir) -> List[Path]` - * `write_manifest(run_info, per_config_stats, out_path) -> Path` - * `make_zip(files, out_zip) -> Path` -* Manifest fields (per file and rollup): - - * `run_id, generated_at, timezone` - * Per config: `name, config_slug, total_hosts, ok_hosts, hosts_with_issues, pdf_path, pdf_sha256, duration_ms` - * Bundle: `zip_path, zip_size_bytes, pdf_count` -* Add SHA-256 hashing utility. - -**Deliverables** - -* `reports_.zip` -* `manifest_.json` - -**Acceptance** - -* ZIP opens; manifest accurately lists all PDFs with correct hashes. - ---- - -# Phase 3 – Email Output - -**Goals** - -* Email either the ZIP (preferred) or top-N PDFs when size overs threshold. - -**Tasks** - -* Create `report_emailer.py`: - - * `send_reports(summary, files, to, cc, subject, size_limit_mb)` - * Compose a concise plain-text body (global counts + path to ZIP). - * Size check: if ZIP > limit → attach top N “issue” PDFs, mention storage path for ZIP. -* Configure SMTP from environment (no secrets in code). - -**Deliverables** - -* Successful email with attachments on a sample run. - -**Acceptance** - -* Emails received by `email_to` with intended attachments. -* Logs include message id / delivery status (if available). - ---- - -# Phase 4 – Unify Templates (Light/Dark via palette) - -**Goals** - -* One structural template; palette includes for dark/light. - -**Tasks** - -* Introduce `_palette_dark.j2` / `_palette_light.j2`. -* Replace hardcoded colors with palette tokens. -* Keep **inline styles** for email/PDF reliability. -* Ensure `reporting.dark_mode` drives palette include. - -**Deliverables** - -* Single `report.html.j2` + two small palette partials. - -**Acceptance** - -* Dark/Light flips correctly; visual parity with previous separate templates. - ---- - -# Phase 5 – Config Flags & CLI ergonomics - -**Goals** - -* Make format and emailing behavior configurable per run. - -**Tasks** - -* New config/env flags: - - * `OUTPUT_HTML=true|false` - * `OUTPUT_PDF=true|false` - * `BUNDLE_ZIP=true|false` - * `EMAIL_ENABLED=true|false` - * `EMAIL_SIZE_LIMIT_MB=15` -* Optional CLI args override envs for ad-hoc runs. - -**Deliverables** - -* Documented flags in `README` or `DECISIONS.md`. - -**Acceptance** - -* Toggling flags changes outputs without code edits. - ---- - -# Phase 6 – Logging Overhaul (Structured, Useful) - -**Goals** - -* Replace scattered prints with consistent, structured logs you can search. - -**Tasks** - -* Adopt one logger (e.g., `structlog` or `logging` with JSON formatter). -* Standard fields on every log line: - - * `run_id`, `config_name`, `config_slug`, `phase`, `duration_ms` -* Log events: - - * Start/end per config, counts (ok/issues), output paths, sizes, hashes. - * PDF render timings and failures (with `exc_info=True`). - * Bundling: zip path/size; Email: recipients, attachment count, result. -* Set sane defaults: - - * Level INFO in prod, DEBUG when `DEBUG=1`. - * Rotate or daily-split logs; permissions via umask `0027`. - -**Deliverables** - -* Consistent logs; sample snippet documented in `LOGGING.md`. - -**Acceptance** - -* You can answer “what happened to corp-wan at 03:05?” from logs alone. - ---- - -# Phase 7 – Docker & Dependencies - -**Goals** - -* Bake the toolchain into the image cleanly. - -**Tasks** - -* `apt-get install -y wkhtmltopdf fonts-dejavu-core` -* Keep pip deps minimal (Jinja2 already present). -* Health check: small mock HTML → PDF during build or startup test (optional). - -**Deliverables** - -* Rebuilt image; size acceptable. - -**Acceptance** - -* Container can render PDFs in a clean runtime (no missing libraries). - ---- - -# Phase 8 – Validation & Load Testing - -**Goals** - -* Confidence under multiple configs and larger reports. - -**Tasks** - -* Run with N configs (e.g., 10–20) and capture: - - * Total runtime, average PDF time, largest PDF size. -* Edge tests: - - * No targets → still produce a PDF with “No hosts”. - * All OK vs many Issues. - * Long hostnames and port lists (layout wrapping). -* Verify email size behavior. - -**Deliverables** - -* Short `QA_NOTES.md` with timings and observations. - -**Acceptance** - -* No crashes; PDFs legible; email strategy holds at size thresholds. - ---- - -# Phase 9 – Docs & Ops - -**Goals** - -* Make future you (or anyone) productive in 5 minutes. - -**Tasks** - -* `README` updates: flow diagram, flags, outputs. -* `DECISIONS.md` finalized. -* Sample manifest + how to verify checksums. -* Runbook snippet: “How to re-send last run’s ZIP.” - -**Deliverables** - -* Docs committed and discoverable. - -**Acceptance** - -* You can hand this to a teammate and they’ll ship a report same day. - ---- - -# Phase 10 – Nice-to-Haves / Backlog - -* “Issues-only” PDF variant (short brief for exec/IT). -* Per-config S3 upload (instead of email attachments). -* HTML preview endpoint for a single report (if you add a tiny web UI). -* Retention policy job: purge artifacts >30 days. -* Watermark/status banner for partial/incomplete scans. -* Per-run metrics export (Prometheus counters/gauges). - ---- - -## Global Punch List (copy/paste into your tracker) - -* [ ] Phase 0: Lock decisions (page size, paths, file naming) -* [ ] Phase 1: Add renderer module and wire PDF creation in `run_repo_scan()` -* [ ] Phase 2: Add bundler + manifest (with SHA-256) -* [ ] Phase 3: Add email sender with size-aware attachment strategy -* [ ] Phase 4: Merge templates; add palette partials; test light/dark -* [ ] Phase 5: Add env/CLI flags for outputs & email -* [ ] Phase 6: Overhaul logging (structured, consistent fields, timings) -* [ ] Phase 7: Update Dockerfile (wkhtmltopdf, fonts), rebuild, smoke test -* [ ] Phase 8: Load test with multiple configs; capture timings; fix edge cases -* [ ] Phase 9: Update README/DECISIONS/LOGGING; add runbook snippet -* [ ] Phase 10: (Optional) Implement backlog items - ---- - -## Quick win(s) to start next session - -* Pick **Letter vs A4** and the exact output path format. -* Decide whether to **keep writing HTML to disk** (handy for debugging) or only in memory. -* Choose **structlog vs stdlib logging + JSON** (I can tailor either). - -When you’re ready, we can start with Phase 1, and I’ll draft the exact renderer function signatures and logging fields so it slots right into your style.