adding Roadmap

2025-11-14 02:40:49 +00:00
parent 212596fa0a
commit e29c839d80
1 changed files with 851 additions and 0 deletions
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -0,0 +1,851 @@
+# SneakyScanner Roadmap
+
+## Vision & Goals
+
+SneakyScanner is evolving from a CLI-based network scanning tool into a comprehensive **Flask web application** for infrastructure monitoring and security auditing. The web application will provide:
+
+- **Centralized dashboard** for viewing scan history and trends
+- **Scheduled scanning** for continuous infrastructure monitoring
+- **Email notifications** for critical changes and certificate expirations
+- **Historical analysis** with charts, graphs, and comparison reports
+- **RESTful API** for integration and automation
+- **Simple deployment** using SQLite3 (single-user, self-hosted)
+
+The CLI scanner will evolve into an **API client**, maintaining backward compatibility while enabling web-based management and visualization.
+
+## Target Users
+
+- **Infrastructure teams** monitoring on-premises networks
+- **Security professionals** performing periodic security audits
+- **DevOps engineers** tracking infrastructure drift
+- **Single users or small teams** (not enterprise multi-tenant)
+
+## Technology Stack
+
+### Backend
+- **Flask** 3.x - Lightweight Python web framework
+- **SQLAlchemy** 2.x - ORM for database abstraction
+- **SQLite3** - Embedded database (easy deployment, sufficient for single-user)
+- **APScheduler** 3.x - Background job scheduling for periodic scans
+- **Flask-Login** - Simple session-based authentication
+- **Flask-CORS** - API access control (optional for CLI API client)
+- **Marshmallow** - API serialization/deserialization
+
+### Frontend
+- **Jinja2** - Server-side templating (already in use)
+- **Bootstrap 5** - Responsive UI framework with dark theme support
+- **Chart.js** 4.x - Lightweight charting library for trends
+- **DataTables.js** - Interactive sortable/filterable tables (Phase 6)
+- **Vanilla JavaScript** - Keep dependencies minimal
+
+### Infrastructure
+- **Docker Compose** - Multi-container orchestration (Flask app + Scanner)
+- **Gunicorn** - WSGI server for production
+- **Nginx** - Reverse proxy (optional, for production deployments)
+
+### Existing Components (Keep)
+- **Masscan** - Fast port discovery
+- **Nmap** - Service detection
+- **Playwright** - Screenshot capture
+- **sslyze** - SSL/TLS analysis
+
+## Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                     Flask Web Application                   │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
+│  │   Web UI     │  │   REST API   │  │   Scheduler      │  │
+│  │  (Dashboard) │  │ (JSON/CRUD)  │  │  (APScheduler)   │  │
+│  └──────┬───────┘  └──────┬───────┘  └────────┬─────────┘  │
+│         │                 │                    │            │
+│         └─────────────────┴────────────────────┘            │
+│                           │                                 │
+│                  ┌────────▼────────┐                        │
+│                  │   SQLAlchemy    │                        │
+│                  │   (ORM Layer)   │                        │
+│                  └────────┬────────┘                        │
+│                           │                                 │
+│                  ┌────────▼────────┐                        │
+│                  │   SQLite3 DB    │                        │
+│                  │  (scan history) │                        │
+│                  └─────────────────┘                        │
+└───────────────────────────┬─────────────────────────────────┘
+                            │
+                 ┌──────────▼──────────┐
+                 │  Scanner Engine     │
+                 │  (scanner.py)       │
+                 │  ┌────────────────┐ │
+                 │  │ Masscan/Nmap   │ │
+                 │  │ Playwright     │ │
+                 │  │ sslyze         │ │
+                 │  └────────────────┘ │
+                 └─────────────────────┘
+                            │
+                 ┌──────────▼──────────┐
+                 │  CLI API Client     │
+                 │  (optional future)  │
+                 └─────────────────────┘
+```
+
+### Component Interaction
+
+1. **Web UI** - User interacts with dashboard, triggers scans, views history
+2. **REST API** - Handles requests from web UI and CLI client
+3. **Scheduler (APScheduler)** - Triggers scans based on cron-like schedules
+4. **SQLAlchemy ORM** - Abstracts database operations
+5. **SQLite3 Database** - Stores scan results, schedules, settings, alerts
+6. **Scanner Engine** - Performs actual network scanning (masscan, nmap, etc.)
+7. **CLI API Client** - Future: thin client that calls Flask API
+
+## Database Schema Design
+
+### Core Tables
+
+#### `scans`
+Stores metadata about each scan execution.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique scan ID |
+| `timestamp` | DATETIME | Scan start time (UTC) |
+| `duration` | FLOAT | Total scan duration (seconds) |
+| `status` | VARCHAR(20) | `running`, `completed`, `failed` |
+| `config_file` | TEXT | Path to YAML config used |
+| `title` | TEXT | Scan title from config |
+| `json_path` | TEXT | Path to JSON report |
+| `html_path` | TEXT | Path to HTML report |
+| `zip_path` | TEXT | Path to ZIP archive |
+| `screenshot_dir` | TEXT | Path to screenshot directory |
+| `created_at` | DATETIME | Record creation time |
+| `triggered_by` | VARCHAR(50) | `manual`, `scheduled`, `api` |
+| `schedule_id` | INTEGER | FK to schedules (if triggered by schedule) |
+
+#### `scan_sites`
+Logical grouping of IPs by site.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique site record ID |
+| `scan_id` | INTEGER | FK to scans |
+| `site_name` | VARCHAR(255) | Site name from config |
+
+#### `scan_ips`
+IP addresses scanned in each scan.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique IP record ID |
+| `scan_id` | INTEGER | FK to scans |
+| `site_id` | INTEGER | FK to scan_sites |
+| `ip_address` | VARCHAR(45) | IPv4 or IPv6 address |
+| `ping_expected` | BOOLEAN | Expected ping response |
+| `ping_actual` | BOOLEAN | Actual ping response |
+
+#### `scan_ports`
+Discovered TCP/UDP ports.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique port record ID |
+| `scan_id` | INTEGER | FK to scans |
+| `ip_id` | INTEGER | FK to scan_ips |
+| `port` | INTEGER | Port number (1-65535) |
+| `protocol` | VARCHAR(10) | `tcp` or `udp` |
+| `expected` | BOOLEAN | Was this port expected? |
+| `state` | VARCHAR(20) | `open`, `closed`, `filtered` |
+
+#### `scan_services`
+Detected services on open ports.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique service record ID |
+| `scan_id` | INTEGER | FK to scans |
+| `port_id` | INTEGER | FK to scan_ports |
+| `service_name` | VARCHAR(100) | Service name (e.g., `ssh`, `http`) |
+| `product` | VARCHAR(255) | Product name (e.g., `OpenSSH`) |
+| `version` | VARCHAR(100) | Version string |
+| `extrainfo` | TEXT | Additional nmap info |
+| `ostype` | VARCHAR(100) | OS type if detected |
+| `http_protocol` | VARCHAR(10) | `http` or `https` (if web service) |
+| `screenshot_path` | TEXT | Relative path to screenshot |
+
+#### `scan_certificates`
+SSL/TLS certificates discovered on HTTPS services.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique certificate record ID |
+| `scan_id` | INTEGER | FK to scans |
+| `service_id` | INTEGER | FK to scan_services |
+| `subject` | TEXT | Certificate subject (CN) |
+| `issuer` | TEXT | Certificate issuer |
+| `serial_number` | TEXT | Serial number |
+| `not_valid_before` | DATETIME | Validity start date |
+| `not_valid_after` | DATETIME | Validity end date |
+| `days_until_expiry` | INTEGER | Days until expiration |
+| `sans` | TEXT | JSON array of SANs |
+| `is_self_signed` | BOOLEAN | Self-signed certificate flag |
+
+#### `scan_tls_versions`
+TLS version support and cipher suites.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique TLS version record ID |
+| `scan_id` | INTEGER | FK to scans |
+| `certificate_id` | INTEGER | FK to scan_certificates |
+| `tls_version` | VARCHAR(20) | `TLS 1.0`, `TLS 1.1`, `TLS 1.2`, `TLS 1.3` |
+| `supported` | BOOLEAN | Is this version supported? |
+| `cipher_suites` | TEXT | JSON array of cipher suites |
+
+### Scheduling & Notifications Tables
+
+#### `schedules`
+Scheduled scan configurations.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique schedule ID |
+| `name` | VARCHAR(255) | Schedule name (e.g., "Daily prod scan") |
+| `config_file` | TEXT | Path to YAML config |
+| `cron_expression` | VARCHAR(100) | Cron-like schedule (e.g., `0 2 * * *`) |
+| `enabled` | BOOLEAN | Is schedule active? |
+| `last_run` | DATETIME | Last execution time |
+| `next_run` | DATETIME | Next scheduled execution |
+| `created_at` | DATETIME | Schedule creation time |
+| `updated_at` | DATETIME | Last modification time |
+
+#### `alerts`
+Alert history and notifications sent.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique alert ID |
+| `scan_id` | INTEGER | FK to scans |
+| `alert_type` | VARCHAR(50) | `new_port`, `cert_expiry`, `service_change`, `ping_failed` |
+| `severity` | VARCHAR(20) | `info`, `warning`, `critical` |
+| `message` | TEXT | Human-readable alert message |
+| `ip_address` | VARCHAR(45) | Related IP (optional) |
+| `port` | INTEGER | Related port (optional) |
+| `email_sent` | BOOLEAN | Was email notification sent? |
+| `email_sent_at` | DATETIME | Email send timestamp |
+| `created_at` | DATETIME | Alert creation time |
+
+#### `alert_rules`
+User-defined alert rules.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique rule ID |
+| `rule_type` | VARCHAR(50) | `unexpected_port`, `cert_expiry`, `service_down`, etc. |
+| `enabled` | BOOLEAN | Is rule active? |
+| `threshold` | INTEGER | Threshold value (e.g., days for cert expiry) |
+| `email_enabled` | BOOLEAN | Send email for this rule? |
+| `created_at` | DATETIME | Rule creation time |
+
+### Settings Table
+
+#### `settings`
+Application configuration key-value store.
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `id` | INTEGER PRIMARY KEY | Unique setting ID |
+| `key` | VARCHAR(255) UNIQUE | Setting key (e.g., `smtp_server`) |
+| `value` | TEXT | Setting value (JSON for complex values) |
+| `updated_at` | DATETIME | Last modification time |
+
+**Example Settings:**
+- `smtp_server` - SMTP server hostname
+- `smtp_port` - SMTP port (587, 465, 25)
+- `smtp_username` - SMTP username
+- `smtp_password` - SMTP password (encrypted)
+- `smtp_from_email` - From email address
+- `smtp_to_emails` - JSON array of recipient emails
+- `app_password` - Single-user password hash (bcrypt)
+- `retention_days` - How long to keep old scans (0 = forever)
+
+## API Design
+
+### REST API Endpoints
+
+All API endpoints return JSON and follow RESTful conventions.
+
+#### Scans
+
+| Method | Endpoint | Description | Request Body | Response |
+|--------|----------|-------------|--------------|----------|
+| `GET` | `/api/scans` | List all scans (paginated) | - | `{ "scans": [...], "total": N, "page": 1 }` |
+| `GET` | `/api/scans/{id}` | Get scan details | - | `{ "scan": {...} }` |
+| `POST` | `/api/scans` | Trigger new scan | `{ "config_file": "path" }` | `{ "scan_id": N, "status": "running" }` |
+| `DELETE` | `/api/scans/{id}` | Delete scan and files | - | `{ "status": "deleted" }` |
+| `GET` | `/api/scans/{id}/status` | Get scan status | - | `{ "status": "running", "progress": "45%" }` |
+| `GET` | `/api/scans/{id1}/compare/{id2}` | Compare two scans | - | `{ "diff": {...} }` |
+
+#### Schedules
+
+| Method | Endpoint | Description | Request Body | Response |
+|--------|----------|-------------|--------------|----------|
+| `GET` | `/api/schedules` | List all schedules | - | `{ "schedules": [...] }` |
+| `GET` | `/api/schedules/{id}` | Get schedule details | - | `{ "schedule": {...} }` |
+| `POST` | `/api/schedules` | Create new schedule | `{ "name": "...", "config_file": "...", "cron_expression": "..." }` | `{ "schedule_id": N }` |
+| `PUT` | `/api/schedules/{id}` | Update schedule | `{ "enabled": true, "cron_expression": "..." }` | `{ "status": "updated" }` |
+| `DELETE` | `/api/schedules/{id}` | Delete schedule | - | `{ "status": "deleted" }` |
+| `POST` | `/api/schedules/{id}/trigger` | Manually trigger scheduled scan | - | `{ "scan_id": N }` |
+
+#### Alerts
+
+| Method | Endpoint | Description | Request Body | Response |
+|--------|----------|-------------|--------------|----------|
+| `GET` | `/api/alerts` | List recent alerts | - | `{ "alerts": [...] }` |
+| `GET` | `/api/alerts/rules` | List alert rules | - | `{ "rules": [...] }` |
+| `POST` | `/api/alerts/rules` | Create alert rule | `{ "rule_type": "...", "threshold": N }` | `{ "rule_id": N }` |
+| `PUT` | `/api/alerts/rules/{id}` | Update alert rule | `{ "enabled": false }` | `{ "status": "updated" }` |
+| `DELETE` | `/api/alerts/rules/{id}` | Delete alert rule | - | `{ "status": "deleted" }` |
+
+#### Settings
+
+| Method | Endpoint | Description | Request Body | Response |
+|--------|----------|-------------|--------------|----------|
+| `GET` | `/api/settings` | Get all settings (sanitized) | - | `{ "settings": {...} }` |
+| `PUT` | `/api/settings` | Update settings | `{ "smtp_server": "...", ... }` | `{ "status": "updated" }` |
+| `POST` | `/api/settings/test-email` | Test email configuration | - | `{ "status": "sent" }` |
+
+#### Statistics & Trends
+
+| Method | Endpoint | Description | Request Body | Response |
+|--------|----------|-------------|--------------|----------|
+| `GET` | `/api/stats/summary` | Dashboard summary stats | - | `{ "total_scans": N, "last_scan": "...", ... }` |
+| `GET` | `/api/stats/trends` | Trend data for charts | `?days=30&metric=port_count` | `{ "data": [...] }` |
+| `GET` | `/api/stats/certificates` | Certificate expiry overview | - | `{ "expiring_soon": [...], "expired": [...] }` |
+
+### Authentication
+
+**Phase 2-3:** Simple session-based authentication (single-user)
+- Login endpoint: `POST /api/auth/login` (username/password)
+- Logout endpoint: `POST /api/auth/logout`
+- Session cookies with Flask-Login
+- Password stored as bcrypt hash in settings table
+
+**Phase 5:** API token authentication for CLI client
+- Generate API token: `POST /api/auth/token`
+- Revoke token: `DELETE /api/auth/token`
+- CLI sends token in `Authorization: Bearer <token>` header
+
+## Phased Roadmap
+
+### Phase 1: Foundation (Weeks 1-2)
+**Priority: CRITICAL** - Database and settings infrastructure
+
+**Goals:**
+- Establish database schema
+- Create settings system
+- Set up Flask project structure
+
+**Tasks:**
+1. Create SQLite database schema (use Alembic for migrations)
+2. Implement SQLAlchemy models for all tables
+3. Create database initialization script (`init_db.py`)
+4. Implement settings system:
+   - Settings model with get/set methods
+   - Default settings initialization
+   - Encrypted storage for passwords (cryptography library)
+5. Set up Flask project structure:
+   ```
+   SneakyScanner/
+   ├── src/
+   │   ├── scanner.py (existing)
+   │   ├── screenshot_capture.py (existing)
+   │   └── report_generator.py (existing)
+   ├── web/
+   │   ├── app.py (Flask app factory)
+   │   ├── models.py (SQLAlchemy models)
+   │   ├── api/ (API blueprints)
+   │   │   ├── scans.py
+   │   │   ├── schedules.py
+   │   │   ├── alerts.py
+   │   │   └── settings.py
+   │   ├── templates/ (Jinja2 templates)
+   │   ├── static/ (CSS, JS, images)
+   │   └── utils/ (helpers, decorators)
+   ├── migrations/ (Alembic migrations)
+   ├── configs/ (existing)
+   ├── output/ (existing)
+   └── templates/ (existing - for reports)
+   ```
+6. Create `requirements-web.txt` for Flask dependencies
+7. Update Dockerfile to support Flask app
+
+**Deliverables:**
+- Working database with schema
+- Settings CRUD functionality
+- Flask app skeleton (no UI yet)
+- Database migration system
+
+**Testing:**
+- Database creates successfully
+- Settings can be stored/retrieved
+- Flask app starts without errors
+
+---
+
+### Phase 2: Flask Web App Core (Weeks 3-4)
+**Priority: HIGH** - Basic web application with API
+
+**Goals:**
+- Implement REST API for scans
+- Add background job queue
+- Create simple authentication
+- Integrate scanner with database
+
+**Tasks:**
+1. Implement scan API endpoints:
+   - `POST /api/scans` - trigger scan, save to DB
+   - `GET /api/scans` - list scans with pagination
+   - `GET /api/scans/{id}` - get scan details from DB
+   - `DELETE /api/scans/{id}` - delete scan
+2. Integrate scanner with database:
+   - Modify `scanner.py` to save results to DB after scan
+   - Create `ScanService` class to handle scan → DB logic
+   - Maintain JSON/HTML/ZIP file generation
+3. Set up background job queue:
+   - Install APScheduler
+   - Create job executor for scans
+   - Implement scan status tracking (`running`, `completed`, `failed`)
+4. Implement authentication:
+   - Flask-Login for session management
+   - Login page (`/login`)
+   - Password verification against settings table
+   - Protect all routes with `@login_required` decorator
+5. Create basic templates:
+   - `base.html` - Base layout with Bootstrap 5 dark theme
+   - `login.html` - Login page
+   - `dashboard.html` - Placeholder dashboard
+6. Error handling and logging:
+   - API error responses (JSON format)
+   - Logging configuration (file + console)
+7. Docker Compose setup:
+   - Flask container (Gunicorn)
+   - Volume mounts for DB, configs, output
+   - Port mapping (5000 for Flask)
+
+**Deliverables:**
+- Working REST API for scans
+- Background scan execution
+- Simple login system
+- Scanner integrated with database
+- Docker Compose deployment
+
+**Testing:**
+- API can trigger scan and return scan_id
+- Scan results saved to database
+- Pagination works for scan list
+- Authentication protects routes
+- Docker Compose brings up Flask app
+
+---
+
+### Phase 3: Dashboard & Scheduling (Weeks 5-6)
+**Priority: HIGH** - User's top requested features
+
+**Goals:**
+- Build web dashboard with scan history
+- Implement trend charts
+- Add scheduled scan management UI
+- Real-time scan progress
+
+**Tasks:**
+1. Dashboard implementation:
+   - Summary cards (total scans, last scan, total IPs, open ports)
+   - Recent scans table (clickable to view details)
+   - Security warnings section (expiring certs, weak TLS)
+   - Drift alerts section (unexpected ports, new services)
+2. Scan detail page:
+   - Display full scan results (sites, IPs, services)
+   - Embedded screenshots or links
+   - Download buttons (JSON, HTML, ZIP)
+   - Delete scan button
+3. Trend charts with Chart.js:
+   - Port count over time (line chart)
+   - Service distribution (bar chart)
+   - Certificate expiration timeline (timeline chart)
+   - Charts update based on date range selector
+4. Scheduled scans UI:
+   - List all schedules (table with enable/disable toggle)
+   - Create schedule form (name, config file, cron expression)
+   - Edit schedule form
+   - Delete schedule button
+   - Next run time display
+5. APScheduler integration:
+   - Schedule manager class
+   - Load schedules from DB on app start
+   - Add/remove jobs dynamically
+   - Cron expression validation
+   - Update `next_run` and `last_run` in DB
+6. Manual scan trigger:
+   - "Run Scan Now" button on dashboard
+   - Config file selector
+   - Show real-time progress (polling or WebSocket)
+7. Navigation menu:
+   - Dashboard
+   - Scans
+   - Schedules
+   - Alerts (placeholder)
+   - Settings (placeholder)
+
+**Deliverables:**
+- Functional dashboard with charts
+- Scan history browser
+- Scheduled scan management UI
+- Background scheduler running scans
+
+**Testing:**
+- Dashboard displays accurate summary stats
+- Charts render correctly with real data
+- Scheduled scans execute at specified times
+- Manual scan trigger works
+- Real-time progress updates
+
+---
+
+### Phase 4: Email & Comparisons (Weeks 7-8)
+**Priority: MEDIUM** - Monitoring and analysis features
+
+**Goals:**
+- Implement email notification system
+- Create scan comparison reports
+- Add alert rule configuration
+
+**Tasks:**
+1. Email notification system:
+   - SMTP integration (using `smtplib` or `Flask-Mail`)
+   - Email template for alerts (Jinja2 HTML email)
+   - Settings page for SMTP configuration
+   - Test email button
+2. Alert rule engine:
+   - Define alert rule types:
+     - Unexpected TCP/UDP port opened
+     - Expected port missing
+     - Certificate expiring in < N days
+     - Certificate expired
+     - Service version changed
+     - Ping failed (host down)
+     - Weak TLS version detected (1.0/1.1)
+   - Alert rule creation UI
+   - Alert rule evaluation after each scan
+   - Store alerts in `alerts` table
+3. Alert history page:
+   - List all alerts (filterable by type, severity, date)
+   - Mark alerts as "acknowledged"
+   - Alert detail view
+4. Scan comparison:
+   - Compare two scans API endpoint
+   - Comparison algorithm:
+     - New ports/services
+     - Removed ports/services
+     - Service version changes
+     - Certificate changes
+     - TLS configuration changes
+   - Comparison report UI:
+     - Side-by-side view
+     - Diff highlighting (green = added, red = removed, yellow = changed)
+   - "Compare" button on scan list (select 2 scans)
+5. Email notification triggers:
+   - Send email when alert rule triggered
+   - Daily digest email (summary of all alerts)
+   - Weekly scan summary email
+6. Settings page:
+   - SMTP configuration form
+   - Alert rule management
+   - Email recipient list
+   - Test email button
+
+**Deliverables:**
+- Working email notification system
+- Alert rules with email triggers
+- Scan comparison functionality
+- Settings UI for configuration
+
+**Testing:**
+- Email sends successfully with SMTP config
+- Alert rules trigger correctly
+- Comparison shows accurate diffs
+- Settings persist correctly
+
+---
+
+### Phase 5: CLI as API Client (Week 9)
+**Priority: MEDIUM** - Backward compatibility and automation
+
+**Goals:**
+- Refactor CLI to optionally call Flask API
+- Maintain standalone mode for testing
+- API token authentication
+
+**Tasks:**
+1. API client mode for `scanner.py`:
+   - Add `--api-mode` flag
+   - Add `--api-url` and `--api-token` arguments
+   - When `--api-mode` enabled:
+     - Send scan request to `POST /api/scans`
+     - Poll `GET /api/scans/{id}/status` for progress
+     - Download results when complete
+   - When `--api-mode` disabled (default):
+     - Run standalone as currently works
+2. API token generation:
+   - UI to generate API tokens (settings page)
+   - Store tokens in `api_tokens` table (hashed)
+   - API token authentication middleware
+   - Token expiration and revocation
+3. CLI documentation:
+   - Update README.md with API mode usage
+   - Example commands for API mode
+   - Token generation instructions
+4. Benefits of API mode:
+   - Scans stored centrally in database
+   - No need to mount volumes for output
+   - Scheduled scans managed through web UI
+   - Scan history accessible via web dashboard
+
+**Deliverables:**
+- CLI with `--api-mode` flag
+- API token system
+- Updated documentation
+
+**Testing:**
+- CLI can trigger scan via API
+- API token authentication works
+- Standalone mode still functional
+- Token revocation works
+
+---
+
+### Phase 6: Advanced Features (Weeks 10+)
+**Priority: LOW** - Nice-to-have enhancements
+
+**Goals:**
+- Enhanced interactive reports
+- Vulnerability detection
+- PDF export
+- Timeline view
+
+**Tasks:**
+1. Enhanced HTML reports:
+   - Sortable/filterable tables (DataTables.js)
+   - Inline screenshot thumbnails (lightbox on click)
+   - Export to PDF button (WeasyPrint or pdfkit)
+   - Print-friendly CSS
+2. Vulnerability detection:
+   - Integrate with CVE databases (NVD API or Vulners API)
+   - Match detected services/versions to known CVEs
+   - Display CVE list with severity scores (CVSS)
+   - CVE detail page with description, remediation
+   - Alert rule for new critical CVEs
+3. Timeline view:
+   - Visual timeline of all scans
+   - Filter by site or IP
+   - Click on timeline event to view scan
+   - Annotations for important events (cert renewals, config changes)
+4. Advanced charts:
+   - Heatmap of port activity
+   - Service version tracking over time
+   - Certificate expiration forecast
+   - Top 10 services pie chart
+5. Export/Import:
+   - Export scan data to CSV
+   - Import scan configs from CSV
+   - Bulk schedule creation
+6. Additional integrations:
+   - Slack notifications (in addition to email)
+   - Webhook support (POST to custom URL on events)
+   - Prometheus metrics export
+
+**Deliverables:**
+- Interactive sortable tables
+- CVE integration
+- PDF export
+- Timeline view
+- Additional integrations
+
+**Testing:**
+- DataTables work with large datasets
+- CVE data fetches correctly
+- PDF exports render properly
+- Timeline view performs well with many scans
+
+---
+
+## Migration Strategy
+
+### From Current CLI to Web App
+
+**Current State:**
+- CLI tool (`scanner.py`) runs standalone
+- Outputs JSON, HTML, ZIP files
+- No database, no web UI
+
+**Migration Path:**
+
+#### Step 1: Add Database Layer (Phase 1)
+- Database runs alongside CLI
+- CLI can optionally save to DB (flag: `--save-to-db`)
+- No breaking changes
+
+#### Step 2: Launch Web App (Phase 2-3)
+- Web app reads from DB
+- Users can trigger scans via web UI
+- CLI still works standalone
+
+#### Step 3: Transition Period (Phase 4-5)
+- Users gradually adopt web UI
+- CLI used for scripting/automation
+- Both modes fully supported
+
+#### Step 4: API Client Mode (Phase 5)
+- CLI becomes thin API client
+- All scans stored in central DB
+- Standalone mode remains for testing
+
+#### Step 5: Full Web App (Phase 6+)
+- Primary interface is web UI
+- CLI optional for power users
+
+### Backward Compatibility
+
+**Maintained:**
+- Existing YAML config format
+- JSON/HTML/ZIP output files
+- Screenshot capture
+- Docker deployment
+
+**Deprecated (eventually):**
+- Standalone CLI mode (Phase 6+)
+- Direct file output (replaced by DB + API)
+
+## Prioritized Feature List
+
+### Must-Have (Phases 1-3)
+1. **Database foundation** (SQLite3 + SQLAlchemy)
+2. **Flask web app core** (REST API, authentication)
+3. **Dashboard with scan history** (list, detail, delete)
+4. **Trend charts** (Chart.js - port counts, service distribution)
+5. **Scheduled scans** (APScheduler + cron expressions)
+6. **Manual scan trigger** (web UI button)
+
+### Should-Have (Phase 4)
+7. **Email notifications** (SMTP integration)
+8. **Alert rules** (cert expiry, unexpected ports, etc.)
+9. **Scan comparison reports** (diff view)
+10. **Settings UI** (SMTP, alerts, retention)
+
+### Nice-to-Have (Phases 5-6)
+11. **CLI as API client** (token auth, backward compat)
+12. **Sortable/filterable tables** (DataTables.js)
+13. **PDF export** (WeasyPrint)
+14. **Vulnerability detection** (CVE integration)
+15. **Timeline view** (visual scan history)
+16. **Embedded screenshot thumbnails** (lightbox)
+
+### Future/Deferred
+17. **Multi-user support** (if requirements change)
+18. **Slack/webhook integrations**
+19. **Prometheus metrics**
+20. **Mobile-responsive dashboard** (Bootstrap handles basics)
+
+## Development Workflow
+
+### Iteration Cycle
+1. **Plan** - Define features for phase
+2. **Implement** - Code backend + frontend
+3. **Test** - Unit tests + manual testing
+4. **Deploy** - Update Docker Compose
+5. **Document** - Update README.md, ROADMAP.md
+6. **Review** - Get user feedback
+7. **Iterate** - Adjust priorities based on feedback
+
+### Git Workflow
+- **main branch** - Stable releases
+- **develop branch** - Active development
+- **feature branches** - Individual features (`feature/dashboard`, `feature/scheduler`)
+- **Pull requests** - Review before merge
+
+### Testing Strategy
+- **Unit tests** - pytest for models, API endpoints
+- **Integration tests** - Full scan → DB → API workflow
+- **Manual testing** - UI/UX testing in browser
+- **Performance tests** - Large scans, database queries
+
+### Documentation
+- **README.md** - User-facing documentation (updated each phase)
+- **ROADMAP.md** - This file (updated as priorities shift)
+- **CLAUDE.md** - Developer documentation (architecture, code references)
+- **API.md** - API documentation (OpenAPI/Swagger in Phase 4)
+
+## Success Metrics
+
+### Phase 1-3 Success
+- [ ] Database stores scan results correctly
+- [ ] Dashboard displays scans and trends
+- [ ] Scheduled scans execute automatically
+- [ ] Web UI is intuitive and responsive
+
+### Phase 4 Success
+- [ ] Email notifications sent for critical alerts
+- [ ] Comparison reports show meaningful diffs
+- [ ] Settings UI allows configuration without editing files
+
+### Phase 5 Success
+- [ ] CLI can trigger scans via API
+- [ ] API tokens work for authentication
+- [ ] Standalone CLI mode still functional
+
+### Phase 6+ Success
+- [ ] CVE integration provides actionable vulnerability data
+- [ ] Timeline view helps track infrastructure changes
+- [ ] PDF exports are shareable and professional
+
+## Open Questions
+
+### Technical Decisions
+- **Flask vs. FastAPI?** - Sticking with Flask for simplicity, but FastAPI offers async and auto-docs
+- **APScheduler vs. Celery?** - APScheduler for simplicity (no Redis/RabbitMQ needed), but Celery scales better
+- **Bootstrap vs. Tailwind?** - Bootstrap for speed (pre-built components), Tailwind for customization
+- **Chart.js vs. Plotly?** - Chart.js for lightweight, Plotly for interactive (decide in Phase 3)
+
+### Product Questions
+- **Should we support multiple configs per schedule?** - Start with 1:1, add later if needed
+- **How many scans to keep in DB?** - Add retention setting (default: keep all)
+- **Support for multi-tenancy?** - Not in scope (single-user), but database schema allows future expansion
+- **Mobile app?** - Out of scope, but responsive web UI covers basics
+
+## Resources & References
+
+### Documentation
+- [Flask Documentation](https://flask.palletsprojects.com/)
+- [SQLAlchemy ORM](https://docs.sqlalchemy.org/)
+- [APScheduler](https://apscheduler.readthedocs.io/)
+- [Chart.js](https://www.chartjs.org/docs/)
+- [Bootstrap 5](https://getbootstrap.com/docs/5.3/)
+
+### Tutorials
+- [Flask Mega-Tutorial](https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world)
+- [SQLAlchemy Tutorial](https://docs.sqlalchemy.org/en/20/tutorial/)
+- [APScheduler with Flask](https://github.com/viniciuschiele/flask-apscheduler)
+
+### Similar Projects (Inspiration)
+- [OpenVAS](https://www.openvas.org/) - Vulnerability scanner with web UI
+- [Nessus](https://www.tenable.com/products/nessus) - Commercial scanner (inspiration for UI/UX)
+- [OWASP ZAP](https://www.zaproxy.org/) - Web app scanner (comparison reports, alerts)
+
+## Changelog
+
+| Date | Version | Changes |
+|------|---------|---------|
+| 2025-11-14 | 1.0 | Initial roadmap created based on user requirements |
+
+---
+
+**Last Updated:** 2025-11-14
+**Next Review:** After Phase 1 completion