sneakygeek/SneakyScan
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reorganize roadmap with versioned planned features

Browse Source 
Condensed completed phases into concise summaries and categorized
planned features into version milestones:
- v1.1.0: Communication & Automation (CLI, Email, CSV)
- v1.2.0: Reporting & Analysis (Scan Comparison, Enhanced Reports)
- v1.3.0: Visualization (Timeline View, Advanced Charts)
- v2.0.0: Security Intelligence (Vulnerability Detection)
This commit is contained in:
Phillip Tarrant
2025-11-20 14:39:14 -06:00
parent 7a14f1602b
commit a682e5233c
1 changed files with 70 additions and 597 deletions
Download Patch File Download Diff File Expand all files Collapse all files

649
docs/ROADMAP.md
View File

@@ -4,642 +4,115 @@
SneakyScanner is a comprehensive **Flask web application** for infrastructure monitoring and security auditing. The primary interface is the web GUI, with a CLI API client planned for scripting and automation needs.
**Current Phase:** Phase 5 Complete ✅ | Phase 6 Next Up
## Version 1.0.0 - Complete ✅
## Progress Overview
**Note:** For detailed architecture and technology stack information, see [README.md](../README.md)
- ✅ **Phase 1: Foundation** - Complete (2025-11-13)
- Database schema, SQLAlchemy models, settings system, Flask app structure
- ✅ **Phase 2: Flask Web App Core** - Complete (2025-11-14)
- REST API, background jobs, authentication, web UI, testing (100 tests)
- ✅ **Phase 3: Dashboard & Scheduling** - Complete (2025-11-14)
- Dashboard, scan history, scheduled scans, trend charts
- ✅ **Phase 4: Config Creator** - Complete (2025-11-17)
- CIDR-based config creation, YAML editor, config management UI
- ✅ **Phase 5: Webhooks & Alerting** - Complete (2025-11-19)
- Webhook notifications, alert rules, notification templates
- 📋 **Phase 6: CLI as API Client** - Planned
- CLI for scripting and automation via API
- 📋 **Phase 7: Advanced Features** - Future
- Email notifications, scan comparison, CVE integration, timeline view, PDF export
## Target Users
- **Infrastructure teams** monitoring on-premises networks
- **Security professionals** performing periodic security audits
- **DevOps engineers** tracking infrastructure drift
- **Single users or small teams** (not enterprise multi-tenant)
## Database Schema Design
### Core Tables
#### `scans`
Stores metadata about each scan execution.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique scan ID |
| `timestamp` | DATETIME | Scan start time (UTC) |
| `duration` | FLOAT | Total scan duration (seconds) |
| `status` | VARCHAR(20) | `running`, `completed`, `failed` |
| `config_id` | INTEGER | FK to scan_configs table |
| `title` | TEXT | Scan title from config |
| `json_path` | TEXT | Path to JSON report |
| `html_path` | TEXT | Path to HTML report |
| `zip_path` | TEXT | Path to ZIP archive |
| `screenshot_dir` | TEXT | Path to screenshot directory |
| `created_at` | DATETIME | Record creation time |
| `triggered_by` | VARCHAR(50) | `manual`, `scheduled`, `api` |
| `schedule_id` | INTEGER | FK to schedules (if triggered by schedule) |
#### `scan_sites`
Logical grouping of IPs by site.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique site record ID |
| `scan_id` | INTEGER | FK to scans |
| `site_name` | VARCHAR(255) | Site name from config |
#### `scan_ips`
IP addresses scanned in each scan.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique IP record ID |
| `scan_id` | INTEGER | FK to scans |
| `site_id` | INTEGER | FK to scan_sites |
| `ip_address` | VARCHAR(45) | IPv4 or IPv6 address |
| `ping_expected` | BOOLEAN | Expected ping response |
| `ping_actual` | BOOLEAN | Actual ping response |
#### `scan_ports`
Discovered TCP/UDP ports.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique port record ID |
| `scan_id` | INTEGER | FK to scans |
| `ip_id` | INTEGER | FK to scan_ips |
| `port` | INTEGER | Port number (1-65535) |
| `protocol` | VARCHAR(10) | `tcp` or `udp` |
| `expected` | BOOLEAN | Was this port expected? |
| `state` | VARCHAR(20) | `open`, `closed`, `filtered` |
#### `scan_services`
Detected services on open ports.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique service record ID |
| `scan_id` | INTEGER | FK to scans |
| `port_id` | INTEGER | FK to scan_ports |
| `service_name` | VARCHAR(100) | Service name (e.g., `ssh`, `http`) |
| `product` | VARCHAR(255) | Product name (e.g., `OpenSSH`) |
| `version` | VARCHAR(100) | Version string |
| `extrainfo` | TEXT | Additional nmap info |
| `ostype` | VARCHAR(100) | OS type if detected |
| `http_protocol` | VARCHAR(10) | `http` or `https` (if web service) |
| `screenshot_path` | TEXT | Relative path to screenshot |
#### `scan_certificates`
SSL/TLS certificates discovered on HTTPS services.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique certificate record ID |
| `scan_id` | INTEGER | FK to scans |
| `service_id` | INTEGER | FK to scan_services |
| `subject` | TEXT | Certificate subject (CN) |
| `issuer` | TEXT | Certificate issuer |
| `serial_number` | TEXT | Serial number |
| `not_valid_before` | DATETIME | Validity start date |
| `not_valid_after` | DATETIME | Validity end date |
| `days_until_expiry` | INTEGER | Days until expiration |
| `sans` | TEXT | JSON array of SANs |
| `is_self_signed` | BOOLEAN | Self-signed certificate flag |
#### `scan_tls_versions`
TLS version support and cipher suites.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique TLS version record ID |
| `scan_id` | INTEGER | FK to scans |
| `certificate_id` | INTEGER | FK to scan_certificates |
| `tls_version` | VARCHAR(20) | `TLS 1.0`, `TLS 1.1`, `TLS 1.2`, `TLS 1.3` |
| `supported` | BOOLEAN | Is this version supported? |
| `cipher_suites` | TEXT | JSON array of cipher suites |
### Scheduling & Notifications Tables
#### `schedules`
Scheduled scan configurations.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique schedule ID |
| `name` | VARCHAR(255) | Schedule name (e.g., "Daily prod scan") |
| `config_id` | INTEGER | FK to scan_configs table |
| `cron_expression` | VARCHAR(100) | Cron-like schedule (e.g., `0 2 * * *`) |
| `enabled` | BOOLEAN | Is schedule active? |
| `last_run` | DATETIME | Last execution time |
| `next_run` | DATETIME | Next scheduled execution |
| `created_at` | DATETIME | Schedule creation time |
| `updated_at` | DATETIME | Last modification time |
#### `alerts`
Alert history and notifications sent.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique alert ID |
| `scan_id` | INTEGER | FK to scans |
| `alert_type` | VARCHAR(50) | `new_port`, `cert_expiry`, `service_change`, `ping_failed` |
| `severity` | VARCHAR(20) | `info`, `warning`, `critical` |
| `message` | TEXT | Human-readable alert message |
| `ip_address` | VARCHAR(45) | Related IP (optional) |
| `port` | INTEGER | Related port (optional) |
| `email_sent` | BOOLEAN | Was email notification sent? |
| `email_sent_at` | DATETIME | Email send timestamp |
| `created_at` | DATETIME | Alert creation time |
#### `alert_rules`
User-defined alert rules.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique rule ID |
| `rule_type` | VARCHAR(50) | `unexpected_port`, `cert_expiry`, `service_down`, etc. |
| `enabled` | BOOLEAN | Is rule active? |
| `threshold` | INTEGER | Threshold value (e.g., days for cert expiry) |
| `email_enabled` | BOOLEAN | Send email for this rule? |
| `created_at` | DATETIME | Rule creation time |
### Settings Table
#### `settings`
Application configuration key-value store.
| Column | Type | Description |
|--------|------|-------------|
| `id` | INTEGER PRIMARY KEY | Unique setting ID |
| `key` | VARCHAR(255) UNIQUE | Setting key (e.g., `smtp_server`) |
| `value` | TEXT | Setting value (JSON for complex values) |
| `updated_at` | DATETIME | Last modification time |
**Example Settings:**
- `smtp_server` - SMTP server hostname
- `smtp_port` - SMTP port (587, 465, 25)
- `smtp_username` - SMTP username
- `smtp_password` - SMTP password (encrypted)
- `smtp_from_email` - From email address
- `smtp_to_emails` - JSON array of recipient emails
- `app_password` - Single-user password hash (bcrypt)
- `retention_days` - How long to keep old scans (0 = forever)
## API Design
### REST API Endpoints
All API endpoints return JSON and follow RESTful conventions.
#### Scans
| Method | Endpoint | Description | Request Body | Response |
|--------|----------|-------------|--------------|----------|
| `GET` | `/api/scans` | List all scans (paginated) | - | `{ "scans": [...], "total": N, "page": 1 }` |
| `GET` | `/api/scans/{id}` | Get scan details | - | `{ "scan": {...} }` |
| `POST` | `/api/scans` | Trigger new scan | `{ "config_id": "path" }` | `{ "scan_id": N, "status": "running" }` |
| `DELETE` | `/api/scans/{id}` | Delete scan and files | - | `{ "status": "deleted" }` |
| `GET` | `/api/scans/{id}/status` | Get scan status | - | `{ "status": "running", "progress": "45%" }` |
| `GET` | `/api/scans/{id1}/compare/{id2}` | Compare two scans | - | `{ "diff": {...} }` |
#### Schedules
| Method | Endpoint | Description | Request Body | Response |
|--------|----------|-------------|--------------|----------|
| `GET` | `/api/schedules` | List all schedules | - | `{ "schedules": [...] }` |
| `GET` | `/api/schedules/{id}` | Get schedule details | - | `{ "schedule": {...} }` |
| `POST` | `/api/schedules` | Create new schedule | `{ "name": "...", "config_id": "...", "cron_expression": "..." }` | `{ "schedule_id": N }` |
| `PUT` | `/api/schedules/{id}` | Update schedule | `{ "enabled": true, "cron_expression": "..." }` | `{ "status": "updated" }` |
| `DELETE` | `/api/schedules/{id}` | Delete schedule | - | `{ "status": "deleted" }` |
| `POST` | `/api/schedules/{id}/trigger` | Manually trigger scheduled scan | - | `{ "scan_id": N }` |
#### Alerts
| Method | Endpoint | Description | Request Body | Response |
|--------|----------|-------------|--------------|----------|
| `GET` | `/api/alerts` | List recent alerts | - | `{ "alerts": [...] }` |
| `GET` | `/api/alerts/rules` | List alert rules | - | `{ "rules": [...] }` |
| `POST` | `/api/alerts/rules` | Create alert rule | `{ "rule_type": "...", "threshold": N }` | `{ "rule_id": N }` |
| `PUT` | `/api/alerts/rules/{id}` | Update alert rule | `{ "enabled": false }` | `{ "status": "updated" }` |
| `DELETE` | `/api/alerts/rules/{id}` | Delete alert rule | - | `{ "status": "deleted" }` |
#### Settings
| Method | Endpoint | Description | Request Body | Response |
|--------|----------|-------------|--------------|----------|
| `GET` | `/api/settings` | Get all settings (sanitized) | - | `{ "settings": {...} }` |
| `PUT` | `/api/settings` | Update settings | `{ "smtp_server": "...", ... }` | `{ "status": "updated" }` |
| `POST` | `/api/settings/test-email` | Test email configuration | - | `{ "status": "sent" }` |
#### Statistics & Trends
| Method | Endpoint | Description | Request Body | Response |
|--------|----------|-------------|--------------|----------|
| `GET` | `/api/stats/summary` | Dashboard summary stats | - | `{ "total_scans": N, "last_scan": "...", ... }` |
| `GET` | `/api/stats/trends` | Trend data for charts | `?days=30&metric=port_count` | `{ "data": [...] }` |
| `GET` | `/api/stats/certificates` | Certificate expiry overview | - | `{ "expiring_soon": [...], "expired": [...] }` |
### Authentication
**Phase 2-3:** Simple session-based authentication (single-user)
- Login endpoint: `POST /api/auth/login` (username/password)
- Logout endpoint: `POST /api/auth/logout`
- Session cookies with Flask-Login
- Password stored as bcrypt hash in settings table
**Phase 6:** API token authentication for CLI client
- Generate API token: `POST /api/auth/token`
- Revoke token: `DELETE /api/auth/token`
- CLI sends token in `Authorization: Bearer <token>` header
## Phased Roadmap
### Phase 1: Foundation ✅ COMPLETE
### Phase 1: Foundation ✅
**Completed:** 2025-11-13
**Deliverables:**
- SQLite database with 11 tables (scans, sites, IPs, ports, services, certificates, TLS versions, schedules, alerts, alert_rules, settings)
- SQLAlchemy ORM models with relationships
- Alembic migration system
- Settings system with encryption (bcrypt for passwords, Fernet for sensitive data)
- Database schema with 11 tables (SQLAlchemy ORM, Alembic migrations)
- Settings system with encryption (bcrypt, Fernet)
- Flask app structure with API blueprints
- Docker Compose deployment configuration
- Validation script for verification
- Docker Compose deployment
---
### Phase 2: Flask Web App Core ✅ COMPLETE
### Phase 2: Flask Web App Core ✅
**Completed:** 2025-11-14
**Deliverables:**
- REST API with 8 endpoints (scans: trigger, list, get, status, delete; settings: get, update, test-email)
- Background job queue using APScheduler (up to 3 concurrent scans)
- Session-based authentication with Flask-Login
- Database integration for scan results (full normalized schema population)
- Web UI templates (dashboard, scan list/detail, login, error pages)
- Error handling with content negotiation (JSON/HTML) and request IDs
- Logging system with rotating file handlers
- Production Docker Compose deployment
- Comprehensive test suite (100 tests, all passing)
- Documentation (API_REFERENCE.md, DEPLOYMENT.md)
- REST API (8 endpoints for scans, settings)
- Background job queue (APScheduler, 3 concurrent scans)
- Session-based authentication (Flask-Login)
- Web UI templates (dashboard, scan list/detail, login)
- Comprehensive test suite (100 tests)
---
### Phase 3: Dashboard & Scheduling ✅ COMPLETE
### Phase 3: Dashboard & Scheduling ✅
**Completed:** 2025-11-14
**Deliverables:**
- Dashboard with summary stats (total scans, IPs, ports, services)
- Recent scans table with clickable details
- Scan detail page with full results display
- Historical trend charts using Chart.js (port counts over time)
- Scheduled scan management UI (create, edit, delete, enable/disable)
- Schedule execution with APScheduler and cron expressions
- Manual scan trigger from web UI
- Navigation menu (Dashboard, Scans, Schedules, Configs, Settings)
- Download buttons for scan reports (JSON, HTML, ZIP)
- Dashboard with summary stats and trend charts (Chart.js)
- Scan detail pages with full results display
- Scheduled scan management (cron expressions)
- Download buttons for reports (JSON, HTML, ZIP)
---
### Phase 4: Config Creator ✅ COMPLETE
### Phase 4: Config Creator ✅
**Completed:** 2025-11-17
**Deliverables:**
- CIDR-based config creation UI (simplified workflow for quick config generation)
- YAML editor with CodeMirror (syntax highlighting, line numbers)
- Config management UI (list, view, edit, download, delete)
- Direct YAML upload for advanced users
- REST API for config operations (7 endpoints: list, get, create, update, delete, upload, download)
- Schedule dependency protection (prevents deleting configs used by schedules)
- Comprehensive testing (25+ unit and integration tests)
- CIDR-based config creation UI
- YAML editor with CodeMirror
- Config management (list, view, edit, download, delete)
- REST API for config operations (7 endpoints)
---
### Phase 5: Webhooks & Alerting ✅ COMPLETE
### Phase 5: Webhooks & Alerting ✅
**Completed:** 2025-11-19
**Goals:**
- ✅ Implement webhook notification system for real-time alerting
- ✅ Add alert rule configuration for unexpected exposure detection
- ✅ Create notification template system for flexible alerting
**Core Use Case:**
Monitor infrastructure for misconfigurations that expose unexpected ports/services to the world. When a scan detects an open port that wasn't defined in the YAML config's `expected_ports` list, trigger immediate notifications via webhooks.
**Implemented Features:**
#### 1. Alert Rule Engine ✅
**Purpose:** Automatically detect and classify infrastructure anomalies after each scan.
**Alert Types:**
- `unexpected_port` - Port open but not in config's `expected_ports` list
- `unexpected_service` - Service detected that doesn't match expected service name
- `cert_expiry` - SSL/TLS certificate expiring soon (configurable threshold)
- `ping_failed` - Expected host not responding to ping
- `service_down` - Previously detected service no longer responding
- `service_change` - Service version/product changed between scans
- `weak_tls` - TLS 1.0/1.1 detected or weak cipher suites
- `new_host` - New IP address responding in CIDR range
- `host_disappeared` - Previously seen IP no longer responding
**Alert Severity Levels:**
- `critical` - Unexpected internet-facing service (ports 80/443/22/3389/etc.)
- `warning` - Minor configuration drift or upcoming cert expiry
- `info` - Informational alerts (new host discovered, service version change)
**Alert Rule Configuration:**
```yaml
# Example alert rule configuration (stored in DB)
alert_rules:
- id: 1
rule_type: unexpected_port
enabled: true
severity: critical
webhook_enabled: true
filter_conditions:
ports: [22, 80, 443, 3389, 3306, 5432, 27017] # High-risk ports
- id: 2
rule_type: cert_expiry
enabled: true
severity: warning
threshold: 30 # Days before expiry
webhook_enabled: true
```
**Implementation:**
- ✅ Evaluate alert rules after each scan completes
- ✅ Compare current scan results to expected configuration
- ✅ Generate alerts and store in `alerts` table
- ✅ Trigger notifications based on rule configuration
- ✅ Alert deduplication (don't spam for same issue)
#### 2. Webhook Notifications ✅
**Purpose:** Real-time HTTP POST notifications for integration with external systems (Slack, PagerDuty, custom dashboards, SIEM tools).
**Webhook Configuration (via Settings API):**
```json
{
"webhook_enabled": true,
"webhook_urls": [
{
"id": 1,
"name": "Slack Security Channel",
"url": "https://hooks.slack.com/services/XXX/YYY/ZZZ",
"enabled": true,
"auth_type": "none",
"custom_headers": {},
"alert_types": ["unexpected_port", "unexpected_service", "weak_tls"],
"severity_filter": ["critical", "warning"]
},
{
"id": 2,
"name": "PagerDuty",
"url": "https://events.pagerduty.com/v2/enqueue",
"enabled": true,
"auth_type": "bearer",
"auth_token": "encrypted_token",
"custom_headers": {
"Content-Type": "application/json"
},
"alert_types": ["unexpected_port"],
"severity_filter": ["critical"]
}
]
}
```
**Webhook Payload Format (JSON):**
```json
{
"event_type": "scan_alert",
"alert_id": 42,
"alert_type": "unexpected_port",
"severity": "critical",
"timestamp": "2025-11-17T14:23:45Z",
"scan": {
"scan_id": 123,
"title": "Production Network Scan",
"timestamp": "2025-11-17T14:15:00Z",
"config_id": "prod_config.yaml",
"triggered_by": "scheduled"
},
"alert_details": {
"message": "Unexpected port 3306 (MySQL) exposed on 192.168.1.100",
"ip_address": "192.168.1.100",
"port": 3306,
"protocol": "tcp",
"state": "open",
"service": {
"name": "mysql",
"product": "MySQL",
"version": "8.0.32"
},
"expected": false,
"site_name": "Production Servers"
},
"recommended_actions": [
"Verify if MySQL should be exposed externally",
"Check firewall rules for 192.168.1.100",
"Review MySQL bind-address configuration"
],
"web_url": "https://sneakyscanner.local/scans/123"
}
```
**Webhook Features:**
- ✅ Multiple webhook URLs with independent configuration
- ✅ Per-webhook filtering by alert type and severity
- ✅ Custom headers support (for API keys, auth tokens)
- ✅ Authentication methods:
- `none` - No authentication
- `bearer` - Bearer token in Authorization header
- `basic` - Basic authentication
- `custom` - Custom header-based auth
- ✅ Retry logic with exponential backoff (3 attempts)
- ✅ Webhook delivery tracking (webhook_sent, webhook_sent_at, webhook_response_code)
- ✅ Test webhook functionality in Settings UI
- ✅ Timeout configuration (default 10 seconds)
- ✅ Webhook delivery history and logs
**Webhook API Endpoints:**
- ✅ `POST /api/webhooks` - Create webhook configuration
- ✅ `GET /api/webhooks` - List all webhooks
- ✅ `PUT /api/webhooks/{id}` - Update webhook configuration
- ✅ `DELETE /api/webhooks/{id}` - Delete webhook
- ✅ `POST /api/webhooks/{id}/test` - Send test webhook
- ✅ `GET /api/webhooks/{id}/history` - Get delivery history
**Notification Templates:**
Flexible template system supporting multiple platforms (Slack, Discord, PagerDuty, etc.):
```json
{
"text": "SneakyScanner Alert: Unexpected Port Detected",
"attachments": [
{
"color": "danger",
"fields": [
{"title": "IP Address", "value": "192.168.1.100", "short": true},
{"title": "Port", "value": "3306/tcp", "short": true},
{"title": "Service", "value": "MySQL 8.0.32", "short": true},
{"title": "Severity", "value": "CRITICAL", "short": true}
],
"footer": "SneakyScanner",
"ts": 1700234625
}
]
}
```
- Alert Rule Engine (9 alert types: unexpected_port, cert_expiry, ping_failed, etc.)
- Webhook notifications with retry logic
- Multiple webhook URLs with independent filtering
- Notification templates (Slack, Discord, PagerDuty support)
- Alert deduplication
---
**Deliverables:**
- ✅ Alert Rule Engine with 9 alert types (unexpected_port, unexpected_service, cert_expiry, ping_failed, service_down, service_change, weak_tls, new_host, host_disappeared)
- ✅ Alert severity classification (critical, warning, info)
- ✅ Alert rule configuration API (CRUD operations)
- ✅ Webhook notification system with retry logic
- ✅ Multiple webhook URL support with independent configuration
- ✅ Notification template system for flexible platform integration (Slack, Discord, PagerDuty, custom)
- ✅ Webhook API endpoints (create, list, update, delete, test, history)
- ✅ Custom headers and authentication support (none, bearer, basic, custom)
- ✅ Webhook delivery tracking and logging
- ✅ Alert deduplication to prevent notification spam
- ✅ Integration with scan completion workflow
## Planned Features
**Success Criteria Met:**
- ✅ Alerts triggered within 30 seconds of scan completion
- ✅ Webhook POST delivered with retry on failure
- ✅ Zero false positives for expected ports/services
- ✅ Alert deduplication prevents notification spam
### Version 1.1.0 - Communication & Automation
---
#### CLI as API Client
- CLI tool for scripting and automation via REST API
- API token authentication (Bearer tokens)
- Commands for scan management, schedules, alerts
### Phase 6: Advanced Features
**Status:** Future/Deferred
**Priority:** LOW
**Planned Features:**
1. **Email Notifications:**
#### Email Notifications
- SMTP integration with Flask-Mail
- Jinja2 email templates (HTML + plain text)
- Settings API for email configuration
- Test email functionality
- Email delivery tracking
- Rate limiting to prevent email flood
- Configurable recipients (multiple emails)
- Configurable recipients and rate limiting
2. **Scan Comparison:**
#### Site CSV Export/Import
- Bulk site management via CSV files
---
### Version 1.2.0 - Reporting & Analysis
#### Scan Comparison
- Compare two scans API endpoint
- Side-by-side comparison view
- Color-coded differences (green=new, red=removed, yellow=changed)
- Filter by change type
- Side-by-side comparison view with color-coded differences
- Export comparison report to PDF/HTML
- "Compare with previous scan" button on scan detail page
3. **Enhanced Reports:**
#### Enhanced Reports
- Sortable/filterable tables (DataTables.js)
- PDF export (WeasyPrint)
4. **Vulnerability Detection:**
- CVE database integration (NVD API)
- Service version matching to known CVEs
- CVSS severity scores
- Alert rules for critical CVEs
---
5. **Timeline View:**
### Version 1.3.0 - Visualization
#### Timeline View
- Visual scan history timeline
- Filter by site/IP
- Event annotations
6. **Advanced Charts:**
#### Advanced Charts
- Port activity heatmap
- Certificate expiration forecast
7. **Additional Integrations:**
- Site CSV export/import
---
## Development Workflow
### Version 2.0.0 - Security Intelligence
### Iteration Cycle
1. **Plan** - Define features for phase
2. **Implement** - Code backend + frontend
3. **Test** - Unit tests + manual testing
4. **Deploy** - Update Docker Compose
5. **Document** - Update README.md, ROADMAP.md
6. **Review** - Get user feedback
7. **Iterate** - Adjust priorities based on feedback
#### Vulnerability Detection
- CVE database integration (NVD API)
- Service version matching to known CVEs
- CVSS severity scores
### Git Workflow
- **main branch** - Stable releases
- **develop branch** - Active development
- **feature branches** - Individual features (`feature/dashboard`, `feature/scheduler`)
- **Pull requests** - Review before merge
### Testing Strategy
- **Unit tests** - pytest for models, API endpoints
- **Integration tests** - Full scan → DB → API workflow
- **Manual testing** - UI/UX testing in browser
- **Performance tests** - Large scans, database queries
### Documentation
- **README.md** - User-facing documentation (updated each phase)
- **ROADMAP.md** - This file (updated as priorities shift)
- **CLAUDE.md** - Developer documentation (architecture, code references)
- **API.md** - API documentation (OpenAPI/Swagger in Phase 4)
## Resources & References
### Documentation
- [Flask Documentation](https://flask.palletsprojects.com/)
- [SQLAlchemy ORM](https://docs.sqlalchemy.org/)
- [APScheduler](https://apscheduler.readthedocs.io/)
- [Chart.js](https://www.chartjs.org/docs/)
- [Bootstrap 5](https://getbootstrap.com/docs/5.3/)
### Tutorials
- [Flask Mega-Tutorial](https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world)
- [SQLAlchemy Tutorial](https://docs.sqlalchemy.org/en/20/tutorial/)
- [APScheduler with Flask](https://github.com/viniciuschiele/flask-apscheduler)
### Similar Projects (Inspiration)
- [OpenVAS](https://www.openvas.org/) - Vulnerability scanner with web UI
- [Nessus](https://www.tenable.com/products/nessus) - Commercial scanner (inspiration for UI/UX)
- [OWASP ZAP](https://www.zaproxy.org/) - Web app scanner (comparison reports, alerts)
---
## Changelog
| Date | Version | Changes |
|------|---------|---------|
| 2025-11-14 | 1.0 | Initial roadmap created based on user requirements |
| 2025-11-13 | 1.1 | **Phase 1 COMPLETE** - Database schema, SQLAlchemy models, Flask app structure, settings system with encryption, Alembic migrations, API blueprints, Docker support, validation script |
| 2025-11-14 | 1.2 | **Phase 2 COMPLETE** - REST API (5 scan endpoints, 3 settings endpoints), background jobs (APScheduler), authentication (Flask-Login), web UI (dashboard, scans, login, errors), error handling (content negotiation, request IDs, logging), 100 tests passing, comprehensive documentation (API_REFERENCE.md, DEPLOYMENT.md, PHASE2_COMPLETE.md) |
| 2025-11-17 | 1.3 | **Bug Fix** - Fixed Chart.js infinite canvas growth issue in scan detail page (duplicate initialization, missing chart.destroy(), missing fixed-height container) |
| 2025-11-17 | 1.4 | **Phase 4 COMPLETE** - Config Creator with CIDR-based creation, YAML editor (CodeMirror), config management UI (list/edit/delete), REST API (7 endpoints), Docker volume permissions fix, comprehensive testing and documentation |
| 2025-11-17 | 1.5 | **Roadmap Compression** - Condensed completed phases (1-4) into concise summaries, updated project scope to emphasize web GUI frontend with CLI as API client coming soon (Phase 6), reorganized phases for clarity |
| 2025-11-19 | 1.6 | **Phase 5 Progress** - Completed webhooks, notification templates, and alerting rules. Alert Rule Engine and Webhook System implemented. |
| 2025-11-19 | 1.7 | **Phase 5 COMPLETE** - Webhooks & Alerting phase completed. Moved Email Notifications and Scan Comparison to Phase 7. Alert rules, webhook notifications, and notification templates fully implemented and tested. |
| 2025-11-13 | 1.0.0-alpha | Phase 1 complete - Foundation |
| 2025-11-14 | 1.0.0-beta | Phases 2-3 complete - Web App Core, Dashboard & Scheduling |
| 2025-11-17 | 1.0.0-rc1 | Phase 4 complete - Config Creator |
| 2025-11-19 | 1.0.0 | Phase 5 complete - Webhooks & Alerting |
---
**Last Updated:** 2025-11-19
**Next Review:** Before Phase 6 kickoff (CLI as API Client)
**Last Updated:** 2025-11-20