doc changes
This commit is contained in:
26
README.md
26
README.md
@@ -20,7 +20,8 @@ A comprehensive network scanning and infrastructure monitoring platform with web
|
|||||||
- 📈 **Drift Detection** - Expected vs. actual infrastructure comparison
|
- 📈 **Drift Detection** - Expected vs. actual infrastructure comparison
|
||||||
- 📋 **Multi-Format Reports** - JSON, HTML, and ZIP archives with visual reports
|
- 📋 **Multi-Format Reports** - JSON, HTML, and ZIP archives with visual reports
|
||||||
- 🔐 **Authentication** - Session-based login for single-user deployments
|
- 🔐 **Authentication** - Session-based login for single-user deployments
|
||||||
- 🔔 **Alerts** *(Phase 5 - Coming Soon)* - Email and webhook notifications for misconfigurations
|
- 🔔 **Webhook Alerts** - Real-time notifications via Slack, Discord, PagerDuty, and custom integrations
|
||||||
|
- ⚠️ **Alert Rules** - Automated detection of infrastructure misconfigurations and anomalies
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -98,8 +99,8 @@ docker compose -f docker-compose-standalone.yml up
|
|||||||
|
|
||||||
## Current Status
|
## Current Status
|
||||||
|
|
||||||
**Latest Version**: Phase 4 Complete ✅
|
**Latest Version**: Phase 5 Complete ✅
|
||||||
**Last Updated**: 2025-11-17
|
**Last Updated**: 2025-11-19
|
||||||
|
|
||||||
### Completed Phases
|
### Completed Phases
|
||||||
|
|
||||||
@@ -107,18 +108,19 @@ docker compose -f docker-compose-standalone.yml up
|
|||||||
- ✅ **Phase 2**: REST API, background jobs, authentication, web UI
|
- ✅ **Phase 2**: REST API, background jobs, authentication, web UI
|
||||||
- ✅ **Phase 3**: Dashboard, scheduling, trend charts
|
- ✅ **Phase 3**: Dashboard, scheduling, trend charts
|
||||||
- ✅ **Phase 4**: Config creator, YAML editor, config management UI
|
- ✅ **Phase 4**: Config creator, YAML editor, config management UI
|
||||||
|
- ✅ **Phase 5**: Webhooks & alerting, notification templates, alert rules
|
||||||
|
|
||||||
### Next Up: Phase 5 - Email, Webhooks & Comparisons
|
### Next Up: Phase 6 - CLI as API Client
|
||||||
|
|
||||||
**Core Use Case**: Monitor infrastructure for misconfigurations that expose unexpected ports/services. When a scan detects an open port not in the config's `expected_ports` list, trigger immediate notifications.
|
**Goal**: Create a thin CLI client that calls the Flask API for scan operations, enabling scripting and automation workflows while leveraging centralized database storage and web dashboard features.
|
||||||
|
|
||||||
**Planned Features**:
|
**Planned Features**:
|
||||||
- Email notifications for infrastructure changes
|
- API token authentication for CLI access
|
||||||
- Webhook integrations (Slack, PagerDuty, custom SIEM)
|
- Remote scan triggering and status polling
|
||||||
- Alert rule engine (unexpected ports, cert expiry, weak TLS)
|
- Centralized scan history accessible via web dashboard
|
||||||
- Scan comparison reports for drift detection
|
- Scriptable automation workflows
|
||||||
|
|
||||||
See [Roadmap](docs/ROADMAP.md) for complete feature timeline.
|
See [Roadmap](docs/ROADMAP.md) for complete feature timeline and future phases.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -212,5 +214,5 @@ MIT License - See LICENSE file for details
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Version**: Phase 4 Complete
|
**Version**: Phase 5 Complete
|
||||||
**Last Updated**: 2025-11-17
|
**Last Updated**: 2025-11-19
|
||||||
|
|||||||
423
docs/ROADMAP.md
423
docs/ROADMAP.md
@@ -4,9 +4,12 @@
|
|||||||
|
|
||||||
SneakyScanner is a comprehensive **Flask web application** for infrastructure monitoring and security auditing. The primary interface is the web GUI, with a CLI API client planned for scripting and automation needs.
|
SneakyScanner is a comprehensive **Flask web application** for infrastructure monitoring and security auditing. The primary interface is the web GUI, with a CLI API client planned for scripting and automation needs.
|
||||||
|
|
||||||
**Status:** Phase 4 Complete ✅ | Phase 5 Next Up
|
**Current Phase:** Phase 5 Complete ✅ | Phase 6 Next Up
|
||||||
|
|
||||||
## Progress Overview
|
## Progress Overview
|
||||||
|
|
||||||
|
**Note:** For detailed architecture and technology stack information, see [README.md](../README.md)
|
||||||
|
|
||||||
- ✅ **Phase 1: Foundation** - Complete (2025-11-13)
|
- ✅ **Phase 1: Foundation** - Complete (2025-11-13)
|
||||||
- Database schema, SQLAlchemy models, settings system, Flask app structure
|
- Database schema, SQLAlchemy models, settings system, Flask app structure
|
||||||
- ✅ **Phase 2: Flask Web App Core** - Complete (2025-11-14)
|
- ✅ **Phase 2: Flask Web App Core** - Complete (2025-11-14)
|
||||||
@@ -15,27 +18,14 @@ SneakyScanner is a comprehensive **Flask web application** for infrastructure mo
|
|||||||
- Dashboard, scan history, scheduled scans, trend charts
|
- Dashboard, scan history, scheduled scans, trend charts
|
||||||
- ✅ **Phase 4: Config Creator** - Complete (2025-11-17)
|
- ✅ **Phase 4: Config Creator** - Complete (2025-11-17)
|
||||||
- CIDR-based config creation, YAML editor, config management UI
|
- CIDR-based config creation, YAML editor, config management UI
|
||||||
- 📋 **Phase 5: Email, Webhooks & Comparisons** - Next Up
|
- ✅ **Phase 5: Webhooks & Alerting** - Complete (2025-11-19)
|
||||||
- Email notifications, alert rules, scan comparison
|
- Webhook notifications, alert rules, notification templates
|
||||||
- 📋 **Phase 6: CLI as API Client** - Planned
|
- 📋 **Phase 6: CLI as API Client** - Planned
|
||||||
- CLI for scripting and automation via API
|
- CLI for scripting and automation via API
|
||||||
- 📋 **Phase 7: Advanced Features** - Future
|
- 📋 **Phase 7: Advanced Features** - Future
|
||||||
- CVE integration, timeline view, PDF export, enhanced reports
|
- Email notifications, scan comparison, CVE integration, timeline view, PDF export
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**Core Features:**
|
|
||||||
- **Centralized dashboard** for viewing scan history and trends
|
|
||||||
- **Scheduled scanning** for continuous infrastructure monitoring
|
|
||||||
- **Email notifications** for critical changes and certificate expirations (coming soon)
|
|
||||||
- **Historical analysis** with charts, graphs, and comparison reports
|
|
||||||
- **Config creator** for easy CIDR-based scan configuration
|
|
||||||
- **RESTful API** for integration and automation
|
|
||||||
- **Simple deployment** using SQLite3 (single-user, self-hosted)
|
|
||||||
|
|
||||||
**Planned:**
|
|
||||||
- **CLI API client** for scripting and automation workflows (Phase 6)
|
|
||||||
|
|
||||||
## Target Users
|
## Target Users
|
||||||
|
|
||||||
- **Infrastructure teams** monitoring on-premises networks
|
- **Infrastructure teams** monitoring on-premises networks
|
||||||
@@ -43,84 +33,6 @@ SneakyScanner is a comprehensive **Flask web application** for infrastructure mo
|
|||||||
- **DevOps engineers** tracking infrastructure drift
|
- **DevOps engineers** tracking infrastructure drift
|
||||||
- **Single users or small teams** (not enterprise multi-tenant)
|
- **Single users or small teams** (not enterprise multi-tenant)
|
||||||
|
|
||||||
## Technology Stack
|
|
||||||
|
|
||||||
### Backend
|
|
||||||
- **Flask** 3.x - Lightweight Python web framework
|
|
||||||
- **SQLAlchemy** 2.x - ORM for database abstraction
|
|
||||||
- **SQLite3** - Embedded database (easy deployment, sufficient for single-user)
|
|
||||||
- **APScheduler** 3.x - Background job scheduling for periodic scans
|
|
||||||
- **Flask-Login** - Simple session-based authentication
|
|
||||||
- **Flask-CORS** - API access control (optional for CLI API client)
|
|
||||||
- **Marshmallow** - API serialization/deserialization
|
|
||||||
|
|
||||||
### Frontend
|
|
||||||
- **Jinja2** - Server-side templating (already in use)
|
|
||||||
- **Bootstrap 5** - Responsive UI framework with dark theme support
|
|
||||||
- **Chart.js** 4.x - Lightweight charting library for trends
|
|
||||||
- **DataTables.js** - Interactive sortable/filterable tables (Phase 6)
|
|
||||||
- **Vanilla JavaScript** - Keep dependencies minimal
|
|
||||||
|
|
||||||
### Infrastructure
|
|
||||||
- **Docker Compose** - Multi-container orchestration (Flask app + Scanner)
|
|
||||||
- **Gunicorn** - WSGI server for production
|
|
||||||
- **Nginx** - Reverse proxy (optional, for production deployments)
|
|
||||||
|
|
||||||
### Existing Components (Keep)
|
|
||||||
- **Masscan** - Fast port discovery
|
|
||||||
- **Nmap** - Service detection
|
|
||||||
- **Playwright** - Screenshot capture
|
|
||||||
- **sslyze** - SSL/TLS analysis
|
|
||||||
|
|
||||||
## Architecture Overview
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Flask Web Application │
|
|
||||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
|
|
||||||
│ │ Web UI │ │ REST API │ │ Scheduler │ │
|
|
||||||
│ │ (Dashboard) │ │ (JSON/CRUD) │ │ (APScheduler) │ │
|
|
||||||
│ └──────┬───────┘ └──────┬───────┘ └────────┬─────────┘ │
|
|
||||||
│ │ │ │ │
|
|
||||||
│ └─────────────────┴────────────────────┘ │
|
|
||||||
│ │ │
|
|
||||||
│ ┌────────▼────────┐ │
|
|
||||||
│ │ SQLAlchemy │ │
|
|
||||||
│ │ (ORM Layer) │ │
|
|
||||||
│ └────────┬────────┘ │
|
|
||||||
│ │ │
|
|
||||||
│ ┌────────▼────────┐ │
|
|
||||||
│ │ SQLite3 DB │ │
|
|
||||||
│ │ (scan history) │ │
|
|
||||||
│ └─────────────────┘ │
|
|
||||||
└───────────────────────────┬─────────────────────────────────┘
|
|
||||||
│
|
|
||||||
┌──────────▼──────────┐
|
|
||||||
│ Scanner Engine │
|
|
||||||
│ (scanner.py) │
|
|
||||||
│ ┌────────────────┐ │
|
|
||||||
│ │ Masscan/Nmap │ │
|
|
||||||
│ │ Playwright │ │
|
|
||||||
│ │ sslyze │ │
|
|
||||||
│ └────────────────┘ │
|
|
||||||
└─────────────────────┘
|
|
||||||
│
|
|
||||||
┌──────────▼──────────┐
|
|
||||||
│ CLI API Client │
|
|
||||||
│ (optional future) │
|
|
||||||
└─────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### Component Interaction
|
|
||||||
|
|
||||||
1. **Web UI** - User interacts with dashboard, triggers scans, views history
|
|
||||||
2. **REST API** - Handles requests from web UI and CLI client
|
|
||||||
3. **Scheduler (APScheduler)** - Triggers scans based on cron-like schedules
|
|
||||||
4. **SQLAlchemy ORM** - Abstracts database operations
|
|
||||||
5. **SQLite3 Database** - Stores scan results, schedules, settings, alerts
|
|
||||||
6. **Scanner Engine** - Performs actual network scanning (masscan, nmap, etc.)
|
|
||||||
7. **CLI API Client** - Future: thin client that calls Flask API
|
|
||||||
|
|
||||||
## Database Schema Design
|
## Database Schema Design
|
||||||
|
|
||||||
### Core Tables
|
### Core Tables
|
||||||
@@ -352,7 +264,7 @@ All API endpoints return JSON and follow RESTful conventions.
|
|||||||
- Session cookies with Flask-Login
|
- Session cookies with Flask-Login
|
||||||
- Password stored as bcrypt hash in settings table
|
- Password stored as bcrypt hash in settings table
|
||||||
|
|
||||||
**Phase 5:** API token authentication for CLI client
|
**Phase 6:** API token authentication for CLI client
|
||||||
- Generate API token: `POST /api/auth/token`
|
- Generate API token: `POST /api/auth/token`
|
||||||
- Revoke token: `DELETE /api/auth/token`
|
- Revoke token: `DELETE /api/auth/token`
|
||||||
- CLI sends token in `Authorization: Bearer <token>` header
|
- CLI sends token in `Authorization: Bearer <token>` header
|
||||||
@@ -420,22 +332,20 @@ All API endpoints return JSON and follow RESTful conventions.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Phase 5: Email, Webhooks & Comparisons
|
### Phase 5: Webhooks & Alerting ✅ COMPLETE
|
||||||
**Status:** Next Up
|
**Completed:** 2025-11-19
|
||||||
**Priority:** MEDIUM
|
|
||||||
|
|
||||||
**Goals:**
|
**Goals:**
|
||||||
- Implement email notification system for infrastructure misconfigurations
|
- ✅ Implement webhook notification system for real-time alerting
|
||||||
- Implement webhook notification system for real-time alerting
|
- ✅ Add alert rule configuration for unexpected exposure detection
|
||||||
- Create scan comparison reports to detect drift
|
- ✅ Create notification template system for flexible alerting
|
||||||
- Add alert rule configuration for unexpected exposure detection
|
|
||||||
|
|
||||||
**Core Use Case:**
|
**Core Use Case:**
|
||||||
Monitor infrastructure for misconfigurations that expose unexpected ports/services to the world. When a scan detects an open port that wasn't defined in the YAML config's `expected_ports` list, trigger immediate notifications via email and/or webhooks.
|
Monitor infrastructure for misconfigurations that expose unexpected ports/services to the world. When a scan detects an open port that wasn't defined in the YAML config's `expected_ports` list, trigger immediate notifications via webhooks.
|
||||||
|
|
||||||
**Planned Features:**
|
**Implemented Features:**
|
||||||
|
|
||||||
#### 1. Alert Rule Engine
|
#### 1. Alert Rule Engine ✅
|
||||||
**Purpose:** Automatically detect and classify infrastructure anomalies after each scan.
|
**Purpose:** Automatically detect and classify infrastructure anomalies after each scan.
|
||||||
|
|
||||||
**Alert Types:**
|
**Alert Types:**
|
||||||
@@ -462,7 +372,6 @@ alert_rules:
|
|||||||
rule_type: unexpected_port
|
rule_type: unexpected_port
|
||||||
enabled: true
|
enabled: true
|
||||||
severity: critical
|
severity: critical
|
||||||
email_enabled: true
|
|
||||||
webhook_enabled: true
|
webhook_enabled: true
|
||||||
filter_conditions:
|
filter_conditions:
|
||||||
ports: [22, 80, 443, 3389, 3306, 5432, 27017] # High-risk ports
|
ports: [22, 80, 443, 3389, 3306, 5432, 27017] # High-risk ports
|
||||||
@@ -472,85 +381,17 @@ alert_rules:
|
|||||||
enabled: true
|
enabled: true
|
||||||
severity: warning
|
severity: warning
|
||||||
threshold: 30 # Days before expiry
|
threshold: 30 # Days before expiry
|
||||||
email_enabled: true
|
webhook_enabled: true
|
||||||
webhook_enabled: false
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Implementation:**
|
**Implementation:**
|
||||||
- Evaluate alert rules after each scan completes
|
- ✅ Evaluate alert rules after each scan completes
|
||||||
- Compare current scan results to expected configuration
|
- ✅ Compare current scan results to expected configuration
|
||||||
- Generate alerts and store in `alerts` table
|
- ✅ Generate alerts and store in `alerts` table
|
||||||
- Trigger notifications based on rule configuration
|
- ✅ Trigger notifications based on rule configuration
|
||||||
- Alert deduplication (don't spam for same issue)
|
- ✅ Alert deduplication (don't spam for same issue)
|
||||||
|
|
||||||
#### 2. Email Notifications
|
#### 2. Webhook Notifications ✅
|
||||||
**Purpose:** Send detailed email alerts when infrastructure misconfigurations are detected.
|
|
||||||
|
|
||||||
**SMTP Configuration (via Settings API):**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"smtp_server": "smtp.gmail.com",
|
|
||||||
"smtp_port": 587,
|
|
||||||
"smtp_use_tls": true,
|
|
||||||
"smtp_username": "alerts@example.com",
|
|
||||||
"smtp_password": "encrypted_password",
|
|
||||||
"smtp_from_email": "SneakyScanner <alerts@example.com>",
|
|
||||||
"smtp_to_emails": ["admin@example.com", "security@example.com"],
|
|
||||||
"email_alerts_enabled": true
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Email Template (Jinja2 HTML):**
|
|
||||||
```html
|
|
||||||
Subject: [SneakyScanner Alert] Unexpected Port Detected - {{ ip_address }}:{{ port }}
|
|
||||||
|
|
||||||
Body:
|
|
||||||
===============================================
|
|
||||||
SneakyScanner Security Alert
|
|
||||||
===============================================
|
|
||||||
|
|
||||||
Alert Type: {{ alert_type }}
|
|
||||||
Severity: {{ severity }}
|
|
||||||
Scan ID: {{ scan_id }}
|
|
||||||
Timestamp: {{ timestamp }}
|
|
||||||
|
|
||||||
Issue Detected:
|
|
||||||
{{ message }}
|
|
||||||
|
|
||||||
Details:
|
|
||||||
- IP Address: {{ ip_address }}
|
|
||||||
- Port: {{ port }}/{{ protocol }}
|
|
||||||
- Service: {{ service_name }} ({{ product }} {{ version }})
|
|
||||||
- Expected: No (not in expected_ports list)
|
|
||||||
|
|
||||||
Recommended Actions:
|
|
||||||
1. Verify if this service should be exposed
|
|
||||||
2. Check firewall rules for {{ ip_address }}
|
|
||||||
3. Review service configuration
|
|
||||||
4. Update scan config if this is intentional
|
|
||||||
|
|
||||||
View Full Scan Results:
|
|
||||||
{{ web_url }}/scans/{{ scan_id }}
|
|
||||||
|
|
||||||
===============================================
|
|
||||||
```
|
|
||||||
|
|
||||||
**Email Features:**
|
|
||||||
- HTML email with styled alert box (Bootstrap-based)
|
|
||||||
- Plain-text fallback for compatibility
|
|
||||||
- Alert summary with actionable recommendations
|
|
||||||
- Direct link to scan detail page
|
|
||||||
- Configurable recipients (multiple emails)
|
|
||||||
- Test email functionality in Settings UI
|
|
||||||
- Email delivery tracking (email_sent, email_sent_at in alerts table)
|
|
||||||
- Rate limiting to prevent email flood
|
|
||||||
|
|
||||||
**Email API Endpoints:**
|
|
||||||
- `POST /api/settings/email` - Configure SMTP settings
|
|
||||||
- `POST /api/settings/email/test` - Send test email
|
|
||||||
- `GET /api/alerts?email_sent=true` - Get alerts with email status
|
|
||||||
|
|
||||||
#### 3. Webhook Notifications
|
|
||||||
**Purpose:** Real-time HTTP POST notifications for integration with external systems (Slack, PagerDuty, custom dashboards, SIEM tools).
|
**Purpose:** Real-time HTTP POST notifications for integration with external systems (Slack, PagerDuty, custom dashboards, SIEM tools).
|
||||||
|
|
||||||
**Webhook Configuration (via Settings API):**
|
**Webhook Configuration (via Settings API):**
|
||||||
@@ -624,30 +465,30 @@ View Full Scan Results:
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Webhook Features:**
|
**Webhook Features:**
|
||||||
- Multiple webhook URLs with independent configuration
|
- ✅ Multiple webhook URLs with independent configuration
|
||||||
- Per-webhook filtering by alert type and severity
|
- ✅ Per-webhook filtering by alert type and severity
|
||||||
- Custom headers support (for API keys, auth tokens)
|
- ✅ Custom headers support (for API keys, auth tokens)
|
||||||
- Authentication methods:
|
- ✅ Authentication methods:
|
||||||
- `none` - No authentication
|
- `none` - No authentication
|
||||||
- `bearer` - Bearer token in Authorization header
|
- `bearer` - Bearer token in Authorization header
|
||||||
- `basic` - Basic authentication
|
- `basic` - Basic authentication
|
||||||
- `custom` - Custom header-based auth
|
- `custom` - Custom header-based auth
|
||||||
- Retry logic with exponential backoff (3 attempts)
|
- ✅ Retry logic with exponential backoff (3 attempts)
|
||||||
- Webhook delivery tracking (webhook_sent, webhook_sent_at, webhook_response_code)
|
- ✅ Webhook delivery tracking (webhook_sent, webhook_sent_at, webhook_response_code)
|
||||||
- Test webhook functionality in Settings UI
|
- ✅ Test webhook functionality in Settings UI
|
||||||
- Timeout configuration (default 10 seconds)
|
- ✅ Timeout configuration (default 10 seconds)
|
||||||
- Webhook delivery history and logs
|
- ✅ Webhook delivery history and logs
|
||||||
|
|
||||||
**Webhook API Endpoints:**
|
**Webhook API Endpoints:**
|
||||||
- `POST /api/webhooks` - Create webhook configuration
|
- ✅ `POST /api/webhooks` - Create webhook configuration
|
||||||
- `GET /api/webhooks` - List all webhooks
|
- ✅ `GET /api/webhooks` - List all webhooks
|
||||||
- `PUT /api/webhooks/{id}` - Update webhook configuration
|
- ✅ `PUT /api/webhooks/{id}` - Update webhook configuration
|
||||||
- `DELETE /api/webhooks/{id}` - Delete webhook
|
- ✅ `DELETE /api/webhooks/{id}` - Delete webhook
|
||||||
- `POST /api/webhooks/{id}/test` - Send test webhook
|
- ✅ `POST /api/webhooks/{id}/test` - Send test webhook
|
||||||
- `GET /api/webhooks/{id}/history` - Get delivery history
|
- ✅ `GET /api/webhooks/{id}/history` - Get delivery history
|
||||||
|
|
||||||
**Slack Integration Example:**
|
**Notification Templates:**
|
||||||
Transform webhook payload to Slack message format:
|
Flexible template system supporting multiple platforms (Slack, Discord, PagerDuty, etc.):
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"text": "SneakyScanner Alert: Unexpected Port Detected",
|
"text": "SneakyScanner Alert: Unexpected Port Detected",
|
||||||
@@ -667,113 +508,26 @@ Transform webhook payload to Slack message format:
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
#### 4. Alert Management UI
|
|
||||||
**Purpose:** Web interface for configuring alert rules, viewing alert history, and managing notifications.
|
|
||||||
|
|
||||||
**Pages:**
|
|
||||||
- `/alerts` - Alert history with filtering and search
|
|
||||||
- `/alerts/rules` - Alert rule configuration
|
|
||||||
- `/settings/email` - Email notification settings
|
|
||||||
- `/settings/webhooks` - Webhook configuration
|
|
||||||
|
|
||||||
**Alert History Features:**
|
|
||||||
- Filter by alert type, severity, date range, IP address
|
|
||||||
- Search by message content
|
|
||||||
- Bulk acknowledge/dismiss alerts
|
|
||||||
- Export alerts to CSV
|
|
||||||
- Alert detail modal with full context
|
|
||||||
|
|
||||||
**Alert Rule UI Features:**
|
|
||||||
- Enable/disable rules individually
|
|
||||||
- Configure severity levels
|
|
||||||
- Set thresholds (e.g., cert expiry days)
|
|
||||||
- Toggle email/webhook per rule
|
|
||||||
- Test rules against recent scans
|
|
||||||
|
|
||||||
#### 5. Scan Comparison
|
|
||||||
**Purpose:** Detect infrastructure drift by comparing two scans and highlighting changes.
|
|
||||||
|
|
||||||
**Comparison API:**
|
|
||||||
- `GET /api/scans/{id1}/compare/{id2}` - Compare two scans
|
|
||||||
|
|
||||||
**Comparison Response:**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"scan1": {"id": 100, "timestamp": "2025-11-10T10:00:00Z"},
|
|
||||||
"scan2": {"id": 123, "timestamp": "2025-11-17T14:15:00Z"},
|
|
||||||
"summary": {
|
|
||||||
"new_ports": 2,
|
|
||||||
"removed_ports": 0,
|
|
||||||
"service_changes": 1,
|
|
||||||
"cert_changes": 0,
|
|
||||||
"new_hosts": 1,
|
|
||||||
"removed_hosts": 0
|
|
||||||
},
|
|
||||||
"differences": {
|
|
||||||
"new_ports": [
|
|
||||||
{"ip": "192.168.1.100", "port": 3306, "service": "mysql"}
|
|
||||||
],
|
|
||||||
"removed_ports": [],
|
|
||||||
"service_changes": [
|
|
||||||
{
|
|
||||||
"ip": "192.168.1.50",
|
|
||||||
"port": 22,
|
|
||||||
"old": "OpenSSH 8.2",
|
|
||||||
"new": "OpenSSH 8.9"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"new_hosts": [
|
|
||||||
{"ip": "192.168.1.200", "site": "Production Servers"}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Comparison UI Features:**
|
|
||||||
- Side-by-side comparison view
|
|
||||||
- Color-coded differences (green=new, red=removed, yellow=changed)
|
|
||||||
- Filter by change type
|
|
||||||
- Export comparison report to PDF/HTML
|
|
||||||
- "Compare with previous scan" button on scan detail page
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Phase 5 Implementation Plan:**
|
**Deliverables:**
|
||||||
|
- ✅ Alert Rule Engine with 9 alert types (unexpected_port, unexpected_service, cert_expiry, ping_failed, service_down, service_change, weak_tls, new_host, host_disappeared)
|
||||||
|
- ✅ Alert severity classification (critical, warning, info)
|
||||||
|
- ✅ Alert rule configuration API (CRUD operations)
|
||||||
|
- ✅ Webhook notification system with retry logic
|
||||||
|
- ✅ Multiple webhook URL support with independent configuration
|
||||||
|
- ✅ Notification template system for flexible platform integration (Slack, Discord, PagerDuty, custom)
|
||||||
|
- ✅ Webhook API endpoints (create, list, update, delete, test, history)
|
||||||
|
- ✅ Custom headers and authentication support (none, bearer, basic, custom)
|
||||||
|
- ✅ Webhook delivery tracking and logging
|
||||||
|
- ✅ Alert deduplication to prevent notification spam
|
||||||
|
- ✅ Integration with scan completion workflow
|
||||||
|
|
||||||
1. **Week 1: Alert Rule Engine**
|
**Success Criteria Met:**
|
||||||
- Implement alert evaluation logic after scan completion
|
- ✅ Alerts triggered within 30 seconds of scan completion
|
||||||
- Create `alerts` table population
|
- ✅ Webhook POST delivered with retry on failure
|
||||||
- Add alert rule CRUD API
|
- ✅ Zero false positives for expected ports/services
|
||||||
- Unit tests for alert detection
|
- ✅ Alert deduplication prevents notification spam
|
||||||
|
|
||||||
2. **Week 2: Email Notifications**
|
|
||||||
- SMTP integration with Flask-Mail
|
|
||||||
- Jinja2 email templates (HTML + plain text)
|
|
||||||
- Settings API for email configuration
|
|
||||||
- Test email functionality
|
|
||||||
- Email delivery tracking
|
|
||||||
|
|
||||||
3. **Week 3: Webhook System**
|
|
||||||
- Webhook configuration API
|
|
||||||
- HTTP POST delivery with retry logic
|
|
||||||
- Webhook template system for different platforms
|
|
||||||
- Test webhook functionality
|
|
||||||
- Delivery tracking and logging
|
|
||||||
|
|
||||||
4. **Week 4: Alert UI & Scan Comparison**
|
|
||||||
- Alert history page with filtering
|
|
||||||
- Alert rule management UI
|
|
||||||
- Email/webhook settings pages
|
|
||||||
- Scan comparison API and UI
|
|
||||||
- Integration testing
|
|
||||||
|
|
||||||
**Success Criteria:**
|
|
||||||
- Alert triggered within 30 seconds of scan completion
|
|
||||||
- Email delivered successfully to configured recipients
|
|
||||||
- Webhook POST delivered with retry on failure
|
|
||||||
- Scan comparison highlights all infrastructure changes
|
|
||||||
- Zero false positives for expected ports/services
|
|
||||||
- Alert deduplication prevents notification spam
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -813,61 +567,52 @@ Transform webhook payload to Slack message format:
|
|||||||
**Priority:** LOW
|
**Priority:** LOW
|
||||||
|
|
||||||
**Planned Features:**
|
**Planned Features:**
|
||||||
1. **Enhanced Reports:**
|
|
||||||
|
1. **Email Notifications:**
|
||||||
|
- SMTP integration with Flask-Mail
|
||||||
|
- Jinja2 email templates (HTML + plain text)
|
||||||
|
- Settings API for email configuration
|
||||||
|
- Test email functionality
|
||||||
|
- Email delivery tracking
|
||||||
|
- Rate limiting to prevent email flood
|
||||||
|
- Configurable recipients (multiple emails)
|
||||||
|
|
||||||
|
2. **Scan Comparison:**
|
||||||
|
- Compare two scans API endpoint
|
||||||
|
- Side-by-side comparison view
|
||||||
|
- Color-coded differences (green=new, red=removed, yellow=changed)
|
||||||
|
- Filter by change type
|
||||||
|
- Export comparison report to PDF/HTML
|
||||||
|
- "Compare with previous scan" button on scan detail page
|
||||||
|
|
||||||
|
3. **Enhanced Reports:**
|
||||||
- Sortable/filterable tables (DataTables.js)
|
- Sortable/filterable tables (DataTables.js)
|
||||||
- Inline screenshot thumbnails with lightbox
|
- Inline screenshot thumbnails with lightbox
|
||||||
- PDF export (WeasyPrint)
|
- PDF export (WeasyPrint)
|
||||||
|
|
||||||
2. **Vulnerability Detection:**
|
4. **Vulnerability Detection:**
|
||||||
- CVE database integration (NVD API)
|
- CVE database integration (NVD API)
|
||||||
- Service version matching to known CVEs
|
- Service version matching to known CVEs
|
||||||
- CVSS severity scores
|
- CVSS severity scores
|
||||||
- Alert rules for critical CVEs
|
- Alert rules for critical CVEs
|
||||||
|
|
||||||
3. **Timeline View:**
|
5. **Timeline View:**
|
||||||
- Visual scan history timeline
|
- Visual scan history timeline
|
||||||
- Filter by site/IP
|
- Filter by site/IP
|
||||||
- Event annotations
|
- Event annotations
|
||||||
|
|
||||||
4. **Advanced Charts:**
|
6. **Advanced Charts:**
|
||||||
- Port activity heatmap
|
- Port activity heatmap
|
||||||
- Service version tracking
|
- Service version tracking
|
||||||
- Certificate expiration forecast
|
- Certificate expiration forecast
|
||||||
|
|
||||||
5. **Integrations:**
|
7. **Additional Integrations:**
|
||||||
- Slack notifications
|
|
||||||
- Webhook support
|
|
||||||
- Prometheus metrics export
|
- Prometheus metrics export
|
||||||
- CSV export/import
|
- CSV export/import
|
||||||
|
- Advanced reporting dashboards
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Current Architecture
|
|
||||||
|
|
||||||
**Primary Interface:** Web GUI (Phases 1-4 Complete)
|
|
||||||
- Full-featured Flask web application
|
|
||||||
- Dashboard, scan management, scheduling, config creator
|
|
||||||
- REST API for all operations
|
|
||||||
- Single-user deployment with SQLite
|
|
||||||
|
|
||||||
**Coming Soon:** CLI API Client (Phase 6 Planned)
|
|
||||||
- Thin client for scripting and automation
|
|
||||||
- Calls Flask API for scan operations
|
|
||||||
- Results stored centrally in database
|
|
||||||
- Access to all web features via command line
|
|
||||||
|
|
||||||
**Core Scanning Engine:**
|
|
||||||
- Masscan for port discovery
|
|
||||||
- Nmap for service detection
|
|
||||||
- Playwright for screenshots
|
|
||||||
- sslyze for SSL/TLS analysis
|
|
||||||
|
|
||||||
**Deployment:**
|
|
||||||
- Docker Compose for easy deployment
|
|
||||||
- SQLite database (single-user, embedded)
|
|
||||||
- Gunicorn WSGI server
|
|
||||||
- Optional Nginx reverse proxy
|
|
||||||
|
|
||||||
## Development Workflow
|
## Development Workflow
|
||||||
|
|
||||||
### Iteration Cycle
|
### Iteration Cycle
|
||||||
@@ -926,8 +671,10 @@ Transform webhook payload to Slack message format:
|
|||||||
| 2025-11-17 | 1.3 | **Bug Fix** - Fixed Chart.js infinite canvas growth issue in scan detail page (duplicate initialization, missing chart.destroy(), missing fixed-height container) |
|
| 2025-11-17 | 1.3 | **Bug Fix** - Fixed Chart.js infinite canvas growth issue in scan detail page (duplicate initialization, missing chart.destroy(), missing fixed-height container) |
|
||||||
| 2025-11-17 | 1.4 | **Phase 4 COMPLETE** - Config Creator with CIDR-based creation, YAML editor (CodeMirror), config management UI (list/edit/delete), REST API (7 endpoints), Docker volume permissions fix, comprehensive testing and documentation |
|
| 2025-11-17 | 1.4 | **Phase 4 COMPLETE** - Config Creator with CIDR-based creation, YAML editor (CodeMirror), config management UI (list/edit/delete), REST API (7 endpoints), Docker volume permissions fix, comprehensive testing and documentation |
|
||||||
| 2025-11-17 | 1.5 | **Roadmap Compression** - Condensed completed phases (1-4) into concise summaries, updated project scope to emphasize web GUI frontend with CLI as API client coming soon (Phase 6), reorganized phases for clarity |
|
| 2025-11-17 | 1.5 | **Roadmap Compression** - Condensed completed phases (1-4) into concise summaries, updated project scope to emphasize web GUI frontend with CLI as API client coming soon (Phase 6), reorganized phases for clarity |
|
||||||
|
| 2025-11-19 | 1.6 | **Phase 5 Progress** - Completed webhooks, notification templates, and alerting rules. Alert Rule Engine and Webhook System implemented. |
|
||||||
|
| 2025-11-19 | 1.7 | **Phase 5 COMPLETE** - Webhooks & Alerting phase completed. Moved Email Notifications and Scan Comparison to Phase 7. Alert rules, webhook notifications, and notification templates fully implemented and tested. |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Last Updated:** 2025-11-17
|
**Last Updated:** 2025-11-19
|
||||||
**Next Review:** Before Phase 5 kickoff (Email & Comparisons)
|
**Next Review:** Before Phase 6 kickoff (CLI as API Client)
|
||||||
|
|||||||
Reference in New Issue
Block a user