restructure of dirs, huge docs update
This commit is contained in:
File diff suppressed because it is too large
Load Diff
502
docs/CLI_SCANNING.md
Normal file
502
docs/CLI_SCANNING.md
Normal file
@@ -0,0 +1,502 @@
|
||||
# CLI Scanner Guide
|
||||
|
||||
The SneakyScanner CLI provides a standalone scanning tool for quick one-off scans, testing, or CI/CD pipelines without requiring the web application.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Quick Start](#quick-start)
|
||||
2. [Configuration](#configuration)
|
||||
3. [Scan Performance](#scan-performance)
|
||||
4. [Output Formats](#output-formats)
|
||||
5. [Screenshot Capture](#screenshot-capture)
|
||||
6. [HTML Reports](#html-reports)
|
||||
7. [Advanced Usage](#advanced-usage)
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Using Docker Compose (Recommended)
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker compose -f docker-compose-standalone.yml build
|
||||
|
||||
# Run a scan
|
||||
docker compose -f docker-compose-standalone.yml up
|
||||
|
||||
# Results saved to ./output/ directory
|
||||
```
|
||||
|
||||
### Using Docker Directly
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker build -t sneakyscanner .
|
||||
|
||||
# Run a scan
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v $(pwd)/output:/app/output \
|
||||
sneakyscanner /app/configs/your-config.yaml
|
||||
```
|
||||
|
||||
### Requirements
|
||||
|
||||
- Docker
|
||||
- Linux host (required for `--privileged` and `--network host`)
|
||||
- Configuration file in `configs/` directory
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
The YAML configuration file defines scan parameters and expectations.
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
```yaml
|
||||
title: "My Infrastructure Scan"
|
||||
sites:
|
||||
- name: "Web Servers"
|
||||
ips:
|
||||
- address: "192.168.1.10"
|
||||
expected:
|
||||
ping: true
|
||||
tcp_ports: [22, 80, 443]
|
||||
udp_ports: []
|
||||
services: ["ssh", "http", "https"]
|
||||
```
|
||||
|
||||
### CIDR Range Configuration
|
||||
|
||||
```yaml
|
||||
title: "Network Scan"
|
||||
sites:
|
||||
- name: "Production Network"
|
||||
cidr: "192.168.1.0/24"
|
||||
expected_ports:
|
||||
- port: 22
|
||||
protocol: tcp
|
||||
service: "ssh"
|
||||
- port: 80
|
||||
protocol: tcp
|
||||
service: "http"
|
||||
- port: 443
|
||||
protocol: tcp
|
||||
service: "https"
|
||||
ping_expected: true
|
||||
```
|
||||
|
||||
### Configuration Reference
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `title` | string | Yes | Report title |
|
||||
| `sites` | array | Yes | List of sites to scan |
|
||||
| `sites[].name` | string | Yes | Site name |
|
||||
| `sites[].ips` | array | Conditional | List of IP addresses (if not using CIDR) |
|
||||
| `sites[].cidr` | string | Conditional | CIDR range (if not using IPs) |
|
||||
| `sites[].ips[].address` | string | Yes | IP address |
|
||||
| `sites[].ips[].expected.ping` | boolean | No | Expected ping response |
|
||||
| `sites[].ips[].expected.tcp_ports` | array | No | Expected TCP ports |
|
||||
| `sites[].ips[].expected.udp_ports` | array | No | Expected UDP ports |
|
||||
| `sites[].ips[].expected.services` | array | No | Expected service names |
|
||||
| `sites[].expected_ports` | array | No | Expected ports for CIDR range |
|
||||
| `sites[].ping_expected` | boolean | No | Expected ping for CIDR range |
|
||||
|
||||
See `configs/example-site.yaml` for a complete example.
|
||||
|
||||
---
|
||||
|
||||
## Scan Performance
|
||||
|
||||
SneakyScanner uses a five-phase approach for comprehensive scanning:
|
||||
|
||||
1. **Ping Scan** (masscan): ICMP echo detection
|
||||
- Duration: ~1-2 seconds
|
||||
- Tests network reachability
|
||||
|
||||
2. **TCP Port Discovery** (masscan): Scans all 65535 TCP ports
|
||||
- Rate: 10,000 packets/second
|
||||
- Duration: ~13 seconds per 2 IPs
|
||||
|
||||
3. **UDP Port Discovery** (masscan): Scans all 65535 UDP ports
|
||||
- Rate: 10,000 packets/second
|
||||
- Duration: ~13 seconds per 2 IPs
|
||||
|
||||
4. **Service Detection** (nmap): Identifies services on discovered TCP ports
|
||||
- Intensity level: 5 (balanced)
|
||||
- Duration: ~20-60 seconds per IP with open ports
|
||||
- Extracts product names and versions
|
||||
|
||||
5. **HTTP/HTTPS Analysis**: Web protocol detection and SSL/TLS analysis
|
||||
- Screenshot capture (Playwright)
|
||||
- Certificate extraction (sslyze)
|
||||
- TLS version testing
|
||||
- Duration: ~10-20 seconds per web service
|
||||
|
||||
**Example**: Scanning 2 IPs with 10 open ports each (including 2-3 web services) typically takes 2-3 minutes total.
|
||||
|
||||
### Performance Tuning
|
||||
|
||||
Adjust scan rate in the scanner code if needed:
|
||||
- Default: 10,000 pps (packets per second)
|
||||
- Increase for faster scans (may cause network congestion)
|
||||
- Decrease for slower, more reliable scans
|
||||
|
||||
---
|
||||
|
||||
## Output Formats
|
||||
|
||||
After each scan completes, SneakyScanner automatically generates three output formats:
|
||||
|
||||
### 1. JSON Report
|
||||
|
||||
**Filename**: `scan_report_YYYYMMDD_HHMMSS.json`
|
||||
|
||||
Machine-readable scan data with all discovered services, ports, and SSL/TLS information.
|
||||
|
||||
**Structure**:
|
||||
```json
|
||||
{
|
||||
"title": "Scan Title",
|
||||
"scan_time": "2025-01-15T10:30:00Z",
|
||||
"scan_duration": 95.3,
|
||||
"config_file": "/app/configs/example-site.yaml",
|
||||
"sites": [
|
||||
{
|
||||
"name": "Site Name",
|
||||
"ips": [
|
||||
{
|
||||
"address": "192.168.1.10",
|
||||
"expected": {...},
|
||||
"actual": {
|
||||
"ping": true,
|
||||
"tcp_ports": [22, 80, 443],
|
||||
"udp_ports": [],
|
||||
"services": [...]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 2. HTML Report
|
||||
|
||||
**Filename**: `scan_report_YYYYMMDD_HHMMSS.html`
|
||||
|
||||
Human-readable report with dark theme, summary dashboard, and detailed service breakdown.
|
||||
|
||||
Features:
|
||||
- Summary statistics
|
||||
- Drift alerts (unexpected ports/services)
|
||||
- Security warnings (expiring certs, weak TLS)
|
||||
- Site-by-site breakdown
|
||||
- Expandable service details
|
||||
- SSL/TLS certificate information
|
||||
|
||||
See [HTML Reports](#html-reports) section for details.
|
||||
|
||||
### 3. ZIP Archive
|
||||
|
||||
**Filename**: `scan_report_YYYYMMDD_HHMMSS.zip`
|
||||
|
||||
Contains:
|
||||
- JSON report
|
||||
- HTML report
|
||||
- All screenshots (if web services were found)
|
||||
|
||||
Useful for:
|
||||
- Easy sharing
|
||||
- Archival
|
||||
- Compliance documentation
|
||||
|
||||
### 4. Screenshots Directory
|
||||
|
||||
**Directory**: `scan_report_YYYYMMDD_HHMMSS_screenshots/`
|
||||
|
||||
PNG screenshots of all discovered web services:
|
||||
- Filename format: `{ip}_{port}.png` (e.g., `192_168_1_10_443.png`)
|
||||
- Viewport size: 1280x720
|
||||
- Referenced in JSON report under `http_info.screenshot`
|
||||
|
||||
---
|
||||
|
||||
## Screenshot Capture
|
||||
|
||||
SneakyScanner automatically captures webpage screenshots for all discovered HTTP and HTTPS services.
|
||||
|
||||
### Automatic Detection
|
||||
|
||||
Screenshots are captured for services identified as web services based on:
|
||||
- Nmap service detection results (http, https, ssl, http-proxy)
|
||||
- Common web ports (80, 443, 8000, 8006, 8080, 8081, 8443, 8888, 9443)
|
||||
|
||||
### Capture Process
|
||||
|
||||
For each web service:
|
||||
|
||||
1. **Launch Browser**: Headless Chromium (once per scan, reused)
|
||||
2. **Navigate**: To service URL (HTTP or HTTPS)
|
||||
3. **Wait**: For network to be idle (up to 15 seconds)
|
||||
4. **Capture**: Viewport screenshot (1280x720 pixels)
|
||||
5. **Save**: As PNG file in screenshots directory
|
||||
|
||||
### Configuration
|
||||
|
||||
Default settings (configured in `src/screenshot_capture.py`):
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Viewport size | 1280x720 |
|
||||
| Timeout | 15 seconds |
|
||||
| Browser | Chromium (headless) |
|
||||
| SSL handling | Ignores HTTPS errors |
|
||||
| User agent | Mozilla/5.0 (Windows NT 10.0; Win64; x64) |
|
||||
|
||||
### Error Handling
|
||||
|
||||
Screenshots are captured on a best-effort basis:
|
||||
- Failed screenshots are logged but don't stop the scan
|
||||
- Services without screenshots omit the `screenshot` field in JSON
|
||||
- Common errors: timeout, connection refused, invalid SSL
|
||||
|
||||
### Disabling Screenshots
|
||||
|
||||
To disable screenshot capture, modify `src/screenshot_capture.py` or comment out the screenshot phase in `src/scanner.py`.
|
||||
|
||||
---
|
||||
|
||||
## HTML Reports
|
||||
|
||||
SneakyScanner automatically generates comprehensive HTML reports after each scan.
|
||||
|
||||
### Automatic Generation
|
||||
|
||||
HTML reports are created after every scan, along with JSON reports and ZIP archives. All outputs share the same timestamp.
|
||||
|
||||
### Manual Generation
|
||||
|
||||
Generate HTML reports from existing JSON scan data:
|
||||
|
||||
```bash
|
||||
# Generate HTML report (creates report in same directory as JSON)
|
||||
cd app/
|
||||
python3 src/report_generator.py ../output/scan_report_20250115_103000.json
|
||||
|
||||
# Specify custom output path
|
||||
python3 src/report_generator.py ../output/scan_report.json /path/to/custom_report.html
|
||||
```
|
||||
|
||||
### Report Features
|
||||
|
||||
**Summary Dashboard**:
|
||||
- **Scan Statistics**: Total IPs, TCP/UDP ports, services, web services, screenshots
|
||||
- **Drift Alerts**: Unexpected ports, missing services, new services
|
||||
- **Security Warnings**: Expiring certificates (<30 days), weak TLS (1.0/1.1), self-signed certs, high ports (>10000)
|
||||
|
||||
**Site-by-Site Breakdown**:
|
||||
- Organized by logical site grouping from configuration
|
||||
- Per-IP sections with status badges (ping, port drift)
|
||||
- Service tables with expandable details (click to expand)
|
||||
- Visual badges: green (expected), red (unexpected), yellow (missing/warning)
|
||||
|
||||
**Service Details** (expandable):
|
||||
- Product name, version, extra information, OS type
|
||||
- HTTP/HTTPS protocol detection
|
||||
- Screenshot links for web services
|
||||
- SSL/TLS certificate details:
|
||||
- Subject, issuer, validity dates, serial number
|
||||
- Days until expiration (color-coded warnings)
|
||||
- Subject Alternative Names (SANs)
|
||||
- TLS version support (1.0, 1.1, 1.2, 1.3) with cipher suites
|
||||
- Weak TLS and self-signed certificate warnings
|
||||
|
||||
**UDP Port Handling**:
|
||||
- Expected UDP ports: green "Expected" badge
|
||||
- Unexpected UDP ports: red "Unexpected" badge
|
||||
- Missing UDP ports: yellow "Missing" badge
|
||||
- Note: Service detection not available for UDP (nmap limitation)
|
||||
|
||||
**Design**:
|
||||
- Dark theme with slate/grey color scheme
|
||||
- Responsive layout
|
||||
- No external dependencies (single HTML file)
|
||||
- Minimal JavaScript for expand/collapse
|
||||
- Optimized hover effects
|
||||
|
||||
### Report Portability
|
||||
|
||||
The HTML report is a standalone file that can be:
|
||||
- Opened in any web browser (Chrome, Firefox, Safari, Edge)
|
||||
- Shared via email or file transfer
|
||||
- Archived for compliance or historical comparison
|
||||
- Viewed without internet connection
|
||||
|
||||
**Note**: Screenshot links use relative paths, so keep the report and screenshot directory together.
|
||||
|
||||
---
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Running on Remote Targets
|
||||
|
||||
```bash
|
||||
# Scan remote network via Docker host
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v $(pwd)/output:/app/output \
|
||||
sneakyscanner /app/configs/remote-network.yaml
|
||||
```
|
||||
|
||||
**Note**: The Docker host must have network access to the target network.
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
```yaml
|
||||
# Example GitLab CI pipeline
|
||||
scan-infrastructure:
|
||||
stage: test
|
||||
image: docker:latest
|
||||
services:
|
||||
- docker:dind
|
||||
script:
|
||||
- docker build -t sneakyscanner .
|
||||
- docker run --rm --privileged --network host \
|
||||
-v $PWD/configs:/app/configs:ro \
|
||||
-v $PWD/output:/app/output \
|
||||
sneakyscanner /app/configs/production.yaml
|
||||
artifacts:
|
||||
paths:
|
||||
- output/
|
||||
expire_in: 30 days
|
||||
```
|
||||
|
||||
### Batch Scanning
|
||||
|
||||
```bash
|
||||
# Scan multiple configs sequentially
|
||||
for config in configs/*.yaml; do
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v $(pwd)/output:/app/output \
|
||||
sneakyscanner "/app/configs/$(basename $config)"
|
||||
done
|
||||
```
|
||||
|
||||
### Custom Output Directory
|
||||
|
||||
```bash
|
||||
# Use custom output directory
|
||||
mkdir -p /path/to/custom/output
|
||||
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v /path/to/custom/output:/app/output \
|
||||
sneakyscanner /app/configs/config.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Permission Denied Errors
|
||||
|
||||
**Problem**: masscan or nmap fails with permission denied
|
||||
|
||||
**Solution**: Ensure Docker is running with `--privileged` flag:
|
||||
```bash
|
||||
docker run --rm --privileged --network host ...
|
||||
```
|
||||
|
||||
### No Ports Found
|
||||
|
||||
**Problem**: Scan completes but finds no open ports
|
||||
|
||||
**Possible Causes**:
|
||||
- Firewall blocking scans
|
||||
- Wrong network (ensure `--network host`)
|
||||
- Target hosts are down
|
||||
- Incorrect IP addresses in config
|
||||
|
||||
**Debug**:
|
||||
```bash
|
||||
# Test ping manually
|
||||
ping 192.168.1.10
|
||||
|
||||
# Check Docker network mode
|
||||
docker inspect <container-id> | grep NetworkMode
|
||||
```
|
||||
|
||||
### Screenshots Failing
|
||||
|
||||
**Problem**: Screenshots not being captured
|
||||
|
||||
**Possible Causes**:
|
||||
- Chromium not installed (check Dockerfile)
|
||||
- Timeout too short (increase in screenshot_capture.py)
|
||||
- Web service requires authentication
|
||||
- SSL certificate errors
|
||||
|
||||
**Debug**: Check scan logs for screenshot errors
|
||||
|
||||
### Scan Takes Too Long
|
||||
|
||||
**Problem**: Scan runs for 30+ minutes
|
||||
|
||||
**Solutions**:
|
||||
- Reduce scan rate (edit scanner.py)
|
||||
- Limit port range (edit scanner.py to scan specific ports)
|
||||
- Reduce number of IPs in config
|
||||
- Disable UDP scanning if not needed
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Privileged Mode
|
||||
|
||||
The CLI scanner requires `--privileged` flag for:
|
||||
- Raw socket access (masscan, nmap)
|
||||
- ICMP echo requests (ping)
|
||||
|
||||
**Security implications**:
|
||||
- Container has extensive host capabilities
|
||||
- Only run on trusted networks
|
||||
- Don't expose to public networks
|
||||
|
||||
### Network Mode: Host
|
||||
|
||||
The scanner uses `--network host` for:
|
||||
- Direct network access without NAT
|
||||
- Raw packet sending
|
||||
- Accurate service detection
|
||||
|
||||
**Security implications**:
|
||||
- Container shares host network namespace
|
||||
- Can access all host network interfaces
|
||||
- Bypass Docker network isolation
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. **Only scan authorized networks**
|
||||
2. **Run on dedicated scan server** (not production)
|
||||
3. **Limit network access** with firewall rules
|
||||
4. **Review scan configs** before running
|
||||
5. **Store results securely** (may contain sensitive data)
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
- **Deployment Guide**: [docs/DEPLOYMENT.md](DEPLOYMENT.md)
|
||||
- **API Reference**: [docs/API_REFERENCE.md](API_REFERENCE.md)
|
||||
- **Roadmap**: [docs/ROADMAP.md](ROADMAP.md)
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-17
|
||||
**Version**: Phase 4 Complete
|
||||
@@ -8,12 +8,13 @@
|
||||
4. [Configuration](#configuration)
|
||||
5. [First-Time Setup](#first-time-setup)
|
||||
6. [Running the Application](#running-the-application)
|
||||
7. [Volume Management](#volume-management)
|
||||
8. [Health Monitoring](#health-monitoring)
|
||||
9. [Troubleshooting](#troubleshooting)
|
||||
10. [Security Considerations](#security-considerations)
|
||||
11. [Upgrading](#upgrading)
|
||||
12. [Backup and Restore](#backup-and-restore)
|
||||
7. [Using the Web Interface](#using-the-web-interface)
|
||||
8. [Volume Management](#volume-management)
|
||||
9. [Health Monitoring](#health-monitoring)
|
||||
10. [Troubleshooting](#troubleshooting)
|
||||
11. [Security Considerations](#security-considerations)
|
||||
12. [Upgrading](#upgrading)
|
||||
13. [Backup and Restore](#backup-and-restore)
|
||||
|
||||
---
|
||||
|
||||
@@ -22,10 +23,12 @@
|
||||
SneakyScanner is deployed as a Docker container running a Flask web application with an integrated network scanner. The application requires privileged mode and host networking to perform network scans using masscan and nmap.
|
||||
|
||||
**Architecture:**
|
||||
- **Web Application**: Flask app on port 5000
|
||||
- **Web Application**: Flask app on port 5000 with modern web UI
|
||||
- **Database**: SQLite (persisted to volume)
|
||||
- **Background Jobs**: APScheduler for async scan execution
|
||||
- **Scanner**: masscan, nmap, sslyze, Playwright
|
||||
- **Config Creator**: Web-based CIDR-to-YAML configuration builder
|
||||
- **Scheduling**: Cron-based scheduled scans with dashboard management
|
||||
|
||||
---
|
||||
|
||||
@@ -69,7 +72,7 @@ docker compose version
|
||||
|
||||
## Quick Start
|
||||
|
||||
For users who want to get started immediately:
|
||||
For users who want to get started immediately with the web application:
|
||||
|
||||
```bash
|
||||
# 1. Clone the repository
|
||||
@@ -82,18 +85,32 @@ cp .env.example .env
|
||||
nano .env
|
||||
|
||||
# 3. Build the Docker image
|
||||
docker compose -f docker-compose-web.yml build
|
||||
docker compose build
|
||||
|
||||
# 4. Initialize the database and set password
|
||||
docker compose -f docker-compose-web.yml run --rm init-db --password "YourSecurePassword"
|
||||
docker compose run --rm init-db --password "YourSecurePassword"
|
||||
|
||||
# 5. Start the application
|
||||
docker compose -f docker-compose-web.yml up -d
|
||||
docker compose up -d
|
||||
|
||||
# 6. Access the web interface
|
||||
# Open browser to: http://localhost:5000
|
||||
```
|
||||
|
||||
**Alternative: Standalone CLI Scanner**
|
||||
|
||||
For quick one-off scans without the web interface:
|
||||
|
||||
```bash
|
||||
# Build and run with standalone compose file
|
||||
docker compose -f docker-compose-standalone.yml build
|
||||
docker compose -f docker-compose-standalone.yml up
|
||||
|
||||
# Results saved to ./output/ directory
|
||||
```
|
||||
|
||||
**Note**: `docker-compose.yml` (web application) is now the default. Use `docker-compose-standalone.yml` for CLI-only scans.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
@@ -153,7 +170,23 @@ mkdir -p configs data output logs
|
||||
|
||||
### Step 2: Configure Scan Targets
|
||||
|
||||
Create YAML configuration files for your scan targets:
|
||||
You can create scan configurations in two ways:
|
||||
|
||||
**Option A: Using the Web UI (Recommended - Phase 4 Feature)**
|
||||
|
||||
1. Navigate to **Configs** in the web interface
|
||||
2. Click **"Create New Config"**
|
||||
3. Use the CIDR-based config creator for quick setup:
|
||||
- Enter site name
|
||||
- Enter CIDR range (e.g., `192.168.1.0/24`)
|
||||
- Select expected ports from dropdowns
|
||||
- Click **"Generate Config"**
|
||||
4. Or use the **YAML Editor** for advanced configurations
|
||||
5. Save and use immediately in scans or schedules
|
||||
|
||||
**Option B: Manual YAML Files**
|
||||
|
||||
Create YAML configuration files manually in the `configs/` directory:
|
||||
|
||||
```bash
|
||||
# Example configuration
|
||||
@@ -161,21 +194,28 @@ cat > configs/my-network.yaml <<EOF
|
||||
title: "My Network Infrastructure"
|
||||
sites:
|
||||
- name: "Web Servers"
|
||||
ips:
|
||||
- address: "192.168.1.10"
|
||||
expected:
|
||||
ping: true
|
||||
tcp_ports: [80, 443]
|
||||
udp_ports: []
|
||||
services: ["http", "https"]
|
||||
cidr: "192.168.1.0/24" # Scan entire subnet
|
||||
expected_ports:
|
||||
- port: 80
|
||||
protocol: tcp
|
||||
service: "http"
|
||||
- port: 443
|
||||
protocol: tcp
|
||||
service: "https"
|
||||
- port: 22
|
||||
protocol: tcp
|
||||
service: "ssh"
|
||||
ping_expected: true
|
||||
EOF
|
||||
```
|
||||
|
||||
**Note**: Phase 4 introduced a powerful config creator in the web UI that makes it easy to generate configs from CIDR ranges without manually editing YAML.
|
||||
|
||||
### Step 3: Build Docker Image
|
||||
|
||||
```bash
|
||||
# Build the image (takes 5-10 minutes on first run)
|
||||
docker compose -f docker-compose-web.yml build
|
||||
docker compose -f docker-compose.yml build
|
||||
|
||||
# Verify image was created
|
||||
docker images | grep sneakyscanner
|
||||
@@ -183,17 +223,20 @@ docker images | grep sneakyscanner
|
||||
|
||||
### Step 4: Initialize Database
|
||||
|
||||
The database must be initialized before first use:
|
||||
The database must be initialized before first use. The init-db service uses a profile, so you need to explicitly run it:
|
||||
|
||||
```bash
|
||||
# Initialize database and set application password
|
||||
docker compose -f docker-compose-web.yml run --rm init-db --password "YourSecurePassword"
|
||||
docker compose -f docker-compose.yml run --rm init-db --password "YourSecurePassword"
|
||||
|
||||
# The init-db command will:
|
||||
# - Create database schema
|
||||
# - Run all Alembic migrations
|
||||
# - Set the application password
|
||||
# - Create default settings
|
||||
# - Set the application password (bcrypt hashed)
|
||||
# - Create default settings with encryption
|
||||
|
||||
# Verify database was created
|
||||
ls -lh data/sneakyscanner.db
|
||||
```
|
||||
|
||||
**Password Requirements:**
|
||||
@@ -201,6 +244,8 @@ docker compose -f docker-compose-web.yml run --rm init-db --password "YourSecure
|
||||
- Use a strong, unique password
|
||||
- Store securely (password manager)
|
||||
|
||||
**Note**: The init-db service is defined with `profiles: [tools]` in docker-compose.yml, which means it won't start automatically with `docker compose up`.
|
||||
|
||||
### Step 5: Verify Configuration
|
||||
|
||||
```bash
|
||||
@@ -208,7 +253,7 @@ docker compose -f docker-compose-web.yml run --rm init-db --password "YourSecure
|
||||
ls -lh data/sneakyscanner.db
|
||||
|
||||
# Verify Docker Compose configuration
|
||||
docker compose -f docker-compose-web.yml config
|
||||
docker compose -f docker-compose.yml config
|
||||
```
|
||||
|
||||
---
|
||||
@@ -219,10 +264,10 @@ docker compose -f docker-compose-web.yml config
|
||||
|
||||
```bash
|
||||
# Start in detached mode (background)
|
||||
docker compose -f docker-compose-web.yml up -d
|
||||
docker compose -f docker-compose.yml up -d
|
||||
|
||||
# View logs during startup
|
||||
docker compose -f docker-compose-web.yml logs -f web
|
||||
docker compose -f docker-compose.yml logs -f web
|
||||
|
||||
# Expected output:
|
||||
# web_1 | INFO:werkzeug: * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
|
||||
@@ -231,47 +276,143 @@ docker compose -f docker-compose-web.yml logs -f web
|
||||
### Accessing the Web Interface
|
||||
|
||||
1. Open browser to: **http://localhost:5000**
|
||||
2. Login with the password you set during database initialization
|
||||
3. Dashboard will display recent scans and statistics
|
||||
2. Login with the password you set during database initialization (username is not required - single-user mode)
|
||||
3. Dashboard will display:
|
||||
- Recent scans with status indicators
|
||||
- Summary statistics (total scans, IPs, ports, services)
|
||||
- Trend charts showing infrastructure changes over time
|
||||
- Quick actions (run scan, create config, view schedules)
|
||||
|
||||
### Stopping the Application
|
||||
|
||||
```bash
|
||||
# Stop containers (preserves data)
|
||||
docker compose -f docker-compose-web.yml down
|
||||
docker compose -f docker-compose.yml down
|
||||
|
||||
# Stop and remove volumes (WARNING: deletes all data!)
|
||||
docker compose -f docker-compose-web.yml down -v
|
||||
docker compose -f docker-compose.yml down -v
|
||||
```
|
||||
|
||||
### Restarting the Application
|
||||
|
||||
```bash
|
||||
# Restart all services
|
||||
docker compose -f docker-compose-web.yml restart
|
||||
docker compose -f docker-compose.yml restart
|
||||
|
||||
# Restart only the web service
|
||||
docker compose -f docker-compose-web.yml restart web
|
||||
docker compose -f docker-compose.yml restart web
|
||||
```
|
||||
|
||||
### Viewing Logs
|
||||
|
||||
```bash
|
||||
# View all logs
|
||||
docker compose -f docker-compose-web.yml logs
|
||||
docker compose -f docker-compose.yml logs
|
||||
|
||||
# Follow logs in real-time
|
||||
docker compose -f docker-compose-web.yml logs -f
|
||||
docker compose -f docker-compose.yml logs -f
|
||||
|
||||
# View last 100 lines
|
||||
docker compose -f docker-compose-web.yml logs --tail=100
|
||||
docker compose -f docker-compose.yml logs --tail=100
|
||||
|
||||
# View logs for specific service
|
||||
docker compose -f docker-compose-web.yml logs web
|
||||
docker compose -f docker-compose.yml logs web
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using the Web Interface
|
||||
|
||||
### Dashboard Overview
|
||||
|
||||
The dashboard provides a central view of your scanning activity:
|
||||
|
||||
**Key Sections:**
|
||||
- **Summary Statistics**: Total scans, IPs discovered, open ports, services detected
|
||||
- **Recent Scans**: Last 10 scans with status, timestamp, and quick actions
|
||||
- **Trend Charts**: Port count trends over time using Chart.js
|
||||
- **Quick Actions**: Buttons to run scans, create configs, manage schedules
|
||||
|
||||
### Managing Scan Configurations (Phase 4)
|
||||
|
||||
**Creating Configs:**
|
||||
1. Navigate to **Configs** → **Create New Config**
|
||||
2. **CIDR Creator Mode**:
|
||||
- Enter site name (e.g., "Production Servers")
|
||||
- Enter CIDR range (e.g., `192.168.1.0/24`)
|
||||
- Select expected TCP/UDP ports from dropdowns
|
||||
- Click **"Generate Config"** to create YAML
|
||||
3. **YAML Editor Mode**:
|
||||
- Switch to editor tab for advanced configurations
|
||||
- Syntax highlighting with line numbers
|
||||
- Validate YAML before saving
|
||||
|
||||
**Editing Configs:**
|
||||
1. Navigate to **Configs** → Select config
|
||||
2. Click **"Edit"** button
|
||||
3. Make changes in YAML editor
|
||||
4. Save changes (validates YAML automatically)
|
||||
|
||||
**Uploading Configs:**
|
||||
1. Navigate to **Configs** → **Upload**
|
||||
2. Select YAML file from your computer
|
||||
3. File is validated and saved to `configs/` directory
|
||||
|
||||
**Downloading Configs:**
|
||||
- Click **"Download"** button next to any config
|
||||
- Saves YAML file to your local machine
|
||||
|
||||
**Deleting Configs:**
|
||||
- Click **"Delete"** button
|
||||
- **Warning**: Cannot delete configs used by active schedules
|
||||
|
||||
### Running Scans
|
||||
|
||||
**Manual Scans:**
|
||||
1. Navigate to **Dashboard** or **Scans**
|
||||
2. Click **"Run Scan Now"**
|
||||
3. Select configuration file from dropdown
|
||||
4. Click **"Start Scan"**
|
||||
5. Scan executes in background (APScheduler)
|
||||
6. Monitor progress on **Scans** page
|
||||
|
||||
**Scheduled Scans:**
|
||||
1. Navigate to **Schedules** → **Create Schedule**
|
||||
2. Enter schedule name (e.g., "Daily production scan")
|
||||
3. Select config file
|
||||
4. Enter cron expression (e.g., `0 2 * * *` for 2 AM daily)
|
||||
5. Enable schedule
|
||||
6. Scans run automatically in background
|
||||
|
||||
**Cron Expression Examples:**
|
||||
- `0 2 * * *` - Daily at 2 AM
|
||||
- `0 */6 * * *` - Every 6 hours
|
||||
- `0 0 * * 0` - Weekly on Sunday at midnight
|
||||
- `0 0 1 * *` - Monthly on 1st at midnight
|
||||
|
||||
### Viewing Scan Results
|
||||
|
||||
**Scan List:**
|
||||
- Navigate to **Scans** page
|
||||
- View all historical scans with filters
|
||||
- Click scan ID to view details
|
||||
|
||||
**Scan Details:**
|
||||
- Full scan results organized by site
|
||||
- Discovered IPs, ports, services
|
||||
- SSL/TLS certificate information
|
||||
- TLS version support and cipher suites
|
||||
- Service version detection
|
||||
- Screenshots of web services
|
||||
- Download buttons (JSON, HTML, ZIP)
|
||||
|
||||
**Trend Analysis:**
|
||||
- Charts showing port count changes over time
|
||||
- Identify infrastructure drift
|
||||
- Track service version updates
|
||||
|
||||
---
|
||||
|
||||
## Volume Management
|
||||
|
||||
### Understanding Volumes
|
||||
@@ -280,10 +421,12 @@ SneakyScanner uses several mounted volumes for data persistence:
|
||||
|
||||
| Volume | Container Path | Purpose | Important? |
|
||||
|--------|----------------|---------|------------|
|
||||
| `./configs` | `/app/configs` | Scan configuration files (read-only) | Yes |
|
||||
| `./data` | `/app/data` | SQLite database | **Critical** |
|
||||
| `./output` | `/app/output` | Scan results (JSON, HTML, ZIP) | Yes |
|
||||
| `./logs` | `/app/logs` | Application logs | No |
|
||||
| `./configs` | `/app/configs` | Scan configuration files (managed via web UI) | Yes |
|
||||
| `./data` | `/app/data` | SQLite database (contains all scan history) | **Critical** |
|
||||
| `./output` | `/app/output` | Scan results (JSON, HTML, ZIP, screenshots) | Yes |
|
||||
| `./logs` | `/app/logs` | Application logs (rotating file handler) | No |
|
||||
|
||||
**Note**: As of Phase 4, the `./configs` volume is read-write to support the web-based config creator and editor. The web UI can now create, edit, and delete configuration files directly.
|
||||
|
||||
### Backing Up Data
|
||||
|
||||
@@ -305,7 +448,7 @@ tar -czf backups/$(date +%Y%m%d)/configs.tar.gz configs/
|
||||
|
||||
```bash
|
||||
# Stop application
|
||||
docker compose -f docker-compose-web.yml down
|
||||
docker compose -f docker-compose.yml down
|
||||
|
||||
# Restore database
|
||||
cp backups/YYYYMMDD/sneakyscanner.db data/
|
||||
@@ -314,35 +457,216 @@ cp backups/YYYYMMDD/sneakyscanner.db data/
|
||||
tar -xzf backups/YYYYMMDD/output.tar.gz
|
||||
|
||||
# Restart application
|
||||
docker compose -f docker-compose-web.yml up -d
|
||||
docker compose -f docker-compose.yml up -d
|
||||
```
|
||||
|
||||
### Cleaning Up Old Scan Results
|
||||
|
||||
**Option A: Using the Web UI (Recommended)**
|
||||
1. Navigate to **Scans** page
|
||||
2. Select scans you want to delete
|
||||
3. Click **"Delete"** button
|
||||
4. Confirm deletion (removes database records and all associated files)
|
||||
|
||||
**Option B: Manual Cleanup**
|
||||
```bash
|
||||
# Find old scan results (older than 30 days)
|
||||
find output/ -type f -name "scan_report_*.json" -mtime +30
|
||||
|
||||
# Delete old scan results
|
||||
find output/ -type f -name "scan_report_*" -mtime +30 -delete
|
||||
# Delete old scan results and screenshots
|
||||
find output/ -type f -mtime +30 -delete
|
||||
find output/ -type d -empty -delete
|
||||
|
||||
# Or use the API to delete scans from UI/API
|
||||
# Note: Manual deletion doesn't remove database records
|
||||
# Use the web UI or API for complete cleanup
|
||||
```
|
||||
|
||||
**Option C: Using the API**
|
||||
```bash
|
||||
# Delete a specific scan (removes DB records + files)
|
||||
curl -X DELETE http://localhost:5000/api/scans/{scan_id} \
|
||||
-b cookies.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Usage Examples
|
||||
|
||||
SneakyScanner provides a comprehensive REST API for automation and integration. All API endpoints require authentication via session cookies.
|
||||
|
||||
### Authentication
|
||||
|
||||
```bash
|
||||
# Login and save session cookie
|
||||
curl -X POST http://localhost:5000/api/auth/login \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"password": "YourPassword"}' \
|
||||
-c cookies.txt
|
||||
|
||||
# Logout
|
||||
curl -X POST http://localhost:5000/api/auth/logout \
|
||||
-b cookies.txt
|
||||
```
|
||||
|
||||
### Config Management (Phase 4)
|
||||
|
||||
```bash
|
||||
# List all configs
|
||||
curl http://localhost:5000/api/configs \
|
||||
-b cookies.txt
|
||||
|
||||
# Get specific config
|
||||
curl http://localhost:5000/api/configs/prod-network.yaml \
|
||||
-b cookies.txt
|
||||
|
||||
# Create new config
|
||||
curl -X POST http://localhost:5000/api/configs \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"filename": "test-network.yaml",
|
||||
"content": "title: Test Network\nsites:\n - name: Test\n cidr: 10.0.0.0/24"
|
||||
}' \
|
||||
-b cookies.txt
|
||||
|
||||
# Update config
|
||||
curl -X PUT http://localhost:5000/api/configs/test-network.yaml \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"content": "title: Updated Test Network\nsites:\n - name: Test Site\n cidr: 10.0.0.0/24"
|
||||
}' \
|
||||
-b cookies.txt
|
||||
|
||||
# Download config
|
||||
curl http://localhost:5000/api/configs/test-network.yaml/download \
|
||||
-b cookies.txt -o test-network.yaml
|
||||
|
||||
# Delete config
|
||||
curl -X DELETE http://localhost:5000/api/configs/test-network.yaml \
|
||||
-b cookies.txt
|
||||
```
|
||||
|
||||
### Scan Management
|
||||
|
||||
```bash
|
||||
# Trigger a scan
|
||||
curl -X POST http://localhost:5000/api/scans \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"config_file": "/app/configs/prod-network.yaml"}' \
|
||||
-b cookies.txt
|
||||
|
||||
# List all scans
|
||||
curl http://localhost:5000/api/scans?page=1&per_page=20 \
|
||||
-b cookies.txt
|
||||
|
||||
# Get scan details
|
||||
curl http://localhost:5000/api/scans/123 \
|
||||
-b cookies.txt
|
||||
|
||||
# Check scan status
|
||||
curl http://localhost:5000/api/scans/123/status \
|
||||
-b cookies.txt
|
||||
|
||||
# Delete scan
|
||||
curl -X DELETE http://localhost:5000/api/scans/123 \
|
||||
-b cookies.txt
|
||||
```
|
||||
|
||||
### Schedule Management
|
||||
|
||||
```bash
|
||||
# List schedules
|
||||
curl http://localhost:5000/api/schedules \
|
||||
-b cookies.txt
|
||||
|
||||
# Create schedule
|
||||
curl -X POST http://localhost:5000/api/schedules \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "Daily Production Scan",
|
||||
"config_file": "/app/configs/prod-network.yaml",
|
||||
"cron_expression": "0 2 * * *",
|
||||
"enabled": true
|
||||
}' \
|
||||
-b cookies.txt
|
||||
|
||||
# Update schedule
|
||||
curl -X PUT http://localhost:5000/api/schedules/1 \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"enabled": false}' \
|
||||
-b cookies.txt
|
||||
|
||||
# Manually trigger scheduled scan
|
||||
curl -X POST http://localhost:5000/api/schedules/1/trigger \
|
||||
-b cookies.txt
|
||||
|
||||
# Delete schedule
|
||||
curl -X DELETE http://localhost:5000/api/schedules/1 \
|
||||
-b cookies.txt
|
||||
```
|
||||
|
||||
### Settings Management
|
||||
|
||||
```bash
|
||||
# Get all settings (sanitized - passwords hidden)
|
||||
curl http://localhost:5000/api/settings \
|
||||
-b cookies.txt
|
||||
|
||||
# Update settings
|
||||
curl -X PUT http://localhost:5000/api/settings \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"retention_days": 90,
|
||||
"smtp_server": "smtp.gmail.com"
|
||||
}' \
|
||||
-b cookies.txt
|
||||
|
||||
# Test email configuration
|
||||
curl -X POST http://localhost:5000/api/settings/test-email \
|
||||
-b cookies.txt
|
||||
|
||||
# Health check (no auth required)
|
||||
curl http://localhost:5000/api/settings/health
|
||||
```
|
||||
|
||||
### Statistics
|
||||
|
||||
```bash
|
||||
# Get dashboard summary
|
||||
curl http://localhost:5000/api/stats/summary \
|
||||
-b cookies.txt
|
||||
|
||||
# Get trend data
|
||||
curl http://localhost:5000/api/stats/trends?days=30&metric=port_count \
|
||||
-b cookies.txt
|
||||
|
||||
# Get certificate expiry overview
|
||||
curl http://localhost:5000/api/stats/certificates \
|
||||
-b cookies.txt
|
||||
```
|
||||
|
||||
For complete API documentation, see `docs/API_REFERENCE.md`.
|
||||
|
||||
---
|
||||
|
||||
## Health Monitoring
|
||||
|
||||
### Health Check Endpoint
|
||||
|
||||
SneakyScanner includes a built-in health check endpoint:
|
||||
SneakyScanner includes a built-in health check endpoint used by Docker's healthcheck:
|
||||
|
||||
```bash
|
||||
# Check application health
|
||||
curl http://localhost:5000/api/settings/health
|
||||
|
||||
# Expected response:
|
||||
# Expected response (200 OK):
|
||||
# {"status": "healthy"}
|
||||
|
||||
# This endpoint is also used by Docker Compose healthcheck
|
||||
# Defined in docker-compose.yml:
|
||||
# - Interval: 60s (check every minute)
|
||||
# - Timeout: 10s
|
||||
# - Retries: 3
|
||||
# - Start period: 40s (grace period for app startup)
|
||||
```
|
||||
|
||||
### Docker Health Status
|
||||
@@ -359,7 +683,7 @@ docker inspect sneakyscanner-web | grep -A 10 Health
|
||||
|
||||
```bash
|
||||
# Watch for errors in logs
|
||||
docker compose -f docker-compose-web.yml logs -f | grep ERROR
|
||||
docker compose -f docker-compose.yml logs -f | grep ERROR
|
||||
|
||||
# Check application log file
|
||||
tail -f logs/sneakyscanner.log
|
||||
@@ -375,7 +699,7 @@ tail -f logs/sneakyscanner.log
|
||||
|
||||
```bash
|
||||
# Check logs for errors
|
||||
docker compose -f docker-compose-web.yml logs web
|
||||
docker compose -f docker-compose.yml logs web
|
||||
|
||||
# Common issues:
|
||||
# 1. Database not initialized - run init-db first
|
||||
@@ -399,7 +723,7 @@ sqlite3 data/sneakyscanner.db "SELECT 1;" 2>&1
|
||||
|
||||
# Remove corrupted database and reinitialize
|
||||
rm data/sneakyscanner.db
|
||||
docker compose -f docker-compose-web.yml run --rm init-db --password "YourPassword"
|
||||
docker compose -f docker-compose.yml run --rm init-db --password "YourPassword"
|
||||
```
|
||||
|
||||
### Scans Fail with "Permission Denied"
|
||||
@@ -415,7 +739,7 @@ docker inspect sneakyscanner-web | grep Privileged
|
||||
docker inspect sneakyscanner-web | grep NetworkMode
|
||||
# Should show: "NetworkMode": "host"
|
||||
|
||||
# If not, verify docker-compose-web.yml has:
|
||||
# If not, verify docker-compose.yml has:
|
||||
# privileged: true
|
||||
# network_mode: host
|
||||
```
|
||||
@@ -429,7 +753,7 @@ docker inspect sneakyscanner-web | grep NetworkMode
|
||||
docker ps | grep sneakyscanner-web
|
||||
|
||||
# Check if Flask is listening
|
||||
docker compose -f docker-compose-web.yml exec web netstat -tlnp | grep 5000
|
||||
docker compose -f docker-compose.yml exec web netstat -tlnp | grep 5000
|
||||
|
||||
# Check firewall rules
|
||||
sudo ufw status | grep 5000
|
||||
@@ -438,7 +762,7 @@ sudo ufw status | grep 5000
|
||||
curl http://localhost:5000/api/settings/health
|
||||
|
||||
# Check logs for binding errors
|
||||
docker compose -f docker-compose-web.yml logs web | grep -i bind
|
||||
docker compose -f docker-compose.yml logs web | grep -i bind
|
||||
```
|
||||
|
||||
### Background Scans Not Running
|
||||
@@ -447,13 +771,39 @@ docker compose -f docker-compose-web.yml logs web | grep -i bind
|
||||
|
||||
```bash
|
||||
# Check scheduler is initialized
|
||||
docker compose -f docker-compose-web.yml logs web | grep -i scheduler
|
||||
docker compose -f docker-compose.yml logs web | grep -i scheduler
|
||||
|
||||
# Check for job execution errors
|
||||
docker compose -f docker-compose-web.yml logs web | grep -i "execute_scan"
|
||||
docker compose -f docker-compose.yml logs web | grep -i "execute_scan"
|
||||
|
||||
# Verify APScheduler environment variables
|
||||
docker compose -f docker-compose-web.yml exec web env | grep SCHEDULER
|
||||
docker compose -f docker-compose.yml exec web env | grep SCHEDULER
|
||||
|
||||
# Check for scan job errors
|
||||
docker compose -f docker-compose.yml logs web | grep -E "(ERROR|Exception|Traceback)"
|
||||
|
||||
# Verify scanner executables are available
|
||||
docker compose -f docker-compose.yml exec web which masscan nmap
|
||||
```
|
||||
|
||||
### Config Files Not Appearing in Web UI
|
||||
|
||||
**Problem**: Manually created configs don't show up in web interface
|
||||
|
||||
```bash
|
||||
# Check file permissions (must be readable by web container)
|
||||
ls -la configs/
|
||||
|
||||
# Fix permissions if needed
|
||||
sudo chown -R 1000:1000 configs/
|
||||
chmod 644 configs/*.yaml
|
||||
|
||||
# Verify YAML syntax is valid
|
||||
docker compose -f docker-compose.yml exec web python3 -c \
|
||||
"import yaml; yaml.safe_load(open('/app/configs/your-config.yaml'))"
|
||||
|
||||
# Check web logs for parsing errors
|
||||
docker compose -f docker-compose.yml logs web | grep -i "config"
|
||||
```
|
||||
|
||||
### Health Check Failing
|
||||
@@ -462,7 +812,7 @@ docker compose -f docker-compose-web.yml exec web env | grep SCHEDULER
|
||||
|
||||
```bash
|
||||
# Run health check manually
|
||||
docker compose -f docker-compose-web.yml exec web \
|
||||
docker compose -f docker-compose.yml exec web \
|
||||
python3 -c "import urllib.request; print(urllib.request.urlopen('http://localhost:5000/api/settings/health').read())"
|
||||
|
||||
# Check if health endpoint exists
|
||||
@@ -480,16 +830,19 @@ curl -v http://localhost:5000/api/settings/health
|
||||
|
||||
### Production Deployment Checklist
|
||||
|
||||
- [ ] Changed `SECRET_KEY` to random value
|
||||
- [ ] Changed `SNEAKYSCANNER_ENCRYPTION_KEY` to random value
|
||||
- [ ] Set strong application password
|
||||
- [ ] Changed `SECRET_KEY` to random value (64+ character hex string)
|
||||
- [ ] Changed `SNEAKYSCANNER_ENCRYPTION_KEY` to random Fernet key
|
||||
- [ ] Set strong application password via init-db
|
||||
- [ ] Set `FLASK_ENV=production`
|
||||
- [ ] Set `FLASK_DEBUG=false`
|
||||
- [ ] Configured proper `CORS_ORIGINS` (not `*`)
|
||||
- [ ] Using HTTPS/TLS (reverse proxy recommended)
|
||||
- [ ] Restricted network access (firewall rules)
|
||||
- [ ] Regular backups configured
|
||||
- [ ] Regular backups configured (database + configs)
|
||||
- [ ] Log monitoring enabled
|
||||
- [ ] Scheduled scans configured with appropriate frequency
|
||||
- [ ] Alert rules configured (Phase 5 - coming soon)
|
||||
- [ ] Webhook/email notifications configured (Phase 5 - coming soon)
|
||||
|
||||
### Network Security
|
||||
|
||||
@@ -552,7 +905,7 @@ chmod 444 configs/*.yaml
|
||||
|
||||
```bash
|
||||
# 1. Stop the application
|
||||
docker compose -f docker-compose-web.yml down
|
||||
docker compose -f docker-compose.yml down
|
||||
|
||||
# 2. Backup database
|
||||
cp data/sneakyscanner.db data/sneakyscanner.db.backup
|
||||
@@ -561,16 +914,16 @@ cp data/sneakyscanner.db data/sneakyscanner.db.backup
|
||||
git pull origin master
|
||||
|
||||
# 4. Rebuild Docker image
|
||||
docker compose -f docker-compose-web.yml build
|
||||
docker compose -f docker-compose.yml build
|
||||
|
||||
# 5. Run database migrations
|
||||
docker compose -f docker-compose-web.yml run --rm web alembic upgrade head
|
||||
docker compose -f docker-compose.yml run --rm web alembic upgrade head
|
||||
|
||||
# 6. Start application
|
||||
docker compose -f docker-compose-web.yml up -d
|
||||
docker compose -f docker-compose.yml up -d
|
||||
|
||||
# 7. Verify upgrade
|
||||
docker compose -f docker-compose-web.yml logs -f
|
||||
docker compose -f docker-compose.yml logs -f
|
||||
curl http://localhost:5000/api/settings/health
|
||||
```
|
||||
|
||||
@@ -580,7 +933,7 @@ If upgrade fails:
|
||||
|
||||
```bash
|
||||
# Stop new version
|
||||
docker compose -f docker-compose-web.yml down
|
||||
docker compose -f docker-compose.yml down
|
||||
|
||||
# Restore database backup
|
||||
cp data/sneakyscanner.db.backup data/sneakyscanner.db
|
||||
@@ -589,8 +942,8 @@ cp data/sneakyscanner.db.backup data/sneakyscanner.db
|
||||
git checkout <previous-version-tag>
|
||||
|
||||
# Rebuild and start
|
||||
docker compose -f docker-compose-web.yml build
|
||||
docker compose -f docker-compose-web.yml up -d
|
||||
docker compose -f docker-compose.yml build
|
||||
docker compose -f docker-compose.yml up -d
|
||||
```
|
||||
|
||||
---
|
||||
@@ -607,7 +960,7 @@ BACKUP_DIR="backups/$(date +%Y%m%d_%H%M%S)"
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
# Stop application for consistent backup
|
||||
docker compose -f docker-compose-web.yml stop web
|
||||
docker compose -f docker-compose.yml stop web
|
||||
|
||||
# Backup database
|
||||
cp data/sneakyscanner.db "$BACKUP_DIR/"
|
||||
@@ -619,7 +972,7 @@ find output/ -type f -mtime -30 -exec cp --parents {} "$BACKUP_DIR/" \;
|
||||
cp -r configs/ "$BACKUP_DIR/"
|
||||
|
||||
# Restart application
|
||||
docker compose -f docker-compose-web.yml start web
|
||||
docker compose -f docker-compose.yml start web
|
||||
|
||||
echo "Backup complete: $BACKUP_DIR"
|
||||
```
|
||||
@@ -639,7 +992,7 @@ crontab -e
|
||||
|
||||
```bash
|
||||
# Stop application
|
||||
docker compose -f docker-compose-web.yml down
|
||||
docker compose -f docker-compose.yml down
|
||||
|
||||
# Restore files
|
||||
cp backups/YYYYMMDD_HHMMSS/sneakyscanner.db data/
|
||||
@@ -647,7 +1000,7 @@ cp -r backups/YYYYMMDD_HHMMSS/configs/* configs/
|
||||
cp -r backups/YYYYMMDD_HHMMSS/output/* output/
|
||||
|
||||
# Start application
|
||||
docker compose -f docker-compose-web.yml up -d
|
||||
docker compose -f docker-compose.yml up -d
|
||||
```
|
||||
|
||||
---
|
||||
@@ -655,12 +1008,41 @@ docker compose -f docker-compose-web.yml up -d
|
||||
## Support and Further Reading
|
||||
|
||||
- **Project README**: `README.md` - General project information
|
||||
- **API Documentation**: `docs/ai/API_REFERENCE.md` - REST API reference
|
||||
- **Developer Guide**: `docs/ai/DEVELOPMENT.md` - Development setup and architecture
|
||||
- **Phase 2 Documentation**: `docs/ai/PHASE2.md` - Implementation details
|
||||
- **API Documentation**: `docs/API_REFERENCE.md` - Complete REST API reference
|
||||
- **Roadmap**: `docs/ROADMAP.md` - Project roadmap, feature plans, and architecture
|
||||
- **Issue Tracker**: File bugs and feature requests on GitHub
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-14
|
||||
**Version**: Phase 2 - Web Application Complete
|
||||
## What's New
|
||||
|
||||
### Phase 4 (2025-11-17) - Config Creator ✅
|
||||
- **CIDR-based Config Creator**: Web UI for generating scan configs from CIDR ranges
|
||||
- **YAML Editor**: Built-in editor with syntax highlighting (CodeMirror)
|
||||
- **Config Management UI**: List, view, edit, download, and delete configs via web interface
|
||||
- **Config Upload**: Direct YAML file upload for advanced users
|
||||
- **REST API**: 7 new config management endpoints
|
||||
- **Schedule Protection**: Prevents deleting configs used by active schedules
|
||||
|
||||
### Phase 3 (2025-11-14) - Dashboard & Scheduling ✅
|
||||
- **Dashboard**: Summary stats, recent scans, trend charts
|
||||
- **Scheduled Scans**: Cron-based scheduling with web UI management
|
||||
- **Scan History**: Detailed scan results with full data display
|
||||
- **Chart.js Integration**: Port count trends over time
|
||||
|
||||
### Phase 2 (2025-11-14) - Web Application Core ✅
|
||||
- **REST API**: Complete API for scan management
|
||||
- **Background Jobs**: APScheduler-based async execution
|
||||
- **Authentication**: Session-based login system
|
||||
- **Database Integration**: SQLite with SQLAlchemy ORM
|
||||
|
||||
### Coming Soon: Phase 5 - Email, Webhooks & Comparisons
|
||||
- Email notifications for infrastructure changes
|
||||
- Webhook integrations (Slack, PagerDuty, custom)
|
||||
- Scan comparison reports
|
||||
- Alert rule configuration
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-17
|
||||
**Version**: Phase 4 - Config Creator Complete
|
||||
|
||||
483
docs/ROADMAP.md
483
docs/ROADMAP.md
@@ -1,5 +1,9 @@
|
||||
# SneakyScanner Roadmap
|
||||
|
||||
## Vision & Goals
|
||||
|
||||
SneakyScanner is a comprehensive **Flask web application** for infrastructure monitoring and security auditing. The primary interface is the web GUI, with a CLI API client planned for scripting and automation needs.
|
||||
|
||||
**Status:** Phase 4 Complete ✅ | Phase 5 Next Up
|
||||
|
||||
## Progress Overview
|
||||
@@ -11,37 +15,14 @@
|
||||
- Dashboard, scan history, scheduled scans, trend charts
|
||||
- ✅ **Phase 4: Config Creator** - Complete (2025-11-17)
|
||||
- CIDR-based config creation, YAML editor, config management UI
|
||||
- 📋 **Phase 5: Email & Comparisons** - Next Up
|
||||
- 📋 **Phase 5: Email, Webhooks & Comparisons** - Next Up
|
||||
- Email notifications, alert rules, scan comparison
|
||||
- 📋 **Phase 6: CLI as API Client** - Planned
|
||||
- CLI for scripting and automation via API
|
||||
- 📋 **Phase 7: Advanced Features** - Future
|
||||
- CVE integration, timeline view, PDF export, enhanced reports
|
||||
|
||||
## Recent Bug Fixes
|
||||
|
||||
### 2025-11-17: Chart.js Infinite Canvas Growth Fix
|
||||
**Issue:** Scan detail page (`scan_detail.html`) was experiencing infinite scrolling and page lock-up due to Chart.js canvas growing infinitely (height reaching 22302px+).
|
||||
|
||||
**Root Causes:**
|
||||
1. Duplicate initialization - `loadScan()` was being called twice on page load
|
||||
2. Multiple Chart.js instances created on the same canvas without destroying previous ones
|
||||
3. Canvas element without fixed-height container caused infinite resize loop with `responsive: true` and `maintainAspectRatio: false`
|
||||
|
||||
**Fixes Applied:**
|
||||
1. **Consolidated initialization** (`scan_detail.html:172-175`) - Moved `findPreviousScan()` and `loadHistoricalChart()` into `DOMContentLoaded` event listener, removed duplicate call
|
||||
2. **Chart instance tracking** (`scan_detail.html:169`) - Added `let historyChart = null;` to store chart reference
|
||||
3. **Destroy old charts** (`scan_detail.html:501-504`) - Added `historyChart.destroy()` before creating new chart instance
|
||||
4. **Fixed-height container** (`scan_detail.html:136-138`) - Wrapped canvas in `<div style="position: relative; height: 300px;">` to prevent infinite resize loop
|
||||
|
||||
**Files Modified:**
|
||||
- `web/templates/scan_detail.html`
|
||||
|
||||
**Status:** ✅ Fixed and tested
|
||||
|
||||
## Vision & Goals
|
||||
|
||||
SneakyScanner is a comprehensive **Flask web application** for infrastructure monitoring and security auditing. The primary interface is the web GUI, with a CLI API client planned for scripting and automation needs.
|
||||
|
||||
**Core Features:**
|
||||
- **Centralized dashboard** for viewing scan history and trends
|
||||
@@ -439,33 +420,360 @@ All API endpoints return JSON and follow RESTful conventions.
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Email & Comparisons
|
||||
### Phase 5: Email, Webhooks & Comparisons
|
||||
**Status:** Next Up
|
||||
**Priority:** MEDIUM
|
||||
|
||||
**Goals:**
|
||||
- Implement email notification system
|
||||
- Create scan comparison reports
|
||||
- Add alert rule configuration
|
||||
- Implement email notification system for infrastructure misconfigurations
|
||||
- Implement webhook notification system for real-time alerting
|
||||
- Create scan comparison reports to detect drift
|
||||
- Add alert rule configuration for unexpected exposure detection
|
||||
|
||||
**Core Use Case:**
|
||||
Monitor infrastructure for misconfigurations that expose unexpected ports/services to the world. When a scan detects an open port that wasn't defined in the YAML config's `expected_ports` list, trigger immediate notifications via email and/or webhooks.
|
||||
|
||||
**Planned Features:**
|
||||
1. **Email Notifications:**
|
||||
- SMTP integration with configurable settings
|
||||
- Alert email templates (Jinja2 HTML)
|
||||
|
||||
#### 1. Alert Rule Engine
|
||||
**Purpose:** Automatically detect and classify infrastructure anomalies after each scan.
|
||||
|
||||
**Alert Types:**
|
||||
- `unexpected_port` - Port open but not in config's `expected_ports` list
|
||||
- `unexpected_service` - Service detected that doesn't match expected service name
|
||||
- `cert_expiry` - SSL/TLS certificate expiring soon (configurable threshold)
|
||||
- `ping_failed` - Expected host not responding to ping
|
||||
- `service_down` - Previously detected service no longer responding
|
||||
- `service_change` - Service version/product changed between scans
|
||||
- `weak_tls` - TLS 1.0/1.1 detected or weak cipher suites
|
||||
- `new_host` - New IP address responding in CIDR range
|
||||
- `host_disappeared` - Previously seen IP no longer responding
|
||||
|
||||
**Alert Severity Levels:**
|
||||
- `critical` - Unexpected internet-facing service (ports 80/443/22/3389/etc.)
|
||||
- `warning` - Minor configuration drift or upcoming cert expiry
|
||||
- `info` - Informational alerts (new host discovered, service version change)
|
||||
|
||||
**Alert Rule Configuration:**
|
||||
```yaml
|
||||
# Example alert rule configuration (stored in DB)
|
||||
alert_rules:
|
||||
- id: 1
|
||||
rule_type: unexpected_port
|
||||
enabled: true
|
||||
severity: critical
|
||||
email_enabled: true
|
||||
webhook_enabled: true
|
||||
filter_conditions:
|
||||
ports: [22, 80, 443, 3389, 3306, 5432, 27017] # High-risk ports
|
||||
|
||||
- id: 2
|
||||
rule_type: cert_expiry
|
||||
enabled: true
|
||||
severity: warning
|
||||
threshold: 30 # Days before expiry
|
||||
email_enabled: true
|
||||
webhook_enabled: false
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- Evaluate alert rules after each scan completes
|
||||
- Compare current scan results to expected configuration
|
||||
- Generate alerts and store in `alerts` table
|
||||
- Trigger notifications based on rule configuration
|
||||
- Alert deduplication (don't spam for same issue)
|
||||
|
||||
#### 2. Email Notifications
|
||||
**Purpose:** Send detailed email alerts when infrastructure misconfigurations are detected.
|
||||
|
||||
**SMTP Configuration (via Settings API):**
|
||||
```json
|
||||
{
|
||||
"smtp_server": "smtp.gmail.com",
|
||||
"smtp_port": 587,
|
||||
"smtp_use_tls": true,
|
||||
"smtp_username": "alerts@example.com",
|
||||
"smtp_password": "encrypted_password",
|
||||
"smtp_from_email": "SneakyScanner <alerts@example.com>",
|
||||
"smtp_to_emails": ["admin@example.com", "security@example.com"],
|
||||
"email_alerts_enabled": true
|
||||
}
|
||||
```
|
||||
|
||||
**Email Template (Jinja2 HTML):**
|
||||
```html
|
||||
Subject: [SneakyScanner Alert] Unexpected Port Detected - {{ ip_address }}:{{ port }}
|
||||
|
||||
Body:
|
||||
===============================================
|
||||
SneakyScanner Security Alert
|
||||
===============================================
|
||||
|
||||
Alert Type: {{ alert_type }}
|
||||
Severity: {{ severity }}
|
||||
Scan ID: {{ scan_id }}
|
||||
Timestamp: {{ timestamp }}
|
||||
|
||||
Issue Detected:
|
||||
{{ message }}
|
||||
|
||||
Details:
|
||||
- IP Address: {{ ip_address }}
|
||||
- Port: {{ port }}/{{ protocol }}
|
||||
- Service: {{ service_name }} ({{ product }} {{ version }})
|
||||
- Expected: No (not in expected_ports list)
|
||||
|
||||
Recommended Actions:
|
||||
1. Verify if this service should be exposed
|
||||
2. Check firewall rules for {{ ip_address }}
|
||||
3. Review service configuration
|
||||
4. Update scan config if this is intentional
|
||||
|
||||
View Full Scan Results:
|
||||
{{ web_url }}/scans/{{ scan_id }}
|
||||
|
||||
===============================================
|
||||
```
|
||||
|
||||
**Email Features:**
|
||||
- HTML email with styled alert box (Bootstrap-based)
|
||||
- Plain-text fallback for compatibility
|
||||
- Alert summary with actionable recommendations
|
||||
- Direct link to scan detail page
|
||||
- Configurable recipients (multiple emails)
|
||||
- Test email functionality in Settings UI
|
||||
- Email delivery tracking (email_sent, email_sent_at in alerts table)
|
||||
- Rate limiting to prevent email flood
|
||||
|
||||
**Email API Endpoints:**
|
||||
- `POST /api/settings/email` - Configure SMTP settings
|
||||
- `POST /api/settings/email/test` - Send test email
|
||||
- `GET /api/alerts?email_sent=true` - Get alerts with email status
|
||||
|
||||
#### 3. Webhook Notifications
|
||||
**Purpose:** Real-time HTTP POST notifications for integration with external systems (Slack, PagerDuty, custom dashboards, SIEM tools).
|
||||
|
||||
**Webhook Configuration (via Settings API):**
|
||||
```json
|
||||
{
|
||||
"webhook_enabled": true,
|
||||
"webhook_urls": [
|
||||
{
|
||||
"id": 1,
|
||||
"name": "Slack Security Channel",
|
||||
"url": "https://hooks.slack.com/services/XXX/YYY/ZZZ",
|
||||
"enabled": true,
|
||||
"auth_type": "none",
|
||||
"custom_headers": {},
|
||||
"alert_types": ["unexpected_port", "unexpected_service", "weak_tls"],
|
||||
"severity_filter": ["critical", "warning"]
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"name": "PagerDuty",
|
||||
"url": "https://events.pagerduty.com/v2/enqueue",
|
||||
"enabled": true,
|
||||
"auth_type": "bearer",
|
||||
"auth_token": "encrypted_token",
|
||||
"custom_headers": {
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"alert_types": ["unexpected_port"],
|
||||
"severity_filter": ["critical"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Webhook Payload Format (JSON):**
|
||||
```json
|
||||
{
|
||||
"event_type": "scan_alert",
|
||||
"alert_id": 42,
|
||||
"alert_type": "unexpected_port",
|
||||
"severity": "critical",
|
||||
"timestamp": "2025-11-17T14:23:45Z",
|
||||
"scan": {
|
||||
"scan_id": 123,
|
||||
"title": "Production Network Scan",
|
||||
"timestamp": "2025-11-17T14:15:00Z",
|
||||
"config_file": "prod_config.yaml",
|
||||
"triggered_by": "scheduled"
|
||||
},
|
||||
"alert_details": {
|
||||
"message": "Unexpected port 3306 (MySQL) exposed on 192.168.1.100",
|
||||
"ip_address": "192.168.1.100",
|
||||
"port": 3306,
|
||||
"protocol": "tcp",
|
||||
"state": "open",
|
||||
"service": {
|
||||
"name": "mysql",
|
||||
"product": "MySQL",
|
||||
"version": "8.0.32"
|
||||
},
|
||||
"expected": false,
|
||||
"site_name": "Production Servers"
|
||||
},
|
||||
"recommended_actions": [
|
||||
"Verify if MySQL should be exposed externally",
|
||||
"Check firewall rules for 192.168.1.100",
|
||||
"Review MySQL bind-address configuration"
|
||||
],
|
||||
"web_url": "https://sneakyscanner.local/scans/123"
|
||||
}
|
||||
```
|
||||
|
||||
**Webhook Features:**
|
||||
- Multiple webhook URLs with independent configuration
|
||||
- Per-webhook filtering by alert type and severity
|
||||
- Custom headers support (for API keys, auth tokens)
|
||||
- Authentication methods:
|
||||
- `none` - No authentication
|
||||
- `bearer` - Bearer token in Authorization header
|
||||
- `basic` - Basic authentication
|
||||
- `custom` - Custom header-based auth
|
||||
- Retry logic with exponential backoff (3 attempts)
|
||||
- Webhook delivery tracking (webhook_sent, webhook_sent_at, webhook_response_code)
|
||||
- Test webhook functionality in Settings UI
|
||||
- Timeout configuration (default 10 seconds)
|
||||
- Webhook delivery history and logs
|
||||
|
||||
**Webhook API Endpoints:**
|
||||
- `POST /api/webhooks` - Create webhook configuration
|
||||
- `GET /api/webhooks` - List all webhooks
|
||||
- `PUT /api/webhooks/{id}` - Update webhook configuration
|
||||
- `DELETE /api/webhooks/{id}` - Delete webhook
|
||||
- `POST /api/webhooks/{id}/test` - Send test webhook
|
||||
- `GET /api/webhooks/{id}/history` - Get delivery history
|
||||
|
||||
**Slack Integration Example:**
|
||||
Transform webhook payload to Slack message format:
|
||||
```json
|
||||
{
|
||||
"text": "SneakyScanner Alert: Unexpected Port Detected",
|
||||
"attachments": [
|
||||
{
|
||||
"color": "danger",
|
||||
"fields": [
|
||||
{"title": "IP Address", "value": "192.168.1.100", "short": true},
|
||||
{"title": "Port", "value": "3306/tcp", "short": true},
|
||||
{"title": "Service", "value": "MySQL 8.0.32", "short": true},
|
||||
{"title": "Severity", "value": "CRITICAL", "short": true}
|
||||
],
|
||||
"footer": "SneakyScanner",
|
||||
"ts": 1700234625
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### 4. Alert Management UI
|
||||
**Purpose:** Web interface for configuring alert rules, viewing alert history, and managing notifications.
|
||||
|
||||
**Pages:**
|
||||
- `/alerts` - Alert history with filtering and search
|
||||
- `/alerts/rules` - Alert rule configuration
|
||||
- `/settings/email` - Email notification settings
|
||||
- `/settings/webhooks` - Webhook configuration
|
||||
|
||||
**Alert History Features:**
|
||||
- Filter by alert type, severity, date range, IP address
|
||||
- Search by message content
|
||||
- Bulk acknowledge/dismiss alerts
|
||||
- Export alerts to CSV
|
||||
- Alert detail modal with full context
|
||||
|
||||
**Alert Rule UI Features:**
|
||||
- Enable/disable rules individually
|
||||
- Configure severity levels
|
||||
- Set thresholds (e.g., cert expiry days)
|
||||
- Toggle email/webhook per rule
|
||||
- Test rules against recent scans
|
||||
|
||||
#### 5. Scan Comparison
|
||||
**Purpose:** Detect infrastructure drift by comparing two scans and highlighting changes.
|
||||
|
||||
**Comparison API:**
|
||||
- `GET /api/scans/{id1}/compare/{id2}` - Compare two scans
|
||||
|
||||
**Comparison Response:**
|
||||
```json
|
||||
{
|
||||
"scan1": {"id": 100, "timestamp": "2025-11-10T10:00:00Z"},
|
||||
"scan2": {"id": 123, "timestamp": "2025-11-17T14:15:00Z"},
|
||||
"summary": {
|
||||
"new_ports": 2,
|
||||
"removed_ports": 0,
|
||||
"service_changes": 1,
|
||||
"cert_changes": 0,
|
||||
"new_hosts": 1,
|
||||
"removed_hosts": 0
|
||||
},
|
||||
"differences": {
|
||||
"new_ports": [
|
||||
{"ip": "192.168.1.100", "port": 3306, "service": "mysql"}
|
||||
],
|
||||
"removed_ports": [],
|
||||
"service_changes": [
|
||||
{
|
||||
"ip": "192.168.1.50",
|
||||
"port": 22,
|
||||
"old": "OpenSSH 8.2",
|
||||
"new": "OpenSSH 8.9"
|
||||
}
|
||||
],
|
||||
"new_hosts": [
|
||||
{"ip": "192.168.1.200", "site": "Production Servers"}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Comparison UI Features:**
|
||||
- Side-by-side comparison view
|
||||
- Color-coded differences (green=new, red=removed, yellow=changed)
|
||||
- Filter by change type
|
||||
- Export comparison report to PDF/HTML
|
||||
- "Compare with previous scan" button on scan detail page
|
||||
|
||||
---
|
||||
|
||||
**Phase 5 Implementation Plan:**
|
||||
|
||||
1. **Week 1: Alert Rule Engine**
|
||||
- Implement alert evaluation logic after scan completion
|
||||
- Create `alerts` table population
|
||||
- Add alert rule CRUD API
|
||||
- Unit tests for alert detection
|
||||
|
||||
2. **Week 2: Email Notifications**
|
||||
- SMTP integration with Flask-Mail
|
||||
- Jinja2 email templates (HTML + plain text)
|
||||
- Settings API for email configuration
|
||||
- Test email functionality
|
||||
- Email triggers for critical events
|
||||
- Email delivery tracking
|
||||
|
||||
2. **Alert Rule Engine:**
|
||||
- Alert types: unexpected ports, cert expiry, service changes, host down, weak TLS
|
||||
- Alert rule creation and management UI
|
||||
- Automatic evaluation after each scan
|
||||
- Alert history with severity filtering
|
||||
3. **Week 3: Webhook System**
|
||||
- Webhook configuration API
|
||||
- HTTP POST delivery with retry logic
|
||||
- Webhook template system for different platforms
|
||||
- Test webhook functionality
|
||||
- Delivery tracking and logging
|
||||
|
||||
3. **Scan Comparison:**
|
||||
- Compare two scans API endpoint
|
||||
- Diff detection (new/removed ports, service changes, cert changes)
|
||||
- Visual comparison UI with highlighting
|
||||
- "Compare" button on scan list
|
||||
4. **Week 4: Alert UI & Scan Comparison**
|
||||
- Alert history page with filtering
|
||||
- Alert rule management UI
|
||||
- Email/webhook settings pages
|
||||
- Scan comparison API and UI
|
||||
- Integration testing
|
||||
|
||||
**Success Criteria:**
|
||||
- Alert triggered within 30 seconds of scan completion
|
||||
- Email delivered successfully to configured recipients
|
||||
- Webhook POST delivered with retry on failure
|
||||
- Scan comparison highlights all infrastructure changes
|
||||
- Zero false positives for expected ports/services
|
||||
- Alert deduplication prevents notification spam
|
||||
|
||||
---
|
||||
|
||||
@@ -560,34 +868,6 @@ All API endpoints return JSON and follow RESTful conventions.
|
||||
- Gunicorn WSGI server
|
||||
- Optional Nginx reverse proxy
|
||||
|
||||
## Prioritized Feature List
|
||||
|
||||
### Completed ✅ (Phases 1-4)
|
||||
1. **Database foundation** (SQLite3 + SQLAlchemy)
|
||||
2. **Flask web app core** (REST API, authentication)
|
||||
3. **Dashboard with scan history** (list, detail, delete)
|
||||
4. **Trend charts** (Chart.js - port counts over time)
|
||||
5. **Scheduled scans** (APScheduler + cron expressions)
|
||||
6. **Config creator** (CIDR-based, YAML editor)
|
||||
|
||||
### Next Up (Phase 5)
|
||||
7. **Email notifications** (SMTP integration)
|
||||
8. **Alert rules** (cert expiry, unexpected ports, etc.)
|
||||
9. **Scan comparison reports** (diff view)
|
||||
|
||||
### Planned (Phase 6-7)
|
||||
10. **CLI as API client** (token auth, scripting)
|
||||
11. **Sortable/filterable tables** (DataTables.js)
|
||||
12. **PDF export** (WeasyPrint)
|
||||
13. **Vulnerability detection** (CVE integration)
|
||||
14. **Timeline view** (visual scan history)
|
||||
|
||||
### Future/Deferred
|
||||
15. **Multi-user support** (if requirements change)
|
||||
16. **Slack/webhook integrations**
|
||||
17. **Prometheus metrics**
|
||||
18. **Advanced charts** (heatmaps, forecasts)
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Iteration Cycle
|
||||
@@ -617,69 +897,6 @@ All API endpoints return JSON and follow RESTful conventions.
|
||||
- **CLAUDE.md** - Developer documentation (architecture, code references)
|
||||
- **API.md** - API documentation (OpenAPI/Swagger in Phase 4)
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Phase 1 Success ✅ ACHIEVED
|
||||
- [x] Database creates successfully with all 11 tables
|
||||
- [x] Settings can be stored/retrieved with encryption
|
||||
- [x] Flask app starts without errors
|
||||
- [x] API blueprints load correctly
|
||||
- [x] All Python modules have valid syntax
|
||||
- [x] Docker deployment configured
|
||||
|
||||
### Phase 2 Success ✅ ACHIEVED
|
||||
- [x] Database stores scan results correctly
|
||||
- [x] REST API functional with all endpoints
|
||||
- [x] Background scans execute asynchronously
|
||||
- [x] Authentication protects all routes
|
||||
- [x] Web UI is intuitive and responsive
|
||||
- [x] 100 tests passing with comprehensive coverage
|
||||
- [x] Docker deployment production-ready
|
||||
|
||||
### Phase 3 Success ✅ ACHIEVED
|
||||
- [x] Dashboard displays scans and trends with charts
|
||||
- [x] Scheduled scans execute automatically
|
||||
- [x] Historical trend charts show scan history
|
||||
- [x] Real-time progress updates for running scans
|
||||
|
||||
### Phase 4 Success ✅ ACHIEVED
|
||||
- [x] Users can create configs from CIDR ranges via web UI
|
||||
- [x] YAML editor with syntax highlighting works correctly
|
||||
- [x] Config management UI provides list/view/edit/download/delete operations
|
||||
- [x] Direct YAML upload works for advanced users
|
||||
- [x] Configs immediately usable in scan triggers and schedules
|
||||
- [x] Delete protection prevents removal of configs used by schedules
|
||||
- [x] All tests passing (25+ unit and integration tests)
|
||||
|
||||
### Phase 5 Success (Email & Comparisons)
|
||||
- [ ] Email notifications sent for critical alerts
|
||||
- [ ] Comparison reports show meaningful diffs
|
||||
- [ ] Settings UI allows SMTP configuration without editing files
|
||||
|
||||
### Phase 6 Success (CLI as API Client)
|
||||
- [ ] CLI can trigger scans via API
|
||||
- [ ] API tokens work for authentication
|
||||
- [ ] Standalone CLI mode still functional
|
||||
|
||||
### Phase 7 Success (Advanced Features)
|
||||
- [ ] CVE integration provides actionable vulnerability data
|
||||
- [ ] Timeline view helps track infrastructure changes
|
||||
- [ ] PDF exports are shareable and professional
|
||||
|
||||
## Open Questions
|
||||
|
||||
### Technical Decisions
|
||||
- **Flask vs. FastAPI?** - Sticking with Flask for simplicity, but FastAPI offers async and auto-docs
|
||||
- **APScheduler vs. Celery?** - APScheduler for simplicity (no Redis/RabbitMQ needed), but Celery scales better
|
||||
- **Bootstrap vs. Tailwind?** - Bootstrap for speed (pre-built components), Tailwind for customization
|
||||
- **Chart.js vs. Plotly?** - Chart.js for lightweight, Plotly for interactive (decide in Phase 3)
|
||||
|
||||
### Product Questions
|
||||
- **Should we support multiple configs per schedule?** - Start with 1:1, add later if needed
|
||||
- **How many scans to keep in DB?** - Add retention setting (default: keep all)
|
||||
- **Support for multi-tenancy?** - Not in scope (single-user), but database schema allows future expansion
|
||||
- **Mobile app?** - Out of scope, but responsive web UI covers basics
|
||||
|
||||
## Resources & References
|
||||
|
||||
### Documentation
|
||||
|
||||
Reference in New Issue
Block a user