sneakygeek/SneakyScan
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

restructure of dirs, huge docs update

Browse Source
This commit is contained in:
Phillip Tarrant
2025-11-17 16:29:14 -06:00
parent 456e052389
commit cd840cb8ca
87 changed files with 2827 additions and 1094 deletions
Download Patch File Download Diff File Expand all files Collapse all files

862
README.md
View File

@@ -1,28 +1,26 @@
# SneakyScanner
A comprehensive network scanning and infrastructure monitoring platform with both CLI and web interfaces. SneakyScanner uses masscan for fast port discovery, nmap for service detection, sslyze for SSL/TLS analysis, and Playwright for webpage screenshots to perform comprehensive infrastructure audits.
A comprehensive network scanning and infrastructure monitoring platform with web interface and CLI scanner. SneakyScanner uses masscan for fast port discovery, nmap for service detection, sslyze for SSL/TLS analysis, and Playwright for webpage screenshots to perform comprehensive infrastructure audits.
**Features:**
- 🔍 **CLI Scanner** - Standalone scanning tool with YAML-based configuration
- 🌐 **Web Application** - Flask-based web UI with REST API for scan management
- 📊 **Database Storage** - SQLite database for scan history and trend analysis
- ⏱️ **Background Jobs** - Asynchronous scan execution with APScheduler
- 🔐 **Authentication** - Secure session-based authentication system
- 📈 **Historical Data** - Track infrastructure changes over time
**Primary Interface**: Web Application (Flask-based GUI)
**Alternative**: Standalone CLI Scanner (for testing and CI/CD)
## Table of Contents
---
1. [Quick Start](#quick-start)
- [Web Application (Recommended)](#web-application-recommended)
- [CLI Scanner (Standalone)](#cli-scanner-standalone)
2. [Features](#features)
3. [Web Application](#web-application)
4. [CLI Scanner](#cli-scanner)
5. [Configuration](#configuration)
6. [Output Formats](#output-formats)
7. [API Documentation](#api-documentation)
8. [Deployment](#deployment)
9. [Development](#development)
## Key Features
- 🌐 **Web Dashboard** - Modern web UI for scan management, scheduling, and historical analysis
- 📊 **Database Storage** - SQLite-based scan history with trend analysis and comparison
- ⏰ **Scheduled Scans** - Cron-based automated scanning with APScheduler
- 🔧 **Config Creator** - CIDR-to-YAML configuration builder for quick setup
- 🔍 **Network Discovery** - Fast port scanning with masscan (all 65535 ports, TCP/UDP)
- 🎯 **Service Detection** - Nmap-based service enumeration with version detection
- 🔒 **SSL/TLS Analysis** - Certificate extraction, TLS version testing, cipher suite analysis
- 📸 **Screenshot Capture** - Automated webpage screenshots for all discovered web services
- 📈 **Drift Detection** - Expected vs. actual infrastructure comparison
- 📋 **Multi-Format Reports** - JSON, HTML, and ZIP archives with visual reports
- 🔐 **Authentication** - Session-based login for single-user deployments
- 🔔 **Alerts** *(Phase 5 - Coming Soon)* - Email and webhook notifications for misconfigurations
---
@@ -30,764 +28,148 @@ A comprehensive network scanning and infrastructure monitoring platform with bot
### Web Application (Recommended)
The web application provides a complete interface for managing scans, viewing history, and analyzing results.
1. **Configure environment:**
```bash
# Copy example environment file
# 1. Clone repository
git clone <repository-url>
cd SneakyScan
# 2. Configure environment
cp .env.example .env
# Edit .env and set SECRET_KEY and SNEAKYSCANNER_ENCRYPTION_KEY
# Generate secure keys (Linux/Mac)
export SECRET_KEY=$(python3 -c 'import secrets; print(secrets.token_hex(32))')
export ENCRYPTION_KEY=$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')
# 3. Build and start
docker compose build
docker compose up -d
# Update .env file with generated keys
sed -i "s/your-secret-key-here/$SECRET_KEY/" .env
sed -i "s/your-encryption-key-here/$ENCRYPTION_KEY/" .env
# 4. Initialize database
docker compose run --rm init-db --password "YourSecurePassword"
# 5. Access web interface
# Open http://localhost:5000
```
2. **Start the web application:**
```bash
docker-compose -f docker-compose-web.yml up -d
```
3. **Access the web interface:**
- Open http://localhost:5000 in your browser
- Default password: `admin` (change immediately after first login)
4. **Trigger your first scan:**
- Click "Run Scan Now" on the dashboard
- Or use the API:
```bash
curl -X POST http://localhost:5000/api/scans \
-H "Content-Type: application/json" \
-d '{"config_file":"/app/configs/example-site.yaml"}' \
-b cookies.txt
```
See [Deployment Guide](docs/ai/DEPLOYMENT.md) for detailed setup instructions.
**See [Deployment Guide](docs/DEPLOYMENT.md) for detailed setup instructions.**
### CLI Scanner (Standalone)
For quick one-off scans or scripting, use the standalone CLI scanner:
For quick one-off scans without the web interface:
```bash
# Build the image
docker-compose build
# Build and run
docker compose -f docker-compose-standalone.yml build
docker compose -f docker-compose-standalone.yml up
# Run a scan
docker-compose up
# Or run directly
docker run --rm --privileged --network host \
-v $(pwd)/configs:/app/configs:ro \
-v $(pwd)/output:/app/output \
sneakyscanner /app/configs/example-site.yaml
# Results saved to ./output/
```
Results are saved to the `output/` directory as JSON, HTML, and ZIP files.
**See [CLI Scanning Guide](docs/CLI_SCANNING.md) for detailed usage.**
---
## Features
## Documentation
### Web Application (Phase 2)
### User Guides
- **[Deployment Guide](docs/DEPLOYMENT.md)** - Installation, configuration, and production deployment
- **[CLI Scanning Guide](docs/CLI_SCANNING.md)** - Standalone scanner usage, configuration, and output formats
- **[API Reference](docs/API_REFERENCE.md)** - Complete REST API documentation
- **Dashboard** - View scan history, statistics, and recent activity
- **REST API** - Programmatic access to all scan management functions
- **Background Jobs** - Scans execute asynchronously without blocking
- **Database Storage** - Complete scan history with queryable data
- **Authentication** - Secure session-based login system
- **Pagination** - Efficiently browse large scan datasets
- **Status Tracking** - Real-time scan progress monitoring
- **Error Handling** - Comprehensive error logging and reporting
### Network Discovery & Port Scanning
- **YAML-based configuration** for defining scan targets and expectations
- **Comprehensive scanning using masscan**:
- Ping/ICMP echo detection (masscan --ping)
- TCP port scanning (all 65535 ports at 10,000 pps)
- UDP port scanning (all 65535 ports at 10,000 pps)
- Fast network-wide discovery in seconds
### Service Detection & Enumeration
- **Service detection using nmap**:
- Identifies services running on discovered TCP ports
- Extracts product names and versions (e.g., "OpenSSH 8.2p1", "nginx 1.18.0")
- Provides detailed service information including extra attributes
- Balanced intensity level (5) for accuracy and speed
### Security Assessment
- **HTTP/HTTPS analysis and SSL/TLS security assessment**:
- Detects HTTP vs HTTPS on web services
- Extracts SSL certificate details (subject, issuer, expiration, SANs)
- Calculates days until certificate expiration for monitoring
- Tests TLS version support (TLS 1.0, 1.1, 1.2, 1.3)
- Lists all accepted cipher suites for each supported TLS version
- Identifies weak cryptographic configurations
### Visual Documentation
- **Webpage screenshot capture** (NEW):
- Automatically captures screenshots of all discovered web services (HTTP/HTTPS)
- Uses Playwright with headless Chromium browser
- Viewport screenshots (1280x720) for consistent sizing
- 15-second timeout per page with graceful error handling
- Handles self-signed certificates without errors
- Saves screenshots as PNG files with references in JSON reports
- Screenshots organized in timestamped directories
- Browser reuse for optimal performance
### Reporting & Output
- **Automatic multi-format output** after each scan:
- Machine-readable JSON reports for post-processing
- Human-readable HTML reports with dark theme
- ZIP archives containing all outputs for easy sharing
- **HTML report features**:
- Comprehensive reports with dark theme for easy reading
- Summary dashboard with scan statistics, drift alerts, and security warnings
- Site-by-site breakdown with expandable service details
- Visual badges for expected vs. unexpected services
- SSL/TLS certificate details with expiration warnings
- Automatically generated after every scan
- **Dockerized** for consistent execution environment and root privilege isolation
- **Expected vs. Actual comparison** to identify infrastructure drift
- Timestamped reports with complete scan duration metrics
### Developer Resources
- **[Roadmap](docs/ROADMAP.md)** - Project roadmap, architecture, and planned features
---
## Web Application
## Current Status
### Overview
**Latest Version**: Phase 4 Complete ✅
**Last Updated**: 2025-11-17
The SneakyScanner web application provides a Flask-based interface for managing network scans. All scans are stored in a SQLite database, enabling historical analysis and trending.
### Completed Phases
### Key Features
- ✅ **Phase 1**: Database schema, SQLAlchemy models, settings system
- ✅ **Phase 2**: REST API, background jobs, authentication, web UI
- ✅ **Phase 3**: Dashboard, scheduling, trend charts
- ✅ **Phase 4**: Config creator, YAML editor, config management UI
**Scan Management:**
- Trigger scans via web UI or REST API
- View complete scan history with pagination
- Monitor real-time scan status
- Delete scans and associated files
### Next Up: Phase 5 - Email, Webhooks & Comparisons
**REST API:**
- Full CRUD operations for scans
- Session-based authentication
- JSON responses for all endpoints
- Comprehensive error handling
**Core Use Case**: Monitor infrastructure for misconfigurations that expose unexpected ports/services. When a scan detects an open port not in the config's `expected_ports` list, trigger immediate notifications.
**Background Processing:**
- APScheduler for async scan execution
- Up to 3 concurrent scans (configurable)
- Status tracking: `running` → `completed`/`failed`
- Error capture and logging
**Planned Features**:
- Email notifications for infrastructure changes
- Webhook integrations (Slack, PagerDuty, custom SIEM)
- Alert rule engine (unexpected ports, cert expiry, weak TLS)
- Scan comparison reports for drift detection
**Database Schema:**
- 11 normalized tables for scan data
- Relationships: Scans → Sites → IPs → Ports → Services → Certificates → TLS Versions
- Efficient queries with indexes
- SQLite WAL mode for better concurrency
### Web UI Routes
| Route | Description |
|-------|-------------|
| `/` | Redirects to dashboard |
| `/login` | Login page |
| `/logout` | Logout and destroy session |
| `/dashboard` | Main dashboard with stats and recent scans |
| `/scans` | Browse scan history (paginated) |
| `/scans/<id>` | View detailed scan results |
### API Endpoints
See [API_REFERENCE.md](docs/ai/API_REFERENCE.md) for complete API documentation.
**Core Endpoints:**
- `POST /api/scans` - Trigger new scan
- `GET /api/scans` - List scans (paginated, filterable)
- `GET /api/scans/{id}` - Get scan details
- `GET /api/scans/{id}/status` - Poll scan status
- `DELETE /api/scans/{id}` - Delete scan and files
**Settings Endpoints:**
- `GET /api/settings` - Get all settings
- `PUT /api/settings/{key}` - Update setting
- `GET /api/settings/health` - Health check
### Authentication
**Login:**
```bash
curl -X POST http://localhost:5000/auth/login \
-H "Content-Type: application/json" \
-d '{"password":"yourpassword"}' \
-c cookies.txt
```
**Use session for API calls:**
```bash
curl -X GET http://localhost:5000/api/scans \
-b cookies.txt
```
**Change password:**
1. Login to web UI
2. Navigate to Settings
3. Update app password
4. Or use CLI: `python3 web/utils/change_password.py`
See [Roadmap](docs/ROADMAP.md) for complete feature timeline.
---
## CLI Scanner
## Architecture
### Requirements
- Docker
- Docker Compose (optional, for easier usage)
### Using Docker Compose
1. Create or modify a configuration file in `configs/`:
```yaml
title: "My Infrastructure Scan"
sites:
- name: "Web Servers"
ips:
- address: "192.168.1.10"
expected:
ping: true
tcp_ports: [22, 80, 443]
udp_ports: []
```
┌─────────────────────────────────────────────────────────────┐
│ Flask Web Application │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Web UI │ │ REST API │ │ Scheduler │ │
│ │ (Dashboard) │ │ (JSON/CRUD) │ │ (APScheduler) │ │
│ └──────┬───────┘ └──────┬───────┘ └────────┬─────────┘ │
│ │ │ │ │
│ └─────────────────┴────────────────────┘ │
│ │ │
│ ┌────────▼────────┐ │
│ │ SQLAlchemy │ │
│ │ (ORM Layer) │ │
│ └────────┬────────┘ │
│ │ │
│ ┌────────▼────────┐ │
│ │ SQLite3 DB │ │
│ │ (scan history) │ │
│ └─────────────────┘ │
└───────────────────────────┬─────────────────────────────────┘
│
┌──────────▼──────────┐
│ Scanner Engine │
│ (scanner.py) │
│ ┌────────────────┐ │
│ │ Masscan/Nmap │ │
│ │ Playwright │ │
│ │ sslyze │ │
│ └────────────────┘ │
└─────────────────────┘
```
2. Build and run:
```bash
docker-compose build
docker-compose up
```
3. Check results in the `output/` directory:
- `scan_report_YYYYMMDD_HHMMSS.json` - JSON report
- `scan_report_YYYYMMDD_HHMMSS.html` - HTML report
- `scan_report_YYYYMMDD_HHMMSS.zip` - ZIP archive
- `scan_report_YYYYMMDD_HHMMSS_screenshots/` - Screenshots directory
## Scan Performance
SneakyScanner uses a five-phase approach for comprehensive scanning:
1. **Ping Scan** (masscan): ICMP echo detection - ~1-2 seconds
2. **TCP Port Discovery** (masscan): Scans all 65535 TCP ports at 10,000 packets/second - ~13 seconds per 2 IPs
3. **UDP Port Discovery** (masscan): Scans all 65535 UDP ports at 10,000 packets/second - ~13 seconds per 2 IPs
4. **Service Detection** (nmap): Identifies services on discovered TCP ports - ~20-60 seconds per IP with open ports
5. **HTTP/HTTPS Analysis** (Playwright, SSL/TLS): Detects web protocols, captures screenshots, and analyzes certificates - ~10-20 seconds per web service
**Example**: Scanning 2 IPs with 10 open ports each (including 2-3 web services) typically takes 2-3 minutes total.
### Using Docker Directly
1. Build the image:
```bash
docker build -t sneakyscanner .
```
2. Run a scan:
```bash
docker run --rm --privileged --network host \
-v $(pwd)/configs:/app/configs:ro \
-v $(pwd)/output:/app/output \
sneakyscanner /app/configs/your-config.yaml
```
**Technology Stack**:
- **Backend**: Flask 3.x, SQLAlchemy 2.x, SQLite3, APScheduler 3.x
- **Frontend**: Jinja2, Bootstrap 5, Chart.js, Vanilla JavaScript
- **Scanner**: Masscan, Nmap, Playwright (Chromium), sslyze
- **Deployment**: Docker Compose, Gunicorn
---
## Configuration
The YAML configuration file defines the scan parameters:
```yaml
title: "Scan Title" # Required: Report title
sites: # Required: List of sites to scan
- name: "Site Name"
ips:
- address: "192.168.1.10"
expected:
ping: true # Expected ping response
tcp_ports: [22, 80] # Expected TCP ports
udp_ports: [53] # Expected UDP ports
```
See `configs/example-site.yaml` for a complete example.
---
## Output Formats
After each scan completes, SneakyScanner automatically generates three output formats:
1. **JSON Report** (`scan_report_YYYYMMDD_HHMMSS.json`): Machine-readable scan data with all discovered services, ports, and SSL/TLS information
2. **HTML Report** (`scan_report_YYYYMMDD_HHMMSS.html`): Human-readable report with dark theme, summary dashboard, and detailed service breakdown
3. **ZIP Archive** (`scan_report_YYYYMMDD_HHMMSS.zip`): Contains JSON report, HTML report, and all screenshots for easy sharing and archival
All files share the same timestamp for easy correlation. Screenshots are saved in a subdirectory (`scan_report_YYYYMMDD_HHMMSS_screenshots/`) and included in the ZIP archive. The report includes the total scan duration (in seconds) covering all phases: ping scan, TCP/UDP port discovery, service detection, screenshot capture, and report generation.
```json
{
"title": "Sneaky Infra Scan",
"scan_time": "2024-01-15T10:30:00Z",
"scan_duration": 95.3,
"config_file": "/app/configs/example-site.yaml",
"sites": [
{
"name": "Production Web Servers",
"ips": [
{
"address": "192.168.1.10",
"expected": {
"ping": true,
"tcp_ports": [22, 80, 443],
"udp_ports": [53]
},
"actual": {
"ping": true,
"tcp_ports": [22, 80, 443, 3000],
"udp_ports": [53],
"services": [
{
"port": 22,
"protocol": "tcp",
"service": "ssh",
"product": "OpenSSH",
"version": "8.2p1"
},
{
"port": 80,
"protocol": "tcp",
"service": "http",
"product": "nginx",
"version": "1.18.0",
"http_info": {
"protocol": "http",
"screenshot": "scan_report_20250115_103000_screenshots/192_168_1_10_80.png"
}
},
{
"port": 443,
"protocol": "tcp",
"service": "https",
"product": "nginx",
"http_info": {
"protocol": "https",
"screenshot": "scan_report_20250115_103000_screenshots/192_168_1_10_443.png",
"ssl_tls": {
"certificate": {
"subject": "CN=example.com",
"issuer": "CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US",
"serial_number": "123456789012345678901234567890",
"not_valid_before": "2025-01-01T00:00:00+00:00",
"not_valid_after": "2025-04-01T23:59:59+00:00",
"days_until_expiry": 89,
"sans": ["example.com", "www.example.com"]
},
"tls_versions": {
"TLS 1.0": {
"supported": false,
"cipher_suites": []
},
"TLS 1.1": {
"supported": false,
"cipher_suites": []
},
"TLS 1.2": {
"supported": true,
"cipher_suites": [
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
]
},
"TLS 1.3": {
"supported": true,
"cipher_suites": [
"TLS_AES_256_GCM_SHA384",
"TLS_AES_128_GCM_SHA256"
]
}
}
}
}
},
{
"port": 3000,
"protocol": "tcp",
"service": "http",
"product": "Node.js",
"http_info": {
"protocol": "http"
}
}
]
}
}
]
}
]
}
```
## Screenshot Capture Details
SneakyScanner automatically captures webpage screenshots for all discovered HTTP and HTTPS services, providing visual documentation of your infrastructure.
### How It Works
1. **Automatic Detection**: During the HTTP/HTTPS analysis phase, SneakyScanner identifies web services based on:
- Nmap service detection results (http, https, ssl, http-proxy)
- Common web ports (80, 443, 8000, 8006, 8080, 8081, 8443, 8888, 9443)
2. **Screenshot Capture**: For each web service:
- Launches headless Chromium browser (once per scan, reused for all screenshots)
- Navigates to the service URL (HTTP or HTTPS)
- Waits for network to be idle (up to 15 seconds)
- Captures viewport screenshot (1280x720 pixels)
- Handles SSL certificate errors gracefully (e.g., self-signed certificates)
3. **Storage**: Screenshots are saved as PNG files:
- Directory: `output/scan_report_YYYYMMDD_HHMMSS_screenshots/`
- Filename format: `{ip}_{port}.png` (e.g., `192_168_1_10_443.png`)
- Referenced in JSON report under `http_info.screenshot`
### Screenshot Configuration
Default settings (configured in `src/screenshot_capture.py`):
- **Viewport size**: 1280x720 (captures visible area only, not full page)
- **Timeout**: 15 seconds per page load
- **Browser**: Chromium (headless mode)
- **SSL handling**: Ignores HTTPS errors (works with self-signed certificates)
- **User agent**: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
### Error Handling
Screenshots are captured on a best-effort basis:
- If a screenshot fails (timeout, connection error, etc.), the scan continues
- Failed screenshots are logged but don't stop the scan
- Services without screenshots simply omit the `screenshot` field in JSON output
## HTML Report Generation
SneakyScanner automatically generates comprehensive HTML reports after each scan, providing an easy-to-read visual interface for analyzing scan results.
### Automatic Generation
HTML reports are automatically created after every scan completes, along with JSON reports and ZIP archives. All three outputs share the same timestamp and are saved to the `output/` directory.
### Manual Generation (Optional)
You can also manually generate HTML reports from existing JSON scan data:
```bash
# Generate HTML report (creates report in same directory as JSON)
python3 src/report_generator.py output/scan_report_20251113_175235.json
# Specify custom output path
python3 src/report_generator.py output/scan_report.json /path/to/custom_report.html
```
### Report Features
The generated HTML report includes:
**Summary Dashboard**:
- **Scan Statistics**: Total IPs scanned, TCP/UDP ports found, services identified, web services, screenshots captured
- **Drift Alerts**: Unexpected TCP/UDP ports, missing expected services, new services detected
- **Security Warnings**: Expiring certificates (<30 days), weak TLS versions (1.0/1.1), self-signed certificates, high port services (>10000)
**Site-by-Site Breakdown**:
- Organized by logical site grouping from configuration
- Per-IP sections with status badges (ping, port drift summary)
- Service tables with expandable details (click any row to expand)
- Visual badges: green (expected), red (unexpected), yellow (missing/warning)
**Service Details** (click to expand):
- Product name, version, extra information, OS type
- HTTP/HTTPS protocol detection
- Screenshot links for web services
- SSL/TLS certificate details (expandable):
- Subject, issuer, validity dates, serial number
- Days until expiration with color-coded warnings
- Subject Alternative Names (SANs)
- TLS version support (1.0, 1.1, 1.2, 1.3) with cipher suites
- Weak TLS and self-signed certificate warnings
**UDP Port Handling**:
- Expected UDP ports shown with green "Expected" badge
- Unexpected UDP ports shown with red "Unexpected" badge
- Missing expected UDP ports shown with yellow "Missing" badge
- Note: Service detection not available for UDP (nmap limitation)
**Design**:
- Dark theme with slate/grey color scheme for comfortable reading
- Responsive layout works on different screen sizes
- No external dependencies - single HTML file
- Minimal JavaScript for expand/collapse functionality
- Optimized hover effects for table rows
### Report Output
The HTML report is a standalone file that can be:
- Opened directly in any web browser (Chrome, Firefox, Safari, Edge)
- Shared via email or file transfer
- Archived for compliance or historical comparison
- Viewed without an internet connection or web server
Screenshot links in the report are relative paths, so keep the report and screenshot directory together.
---
## API Documentation
Complete API reference available at [docs/ai/API_REFERENCE.md](docs/ai/API_REFERENCE.md).
**Quick Reference:**
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/scans` | POST | Trigger new scan |
| `/api/scans` | GET | List all scans (paginated) |
| `/api/scans/{id}` | GET | Get scan details |
| `/api/scans/{id}/status` | GET | Get scan status |
| `/api/scans/{id}` | DELETE | Delete scan |
| `/api/settings` | GET | Get all settings |
| `/api/settings/{key}` | PUT | Update setting |
| `/api/settings/health` | GET | Health check |
**Authentication:** All endpoints (except `/api/settings/health`) require session authentication via `/auth/login`.
---
## Deployment
### Production Deployment
See [DEPLOYMENT.md](docs/ai/DEPLOYMENT.md) for comprehensive deployment guide.
**Quick Steps:**
1. **Configure environment variables:**
```bash
cp .env.example .env
# Edit .env and set secure keys
```
2. **Initialize database:**
```bash
docker-compose -f docker-compose-web.yml run --rm web python3 init_db.py
```
3. **Start services:**
```bash
docker-compose -f docker-compose-web.yml up -d
```
4. **Verify health:**
```bash
curl http://localhost:5000/api/settings/health
```
### Docker Volumes
The web application uses persistent volumes:
| Volume | Path | Description |
|--------|------|-------------|
| `data` | `/app/data` | SQLite database |
| `output` | `/app/output` | Scan results (JSON, HTML, ZIP, screenshots) |
| `logs` | `/app/logs` | Application logs |
| `configs` | `/app/configs` | YAML scan configurations |
**Backup:**
```bash
# Backup database
docker cp sneakyscanner_web:/app/data/sneakyscanner.db ./backup/
# Backup all scan results
docker cp sneakyscanner_web:/app/output ./backup/
# Or use docker-compose volumes
docker run --rm -v sneakyscanner_data:/data -v $(pwd)/backup:/backup alpine tar czf /backup/data.tar.gz /data
```
### Environment Variables
See `.env.example` for complete configuration options:
**Flask Configuration:**
- `FLASK_ENV` - Environment mode (production/development)
- `FLASK_DEBUG` - Debug mode (true/false)
- `SECRET_KEY` - Flask secret key for sessions (generate with `secrets.token_hex(32)`)
**Database:**
- `DATABASE_URL` - Database connection string (default: SQLite)
**Security:**
- `SNEAKYSCANNER_ENCRYPTION_KEY` - Encryption key for sensitive settings (generate with `secrets.token_urlsafe(32)`)
**Scheduler:**
- `SCHEDULER_EXECUTORS` - Number of concurrent scan workers (default: 2)
- `SCHEDULER_JOB_DEFAULTS_MAX_INSTANCES` - Max concurrent jobs (default: 3)
---
## Development
### Project Structure
```
SneakyScanner/
├── src/ # Scanner engine (CLI)
│ ├── scanner.py # Main scanner application
│ ├── screenshot_capture.py # Webpage screenshot capture
│ └── report_generator.py # HTML report generation
├── web/ # Web application (Flask)
│ ├── app.py # Flask app factory
│ ├── models.py # SQLAlchemy models (11 tables)
│ ├── api/ # API blueprints
│ │ ├── scans.py # Scan management endpoints
│ │ ├── settings.py # Settings endpoints
│ │ └── ...
│ ├── auth/ # Authentication
│ │ ├── routes.py # Login/logout routes
│ │ ├── decorators.py # Auth decorators
│ │ └── models.py # User model
│ ├── routes/ # Web UI routes
│ │ └── main.py # Dashboard, scans pages
│ ├── services/ # Business logic
│ │ ├── scan_service.py # Scan CRUD operations
│ │ └── scheduler_service.py # APScheduler integration
│ ├── jobs/ # Background jobs
│ │ └── scan_job.py # Async scan execution
│ ├── utils/ # Utilities
│ │ ├── settings.py # Settings manager
│ │ ├── pagination.py # Pagination helper
│ │ └── validators.py # Input validation
│ ├── templates/ # Jinja2 templates
│ │ ├── base.html # Base layout
│ │ ├── login.html # Login page
│ │ ├── dashboard.html # Dashboard
│ │ └── errors/ # Error templates
│ └── static/ # Static assets
│ ├── css/
│ ├── js/
│ └── images/
├── templates/ # Report templates (CLI)
│ └── report_template.html # HTML report template
├── tests/ # Test suite
│ ├── conftest.py # Pytest fixtures
│ ├── test_scan_service.py # Service tests
│ ├── test_scan_api.py # API tests
│ ├── test_authentication.py # Auth tests
│ ├── test_background_jobs.py # Scheduler tests
│ └── test_error_handling.py # Error handling tests
├── migrations/ # Alembic database migrations
│ └── versions/
│ ├── 001_initial_schema.py
│ ├── 002_add_scan_indexes.py
│ └── 003_add_scan_timing_fields.py
├── configs/ # Scan configurations
│ └── example-site.yaml
├── output/ # Scan results
├── docs/ # Documentation
│ ├── ai/ # Development docs
│ │ ├── API_REFERENCE.md
│ │ ├── DEPLOYMENT.md
│ │ ├── PHASE2.md
│ │ ├── PHASE2_COMPLETE.md
│ │ └── ROADMAP.md
│ └── human/
├── Dockerfile # Scanner + web app image
├── docker-compose.yml # CLI scanner compose
├── docker-compose-web.yml # Web app compose
├── requirements.txt # Scanner dependencies
├── requirements-web.txt # Web app dependencies
├── alembic.ini # Alembic configuration
├── init_db.py # Database initialization
├── .env.example # Environment template
├── CLAUDE.md # Developer guide
└── README.md # This file
```
### Running Tests
**In Docker:**
```bash
docker-compose -f docker-compose-web.yml run --rm web pytest tests/ -v
```
**Locally (requires Python 3.12+):**
```bash
pip install -r requirements-web.txt
pytest tests/ -v
# With coverage
pytest tests/ --cov=web --cov-report=html
```
**Test Coverage:**
- 100 test functions across 6 test files
- 1,825 lines of test code
- Coverage: Service layer, API endpoints, authentication, error handling, background jobs
### Database Migrations
**Create new migration:**
```bash
docker-compose -f docker-compose-web.yml run --rm web alembic revision --autogenerate -m "Description"
```
**Apply migrations:**
```bash
docker-compose -f docker-compose-web.yml run --rm web alembic upgrade head
```
**Rollback:**
```bash
docker-compose -f docker-compose-web.yml run --rm web alembic downgrade -1
```
## Security Notice
This tool requires:
- `--privileged` flag or `CAP_NET_RAW` capability for masscan and nmap raw socket access
⚠️ **Important**: This tool requires:
- `--privileged` flag or `CAP_NET_RAW` capability for raw socket access (masscan/nmap)
- `--network host` for direct network access
Only use this tool on networks you own or have explicit authorization to scan. Unauthorized network scanning may be illegal in your jurisdiction.
**Only use this tool on networks you own or have explicit authorization to scan.** Unauthorized network scanning may be illegal in your jurisdiction.
---
### Security Best Practices
## Roadmap
1. Run on dedicated scan server (not production systems)
2. Restrict network access with firewall rules
3. Use strong passwords and encryption keys
4. Enable HTTPS in production (reverse proxy recommended)
5. Regularly update Docker images and dependencies
**Current Phase:** Phase 2 Complete ✅
**Completed Phases:**
- ✅ **Phase 1** - Database foundation, Flask app structure, settings system
- ✅ **Phase 2** - REST API, background jobs, authentication, basic UI
**Upcoming Phases:**
- 📋 **Phase 3** - Enhanced dashboard, trend charts, scheduled scans (Weeks 5-6)
- 📋 **Phase 4** - Email notifications, scan comparison, alert rules (Weeks 7-8)
- 📋 **Phase 5** - CLI as API client, token authentication (Week 9)
- 📋 **Phase 6** - Advanced features (vulnerability detection, PDF export, timeline view)
See [ROADMAP.md](docs/ai/ROADMAP.md) for detailed feature planning.
See [Deployment Guide](docs/DEPLOYMENT.md) for production security checklist.
---
## Contributing
This is a personal/small team project. For bugs or feature requests:
1. Check existing issues
2. Create detailed bug reports with reproduction steps
3. Submit pull requests with tests
@@ -800,27 +182,17 @@ MIT License - See LICENSE file for details
---
## Security Notice
This tool requires:
- `--privileged` flag or `CAP_NET_RAW` capability for masscan and nmap raw socket access
- `--network host` for direct network access
**⚠️ Important:** Only use this tool on networks you own or have explicit authorization to scan. Unauthorized network scanning may be illegal in your jurisdiction.
---
## Support
**Documentation:**
- [API Reference](docs/ai/API_REFERENCE.md)
- [Deployment Guide](docs/ai/DEPLOYMENT.md)
- [Developer Guide](CLAUDE.md)
- [Roadmap](docs/ai/ROADMAP.md)
**Documentation**:
- [Deployment Guide](docs/DEPLOYMENT.md)
- [CLI Scanning Guide](docs/CLI_SCANNING.md)
- [API Reference](docs/API_REFERENCE.md)
- [Roadmap](docs/ROADMAP.md)
**Issues:** https://github.com/anthropics/sneakyscanner/issues
**Issues**: email me ptarrant at gmail dot com
---
**Version:** 2.0 (Phase 2 Complete)
**Last Updated:** 2025-11-14
**Version**: Phase 4 Complete
**Last Updated**: 2025-11-17