SneakyScan/PHASE1_COMPLETE.md

Phase 1: Foundation - COMPLETE ✓

Date Completed: 2025-11-13

Phase 1 of the SneakyScanner roadmap has been successfully implemented. This document summarizes what was delivered and how to use the new infrastructure.

---

✓ Deliverables Completed

1. Database Schema & Models

• SQLAlchemy models for all 11 database tables (web/models.py)
  • Core tables: Scan, ScanSite, ScanIP, ScanPort, ScanService, ScanCertificate, ScanTLSVersion
  • Scheduling tables: Schedule, Alert, AlertRule
  • Configuration: Setting
• Alembic migrations system configured (migrations/)
• Initial migration created (migrations/versions/001_initial_schema.py)

2. Settings System with Encryption

• SettingsManager class with CRUD operations (web/utils/settings.py)
• Automatic encryption for sensitive values (SMTP passwords, API tokens)
• PasswordManager for bcrypt password hashing
• Default settings initialization for SMTP, authentication, retention policies

3. Flask Application Structure

• Flask app factory pattern implemented (web/app.py)
• API blueprints for all major endpoints:
  • /api/scans - Scan management (stub for Phase 2)
  • /api/schedules - Schedule management (stub for Phase 3)
  • /api/alerts - Alert management (stub for Phase 4)
  • /api/settings - Settings API (functional in Phase 1!)
• Error handlers for common HTTP status codes
• CORS support for API access
• Logging to file and console
• Database session management with scoped sessions

4. Database Initialization

• init_db.py script for easy database setup
• Supports both Alembic migrations and direct table creation
• Password setting during initialization
• Database verification and settings display

5. Docker Support

• Updated Dockerfile with Flask dependencies
• docker-compose-web.yml for running the web application
• Separate service definition for database initialization
• Volume mounts for persistence (database, output, logs)

6. Validation & Testing

• validate_phase1.py script to verify all deliverables
• Validates directory structure, files, Python syntax, models, and API endpoints
• All checks passing ✓

---

📁 New Project Structure

SneakyScanner/
├── web/                          # Flask web application (NEW)
│   ├── __init__.py
│   ├── app.py                    # Flask app factory
│   ├── models.py                 # SQLAlchemy models (11 tables)
│   ├── api/                      # API blueprints
│   │   ├── __init__.py
│   │   ├── scans.py              # Scans API
│   │   ├── schedules.py          # Schedules API
│   │   ├── alerts.py             # Alerts API
│   │   └── settings.py           # Settings API (functional!)
│   ├── templates/                # Jinja2 templates (for Phase 3)
│   ├── static/                   # CSS, JS, images (for Phase 3)
│   │   ├── css/
│   │   ├── js/
│   │   └── images/
│   └── utils/                    # Utility modules
│       ├── __init__.py
│       └── settings.py           # Settings manager with encryption
├── migrations/                   # Alembic migrations (NEW)
│   ├── env.py                    # Alembic environment
│   ├── script.py.mako            # Migration template
│   └── versions/
│       └── 001_initial_schema.py # Initial database migration
├── alembic.ini                   # Alembic configuration (NEW)
├── init_db.py                    # Database initialization script (NEW)
├── validate_phase1.py            # Phase 1 validation script (NEW)
├── requirements-web.txt          # Flask dependencies (NEW)
├── docker-compose-web.yml        # Docker Compose for web app (NEW)
├── Dockerfile                    # Updated with Flask support
├── src/                          # Existing scanner code (unchanged)
├── templates/                    # Existing report templates (unchanged)
├── configs/                      # Existing YAML configs (unchanged)
└── output/                       # Existing scan outputs (unchanged)

---

🚀 Getting Started

Option 1: Local Development (without Docker)

1. Install Dependencies

# Install Flask and web dependencies
pip install -r requirements-web.txt

2. Initialize Database

# Create database and set password
python3 init_db.py --password YOUR_SECURE_PASSWORD

# Verify database
python3 init_db.py --verify-only

3. Run Flask Application

# Run development server
python3 -m web.app

# Application will be available at http://localhost:5000

4. Test API Endpoints

# Health check
curl http://localhost:5000/api/settings/health

# Get all settings (sanitized)
curl http://localhost:5000/api/settings

# Get specific setting
curl http://localhost:5000/api/settings/smtp_server

# Update a setting
curl -X PUT http://localhost:5000/api/settings/smtp_server \
  -H "Content-Type: application/json" \
  -d '{"value": "smtp.gmail.com"}'

# Set application password
curl -X POST http://localhost:5000/api/settings/password \
  -H "Content-Type: application/json" \
  -d '{"password": "newsecurepassword"}'

---

Option 2: Docker Deployment

1. Build Docker Image

docker-compose -f docker-compose-web.yml build

2. Initialize Database (one-time)

# Create data directory
mkdir -p data

# Initialize database
docker-compose -f docker-compose-web.yml run --rm init-db --password YOUR_SECURE_PASSWORD

3. Run Web Application

# Start Flask web server
docker-compose -f docker-compose-web.yml up -d web

# View logs
docker-compose -f docker-compose-web.yml logs -f web

4. Access Application

• Web API: http://localhost:5000
• Health checks:
  • http://localhost:5000/api/scans/health
  • http://localhost:5000/api/schedules/health
  • http://localhost:5000/api/alerts/health
  • http://localhost:5000/api/settings/health

---

🔐 Security Features

Encryption

• Fernet encryption for sensitive settings (SMTP passwords, API tokens)
• Encryption key auto-generated and stored in settings table
• Can be overridden via SNEAKYSCANNER_ENCRYPTION_KEY environment variable

Password Hashing

• Bcrypt for application password hashing (work factor 12)
• Password stored as irreversible hash in settings table
• Minimum 8 characters enforced

Session Management

• Flask sessions with configurable SECRET_KEY
• Set via environment variable or config

---

📊 Database Schema

Core Tables

• scans - Scan metadata and status
• scan_sites - Site groupings
• scan_ips - IP addresses scanned
• scan_ports - Discovered ports
• scan_services - Service detection results
• scan_certificates - SSL/TLS certificates
• scan_tls_versions - TLS version support

Scheduling & Alerts

• schedules - Cron-like scan schedules
• alerts - Alert history
• alert_rules - Alert rule definitions

Configuration

• settings - Application settings (key-value store)

All tables include proper foreign keys, indexes, and cascade delete rules.

---

🧪 Validation

Run the Phase 1 validation script to verify everything is in place:

python3 validate_phase1.py

Expected output:

✓ All Phase 1 validation checks passed!

---

🔧 Environment Variables

Configure the Flask app via environment variables:

# Flask configuration
export FLASK_ENV=development
export FLASK_DEBUG=true
export FLASK_HOST=0.0.0.0
export FLASK_PORT=5000

# Database
export DATABASE_URL=sqlite:///./sneakyscanner.db

# Security
export SECRET_KEY=your-secret-key-here
export SNEAKYSCANNER_ENCRYPTION_KEY=your-encryption-key-here

# CORS (comma-separated origins)
export CORS_ORIGINS=http://localhost:3000,https://your-domain.com

# Logging
export LOG_LEVEL=INFO

Or use a .env file (supported via python-dotenv).

---

📝 API Endpoints Summary

Settings API (Functional in Phase 1)

Method	Endpoint	Description	Status
GET	/api/settings	Get all settings (sanitized)	✓ Working
PUT	/api/settings	Update multiple settings	✓ Working
GET	/api/settings/{key}	Get specific setting	✓ Working
PUT	/api/settings/{key}	Update specific setting	✓ Working
DELETE	/api/settings/{key}	Delete setting	✓ Working
POST	/api/settings/password	Set app password	✓ Working
GET	/api/settings/health	Health check	✓ Working

Scans API (Stubs for Phase 2)

Method	Endpoint	Description	Status
GET	/api/scans	List scans	Phase 2
GET	/api/scans/{id}	Get scan details	Phase 2
POST	/api/scans	Trigger scan	Phase 2
DELETE	/api/scans/{id}	Delete scan	Phase 2
GET	/api/scans/{id}/status	Get scan status	Phase 2
GET	/api/scans/health	Health check	✓ Working

Schedules API (Stubs for Phase 3)

Method	Endpoint	Description	Status
GET	/api/schedules	List schedules	Phase 3
POST	/api/schedules	Create schedule	Phase 3
PUT	/api/schedules/{id}	Update schedule	Phase 3
DELETE	/api/schedules/{id}	Delete schedule	Phase 3
POST	/api/schedules/{id}/trigger	Trigger schedule	Phase 3
GET	/api/schedules/health	Health check	✓ Working

Alerts API (Stubs for Phase 4)

Method	Endpoint	Description	Status
GET	/api/alerts	List alerts	Phase 4
GET	/api/alerts/rules	List alert rules	Phase 4
POST	/api/alerts/rules	Create alert rule	Phase 4
PUT	/api/alerts/rules/{id}	Update alert rule	Phase 4
DELETE	/api/alerts/rules/{id}	Delete alert rule	Phase 4
GET	/api/alerts/health	Health check	✓ Working

---

✅ Testing Checklist

• Database creates successfully
• Settings can be stored/retrieved
• Encryption works for sensitive values
• Password hashing works
• Flask app starts without errors
• API blueprints load correctly
• Health check endpoints respond
• All Python files have valid syntax
• All models defined correctly
• Database migrations work

---

🎯 Next Steps: Phase 2

Phase 2 will implement:

1. REST API for scans - Trigger scans, list history, get results
2. Background job queue - APScheduler for async scan execution
3. Authentication - Flask-Login for session management
4. Scanner integration - Save scan results to database
5. Docker Compose deployment - Production-ready setup

Estimated timeline: 2 weeks (as per roadmap)

---

📚 References

Key Files

• web/models.py - Database models (lines 1-400+)
• web/app.py - Flask app factory (lines 1-250+)
• web/utils/settings.py - Settings manager (lines 1-300+)
• init_db.py - Database initialization (lines 1-200+)
• migrations/versions/001_initial_schema.py - Initial migration (lines 1-250+)

Documentation

• Flask Documentation
• SQLAlchemy ORM
• Alembic Migrations
• Cryptography Library
• Bcrypt

---

🐛 Troubleshooting

Database Issues

# Reset database
rm sneakyscanner.db
python3 init_db.py --password newpassword

# Check database
sqlite3 sneakyscanner.db ".schema"

Flask Won't Start

# Check dependencies installed
pip list | grep -i flask

# Check syntax errors
python3 validate_phase1.py

# Run with debug output
FLASK_DEBUG=true python3 -m web.app

Encryption Errors

# Generate new encryption key
python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"

# Set in environment
export SNEAKYSCANNER_ENCRYPTION_KEY="your-key-here"

---

Phase 1 Status: ✅ COMPLETE

All deliverables implemented, tested, and validated. Ready to proceed with Phase 2.