Updated init_db.py to use config_id field after database migration,
fixing container startup error on new systems.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Remove the database init marker when regenerating .env file so that
the docker entrypoint will re-run password initialization with the
new INITIAL_PASSWORD value on next container start.
Major architectural changes:
- Replace YAML config files with database-stored ScanConfig model
- Remove CIDR block support in favor of individual IP addresses per site
- Each IP now has its own expected_ping, expected_tcp_ports, expected_udp_ports
- AlertRule now uses config_id FK instead of config_file string
API changes:
- POST /api/scans now requires config_id instead of config_file
- Alert rules API uses config_id with validation
- All config dropdowns fetch from /api/configs dynamically
Template updates:
- scans.html, dashboard.html, alert_rules.html load configs via API
- Display format: Config Title (X sites) in dropdowns
- Removed Jinja2 config_files loops
Migrations:
- 008: Expand CIDRs to individual IPs with per-IP port configs
- 009: Remove CIDR-related columns
- 010: Add config_id to alert_rules, remove config_file
Implemented comprehensive scan comparison functionality with historical
analysis and improved user experience for scan triggering.
Features Added:
- Scan comparison engine with ports, services, and certificates analysis
- Drift score calculation (0.0-1.0 scale) for infrastructure changes
- Side-by-side comparison UI with color-coded changes (added/removed/changed)
- Historical trend charts showing port counts over time
- "Compare with Previous" button on scan detail pages
- Scan history API endpoint for trending data
API Endpoints:
- GET /api/scans/<id1>/compare/<id2> - Compare two scans
- GET /api/stats/scan-history/<id> - Historical scan data for charts
UI Improvements:
- Replaced config file text inputs with dropdown selectors
- Added config file selection to dashboard and scans pages
- Improved delete scan confirmation with proper async handling
- Enhanced error messages with detailed validation feedback
- Added 2-second delay before redirect to ensure deletion completes
Comparison Features:
- Port changes: tracks added, removed, and unchanged ports
- Service changes: detects version updates and service modifications
- Certificate changes: monitors SSL/TLS certificate updates
- Interactive historical charts with clickable data points
- Automatic detection of previous scan for comparison
Bug Fixes:
- Fixed scan deletion UI alert appearing on successful deletion
- Prevented config file path duplication (configs/configs/...)
- Improved error handling for failed API responses
- Added proper JSON response parsing with fallback handling
Testing:
- Created comprehensive test suite for comparison functionality
- Tests cover comparison API, service methods, and drift scoring
- Added edge case tests for identical scans and missing data
Implemented complete scheduler integration with automatic schedule loading,
orphaned scan cleanup, and conversion to local timezone for better UX.
Backend Changes:
- Added load_schedules_on_startup() to load enabled schedules on app start
- Implemented cleanup_orphaned_scans() to handle crashed/interrupted scans
- Converted scheduler from UTC to local system timezone throughout
- Enhanced scheduler service with robust error handling and logging
Frontend Changes:
- Updated all schedule UI templates to display local time instead of UTC
- Improved timezone indicators and user messaging
- Removed confusing timezone converter (no longer needed)
- Updated quick templates and help text for local time
Bug Fixes:
- Fixed critical timezone bug causing cron expressions to run at wrong times
- Fixed orphaned scans stuck in 'running' status after system crashes
- Improved time display clarity across all schedule pages
All schedules now use local system time for intuitive scheduling.
Implemented dashboard visualizations and statistics API endpoints:
New Features:
- Stats API endpoints (/api/stats/scan-trend, /api/stats/summary)
- Chart.js trending chart showing 30-day scan activity
- Schedules widget displaying next 3 upcoming scheduled scans
- Enhanced Quick Actions with Manage Schedules button
Stats API (web/api/stats.py):
- scan-trend endpoint with configurable days (1-365)
- Summary endpoint for dashboard statistics
- Automatic date range filling with zeros for missing days
- Proper authentication and validation
Dashboard Enhancements (web/templates/dashboard.html):
- Chart.js line chart with dark theme styling
- Real-time schedules widget with human-readable time display
- Auto-refresh for schedules every 30 seconds
- Responsive 8-4 column layout for chart and schedules
Tests (tests/test_stats_api.py):
- 18 comprehensive test cases for stats API
- Coverage for date validation, authentication, edge cases
- Tests for empty data handling and date formatting
Progress: 64% complete (9/14 days)
Next: Step 6 - Scheduler Integration
Implemented full schedule management system with API endpoints and
user interface for creating, editing, and managing scheduled scans.
API Implementation:
- Implemented all 6 schedules API endpoints (list, get, create, update, delete, trigger)
- Added comprehensive error handling and validation
- Integrated with ScheduleService and SchedulerService
- Added manual trigger endpoint for on-demand execution
Schedule Management UI:
- Created schedules list page with stats cards and enable/disable toggles
- Built schedule creation form with cron expression builder and quick templates
- Implemented schedule edit page with execution history
- Added "Schedules" navigation link to main menu
- Real-time validation and human-readable cron descriptions
Config File Path Resolution:
- Fixed config file path handling to support relative filenames
- Updated validators.py to resolve relative paths to /app/configs/
- Modified schedule_service.py, scan_service.py, and scan_job.py for consistency
- Ensures UI can use simple filenames while backend uses absolute paths
Scheduler Integration:
- Completed scheduled scan execution in scheduler_service.py
- Added cron job management with APScheduler
- Implemented automatic schedule loading on startup
- Updated run times after each execution
Testing:
- Added comprehensive API integration tests (test_schedule_api.py)
- 22+ test cases covering all endpoints and workflows
Progress: Phase 3 Steps 1-4 complete (36% - 5/14 days)
Next: Step 5 - Enhanced Dashboard with Charts
Implement comprehensive schedule management service for automated scans:
New Files:
- web/services/schedule_service.py (470 lines)
* Complete CRUD operations for schedules
* Cron expression validation using croniter
* Next run time calculation
* Execution history tracking
* Human-readable relative time formatting
- tests/test_schedule_service.py (671 lines, 40+ tests)
* Create/get/list/update/delete schedule tests
* Cron validation and next run calculation tests
* Pagination and filtering tests
* Schedule history and serialization tests
Changes:
- requirements-web.txt: Add croniter==2.0.1 dependency
- docs/ai/PHASE3.md: Mark Step 1 complete, Step 2 next
Key Features:
- Validates cron expressions before saving
- Automatically calculates next execution time
- Preserves historical scans when schedules deleted
- Supports pagination and filtering by enabled status
- Provides relative time display (e.g., "in 2 hours")
Extracted inline CSS to external stylesheet and fixed white row bug
affecting dynamically created table rows across all scan views.
Changes:
- Created web/static/css/styles.css with extracted CSS from base.html
- Added CSS variables for consistent theming and maintainability
- Added Bootstrap 5 CSS variable overrides to fix table styling
- Integrated Chart.js 4.4.0 for future dashboard visualizations
- Added Bootstrap Icons for enhanced UI components
Template Updates:
- Updated base.html to use external CSS instead of inline styles
- Added Chart.js dark theme configuration
- Fixed white row bug in dashboard.html (added .scan-row class)
- Fixed white row bug in scans.html (added .scan-row class)
- Fixed white row bug in scan_detail.html port tables (added .scan-row class)
The white row bug was caused by Bootstrap 5's CSS variables overriding
custom styles. Fixed by setting --bs-table-bg and related variables.
Phase 3 Documentation:
- Added PHASE3.md with complete implementation plan (2204 lines)
- Includes 8 implementation steps, file changes, and success criteria
This completes Phase 3 Step 1 (Day 1 of 14).
Implement production-ready Docker deployment with comprehensive configuration
and documentation for SneakyScanner web application.
Changes:
- Update docker-compose-web.yml with production configuration
- Add scheduler environment variables (SCHEDULER_EXECUTORS, SCHEDULER_JOB_DEFAULTS_MAX_INSTANCES)
- Enable privileged mode and host networking for scanner operations
- Configure health check endpoint monitoring (30s interval, 40s start period)
- Set production defaults (FLASK_ENV=production, FLASK_DEBUG=false)
- Add SNEAKYSCANNER_ENCRYPTION_KEY support
- Create .env.example configuration template
- Flask, database, and security settings
- Scheduler configuration options
- Detailed comments with key generation examples
- Production deployment guidance
- Create comprehensive deployment documentation (docs/ai/DEPLOYMENT.md)
- Quick start guide and prerequisites
- Detailed configuration instructions
- Volume management and backup procedures
- Health monitoring and troubleshooting
- Security considerations and best practices
- Upgrade/rollback and backup/restore procedures
- Update PHASE2.md progress tracker
- Mark Step 6 as complete
- Update progress to 11/14 days (79%)
- Document deliverables and implementation details
Deployment is now production-ready with proper security defaults, health
monitoring, and comprehensive documentation for system administrators.
Implement comprehensive web UI with dark slate theme matching HTML reports:
Templates:
- Create base.html with navigation, dark theme (#0f172a background)
- Update dashboard.html with stats cards and recent scans table
- Update scans.html with pagination, filtering, and status badges
- Update scan_detail.html with comprehensive scan results display
- Update login.html to extend base template with centered design
Features:
- AJAX-powered dynamic data loading from API endpoints
- Auto-refresh for running scans (10-15 second intervals)
- Responsive Bootstrap 5 grid layout
- Color scheme matches report_mockup.html (slate dark theme)
- Status badges (success/danger/warning/info) with proper colors
- Modal dialogs for triggering scans
- Pagination with ellipsis for large result sets
- Delete confirmation dialogs
- Loading spinners for async operations
Bug Fixes:
- Fix scanner.py imports to use 'src.' prefix for module imports
- Fix scans.py to import validate_page_params from pagination module
All templates use consistent color palette:
- Background: #0f172a, Cards: #1e293b, Accent: #60a5fa
- Success: #065f46/#6ee7b7, Danger: #7f1d1d/#fca5a5
- Warning: #78350f/#fcd34d, Info: #1e3a8a/#93c5fd
Implemented comprehensive Flask-Login authentication with single-user support.
New Features:
- Flask-Login integration with User model
- Bcrypt password hashing via PasswordManager
- Login, logout, and initial password setup routes
- @login_required and @api_auth_required decorators
- All API endpoints now require authentication
- Bootstrap 5 dark theme UI templates
- Dashboard with navigation
- Remember me and next parameter redirect support
Files Created (12):
- web/auth/__init__.py, models.py, decorators.py, routes.py
- web/routes/__init__.py, main.py
- web/templates/login.html, setup.html, dashboard.html, scans.html, scan_detail.html
- tests/test_authentication.py (30+ tests)
Files Modified (6):
- web/app.py: Added Flask-Login initialization and main routes
- web/api/scans.py: Protected all endpoints with @api_auth_required
- web/api/settings.py: Protected all endpoints with @api_auth_required
- web/api/schedules.py: Protected all endpoints with @api_auth_required
- web/api/alerts.py: Protected all endpoints with @api_auth_required
- tests/conftest.py: Added authentication test fixtures
Security:
- Session-based authentication for both web UI and API
- Secure password storage with bcrypt
- Protected routes redirect to login page
- Protected API endpoints return 401 Unauthorized
- Health check endpoints remain accessible for monitoring
Testing:
- User model authentication and properties
- Login success/failure flows
- Logout and session management
- Password setup workflow
- API endpoint authentication requirements
- Session persistence and remember me functionality
- Next parameter redirect behavior
Total: ~1,200 lines of code added
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implemented all 5 scan management endpoints with comprehensive error
handling, logging, and integration tests.
## Changes
### API Endpoints (web/api/scans.py)
- POST /api/scans - Trigger new scan with config file validation
- GET /api/scans - List scans with pagination and status filtering
- GET /api/scans/<id> - Retrieve scan details with all relationships
- DELETE /api/scans/<id> - Delete scan and associated files
- GET /api/scans/<id>/status - Poll scan status for long-running scans
### Features
- Comprehensive error handling (400, 404, 500)
- Structured logging with appropriate levels
- Input validation via validators
- Consistent JSON error format
- SQLAlchemy error handling with graceful degradation
- HTTP status codes following REST conventions
### Testing (tests/test_scan_api.py)
- 24 integration tests covering all endpoints
- Empty/populated scan lists
- Pagination with multiple pages
- Status filtering
- Error scenarios (invalid input, not found, etc.)
- Complete workflow integration test
### Test Infrastructure (tests/conftest.py)
- Flask app fixture with test database
- Flask test client fixture
- Database session fixture compatible with app context
- Sample scan fixture for testing
### Documentation (docs/ai/PHASE2.md)
- Updated progress: 4/14 days complete (29%)
- Marked Step 2 as complete
- Added implementation details and testing results
## Implementation Notes
- All endpoints use ScanService for business logic separation
- Scan triggering returns immediately; client polls status endpoint
- Background job execution will be added in Step 3
- Authentication will be added in Step 4
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Complete the foundation for Phase 2 by implementing the service layer,
utilities, and comprehensive test suite. This establishes the core
business logic for scan management.
Service Layer:
- Add ScanService class with complete scan lifecycle management
* trigger_scan() - Create scan record and prepare for execution
* get_scan() - Retrieve scan with all related data (eager loading)
* list_scans() - Paginated scan list with status filtering
* delete_scan() - Remove scan from DB and delete all files
* get_scan_status() - Poll current scan status and progress
* _save_scan_to_db() - Persist scan results to database
* _map_report_to_models() - Complex JSON-to-DB mapping logic
Database Mapping:
- Comprehensive mapping from scanner JSON output to normalized schema
- Handles nested relationships: sites → IPs → ports → services → certs → TLS
- Processes both TCP and UDP ports with expected/actual tracking
- Maps service detection results with HTTP/HTTPS information
- Stores SSL/TLS certificates with expiration tracking
- Records TLS version support and cipher suites
- Links screenshots to services
Utilities:
- Add pagination.py with PaginatedResult class
* paginate() function for SQLAlchemy queries
* validate_page_params() for input sanitization
* Metadata: total, pages, has_prev, has_next, etc.
- Add validators.py with comprehensive validation functions
* validate_config_file() - YAML structure and required fields
* validate_scan_status() - Enum validation (running/completed/failed)
* validate_scan_id() - Positive integer validation
* validate_port() - Port range validation (1-65535)
* validate_ip_address() - Basic IPv4 format validation
* sanitize_filename() - Path traversal prevention
Database Migration:
- Add migration 002 for scan status index
- Optimizes queries filtering by scan status
- Timestamp index already exists from migration 001
Testing:
- Add pytest infrastructure with conftest.py
* test_db fixture - Temporary SQLite database per test
* sample_scan_report fixture - Realistic scanner output
* sample_config_file fixture - Valid YAML config
* sample_invalid_config_file fixture - For validation tests
- Add comprehensive test_scan_service.py (15 tests)
* Test scan trigger with valid/invalid configs
* Test scan retrieval (found/not found cases)
* Test scan listing with pagination and filtering
* Test scan deletion with cascade cleanup
* Test scan status retrieval
* Test database mapping from JSON to models
* Test expected vs actual port flagging
* Test certificate and TLS data mapping
* Test full scan retrieval with all relationships
* All tests passing
Files Added:
- web/services/__init__.py
- web/services/scan_service.py (545 lines)
- web/utils/pagination.py (153 lines)
- web/utils/validators.py (245 lines)
- migrations/versions/002_add_scan_indexes.py
- tests/__init__.py
- tests/conftest.py (142 lines)
- tests/test_scan_service.py (374 lines)
Next Steps (Step 2):
- Implement scan API endpoints in web/api/scans.py
- Add authentication decorators
- Integrate ScanService with API routes
- Test API endpoints with integration tests
Phase 2 Step 1 Complete ✓
Implement complete database schema and Flask application structure for
SneakyScan web interface. This establishes the foundation for web-based
scan management, scheduling, and visualization.
Database & ORM:
- Add 11 SQLAlchemy models for comprehensive scan data storage
(Scan, ScanSite, ScanIP, ScanPort, ScanService, ScanCertificate,
ScanTLSVersion, Schedule, Alert, AlertRule, Setting)
- Configure Alembic migrations system with initial schema migration
- Add init_db.py script for database initialization and password setup
- Support both migration-based and direct table creation
Settings System:
- Implement SettingsManager with automatic encryption for sensitive values
- Add Fernet encryption for SMTP passwords and API tokens
- Implement PasswordManager with bcrypt password hashing (work factor 12)
- Initialize default settings for SMTP, authentication, and retention
Flask Application:
- Create Flask app factory pattern with scoped session management
- Add 4 API blueprints: scans, schedules, alerts, settings
- Implement functional Settings API (GET/PUT/DELETE endpoints)
- Add CORS support, error handlers, and request/response logging
- Configure development and production logging to file and console
Docker & Deployment:
- Update Dockerfile to install Flask dependencies
- Add docker-compose-web.yml for web application deployment
- Configure volume mounts for database, output, and logs persistence
- Expose port 5000 for Flask web server
Testing & Validation:
- Add validate_phase1.py script to verify all deliverables
- Validate directory structure, Python syntax, models, and endpoints
- All validation checks passing
Documentation:
- Add PHASE1_COMPLETE.md with comprehensive Phase 1 summary
- Update ROADMAP.md with Phase 1 completion status
- Update .gitignore to exclude database files and documentation
Files changed: 21 files
- New: web/ directory with complete Flask app structure
- New: migrations/ with Alembic configuration
- New: requirements-web.txt with Flask dependencies
- Modified: Dockerfile, ROADMAP.md, .gitignore
Implements automatic generation of JSON, HTML, and ZIP outputs after every scan,
with all files sharing the same timestamp for easy correlation.
Features:
- Automatic HTML report generation after every scan
- ZIP archive creation containing JSON, HTML, and all screenshots
- Unified timestamp across all outputs (JSON, HTML, ZIP, screenshots)
- Graceful error handling (scan continues if HTML/ZIP generation fails)
- Email-ready ZIP archives for easy sharing
Technical changes:
- Fixed timestamp mismatch between scan() and save_report()
- Added generate_outputs() method to SneakyScanner class
- scan() now returns (report, timestamp) tuple
- save_report() accepts timestamp parameter instead of generating new one
- main() updated to call generate_outputs() for all output formats
- Added zipfile import and HTMLReportGenerator import
- Dockerfile updated to copy templates/ directory
Output structure:
- scan_report_YYYYMMDD_HHMMSS.json (JSON report)
- scan_report_YYYYMMDD_HHMMSS.html (HTML report)
- scan_report_YYYYMMDD_HHMMSS.zip (archive with JSON, HTML, screenshots)
- scan_report_YYYYMMDD_HHMMSS_screenshots/ (screenshots directory)
Documentation updated:
- README.md: Updated Output Format, Features, Quick Start sections
- CLAUDE.md: Updated Core Components, Scan Workflow, Key Design Decisions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements comprehensive HTML report generation from JSON scan data with Jinja2 templates. Reports feature a dark slate theme with summary dashboard, drift alerts, security warnings, and expandable service details.
Features:
- Dark theme HTML reports with slate/grey color scheme
- Summary dashboard: scan statistics, drift alerts, security warnings
- Site-by-site breakdown with IP grouping and status badges
- Expandable service details and SSL/TLS certificate information
- Visual badges: green (expected), red (unexpected), yellow (missing)
- UDP port handling: shows expected, unexpected, and missing UDP ports
- Screenshot links with relative paths for portability
- Optimized hover effects for table rows
- Standalone HTML output (no external dependencies)
Technical changes:
- Added src/report_generator.py: HTMLReportGenerator class with summary calculations
- Added templates/report_template.html: Jinja2 template for dynamic reports
- Added templates/report_mockup.html: Static mockup for design testing
- Updated requirements.txt: Added Jinja2==3.1.2
- Updated README.md: Added HTML report generation section with usage and features
- Updated CLAUDE.md: Added implementation details, usage guide, and troubleshooting
Usage:
python3 src/report_generator.py output/scan_report.json
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
