Commit Graph

7 Commits

Author SHA1 Message Date
6792d69eb1 Phase 3 Step 7: Scan Comparison Features & UX Improvements
Implemented comprehensive scan comparison functionality with historical
analysis and improved user experience for scan triggering.

Features Added:
- Scan comparison engine with ports, services, and certificates analysis
- Drift score calculation (0.0-1.0 scale) for infrastructure changes
- Side-by-side comparison UI with color-coded changes (added/removed/changed)
- Historical trend charts showing port counts over time
- "Compare with Previous" button on scan detail pages
- Scan history API endpoint for trending data

API Endpoints:
- GET /api/scans/<id1>/compare/<id2> - Compare two scans
- GET /api/stats/scan-history/<id> - Historical scan data for charts

UI Improvements:
- Replaced config file text inputs with dropdown selectors
- Added config file selection to dashboard and scans pages
- Improved delete scan confirmation with proper async handling
- Enhanced error messages with detailed validation feedback
- Added 2-second delay before redirect to ensure deletion completes

Comparison Features:
- Port changes: tracks added, removed, and unchanged ports
- Service changes: detects version updates and service modifications
- Certificate changes: monitors SSL/TLS certificate updates
- Interactive historical charts with clickable data points
- Automatic detection of previous scan for comparison

Bug Fixes:
- Fixed scan deletion UI alert appearing on successful deletion
- Prevented config file path duplication (configs/configs/...)
- Improved error handling for failed API responses
- Added proper JSON response parsing with fallback handling

Testing:
- Created comprehensive test suite for comparison functionality
- Tests cover comparison API, service methods, and drift scoring
- Added edge case tests for identical scans and missing data
2025-11-14 16:15:13 -06:00
167ab803a6 Phase 2 Step 7: Implement Error Handling & Logging
Comprehensive error handling and logging system with production-ready
features for monitoring, debugging, and user experience.

Enhanced Logging System:
- Implemented RotatingFileHandler (10MB per file, 10 backups, 100MB total)
- Separate error log file for ERROR level messages with detailed tracebacks
- Structured logging with request IDs, timestamps, and module names
- RequestIDLogFilter for automatic request context injection
- Console logging in debug mode with simplified format

Request/Response Middleware:
- Request ID generation using UUID (8-character prefix for readability)
- Request timing with millisecond precision
- User authentication context in all logs
- Response duration tracking and headers (X-Request-ID, X-Request-Duration-Ms)
- Security headers: X-Content-Type-Options, X-Frame-Options, X-XSS-Protection

Database Error Handling:
- Enabled SQLite WAL mode for better concurrency with background jobs
- Busy timeout configuration (15 seconds) for lock handling
- Automatic rollback on request exceptions via teardown handler
- Dedicated SQLAlchemyError handler with explicit rollback
- Connection pooling with pre-ping validation

Comprehensive Error Handlers:
- Content negotiation: JSON responses for API, HTML for web requests
- Error handlers for 400, 401, 403, 404, 405, 500
- Database rollback in all error handlers
- Full exception logging with traceback for debugging

Custom Error Templates:
- Created web/templates/errors/ directory with 6 templates
- Dark theme matching application design (slate colors)
- User-friendly error messages with navigation
- Templates: 400, 401, 403, 404, 405, 500

Testing:
- Comprehensive test suite (320+ lines) in tests/test_error_handling.py
- Tests for JSON vs HTML error responses
- Request ID and duration header verification
- Security header validation
- Log rotation configuration tests
- Structured logging tests

Bug Fix:
- Fixed pagination bug in scans API endpoint
- Changed paginated_result.total_pages to paginated_result.pages
- Resolves AttributeError when listing scans

Files Added:
- tests/test_error_handling.py
- web/templates/errors/400.html
- web/templates/errors/401.html
- web/templates/errors/403.html
- web/templates/errors/404.html
- web/templates/errors/405.html
- web/templates/errors/500.html

Files Modified:
- web/app.py (logging, error handlers, request handlers, database config)
- web/api/scans.py (pagination bug fix)
- docs/ai/PHASE2.md (mark Step 7 complete, update progress to 86%)

Phase 2 Progress: 12/14 days complete (86%)
2025-11-14 12:19:07 -06:00
a64096ece3 Phase 2 Step 5: Implement Basic UI Templates
Implement comprehensive web UI with dark slate theme matching HTML reports:

Templates:
- Create base.html with navigation, dark theme (#0f172a background)
- Update dashboard.html with stats cards and recent scans table
- Update scans.html with pagination, filtering, and status badges
- Update scan_detail.html with comprehensive scan results display
- Update login.html to extend base template with centered design

Features:
- AJAX-powered dynamic data loading from API endpoints
- Auto-refresh for running scans (10-15 second intervals)
- Responsive Bootstrap 5 grid layout
- Color scheme matches report_mockup.html (slate dark theme)
- Status badges (success/danger/warning/info) with proper colors
- Modal dialogs for triggering scans
- Pagination with ellipsis for large result sets
- Delete confirmation dialogs
- Loading spinners for async operations

Bug Fixes:
- Fix scanner.py imports to use 'src.' prefix for module imports
- Fix scans.py to import validate_page_params from pagination module

All templates use consistent color palette:
- Background: #0f172a, Cards: #1e293b, Accent: #60a5fa
- Success: #065f46/#6ee7b7, Danger: #7f1d1d/#fca5a5
- Warning: #78350f/#fcd34d, Info: #1e3a8a/#93c5fd
2025-11-14 11:51:27 -06:00
abc682a634 Phase 2 Step 4: Implement Authentication System
Implemented comprehensive Flask-Login authentication with single-user support.

New Features:
- Flask-Login integration with User model
- Bcrypt password hashing via PasswordManager
- Login, logout, and initial password setup routes
- @login_required and @api_auth_required decorators
- All API endpoints now require authentication
- Bootstrap 5 dark theme UI templates
- Dashboard with navigation
- Remember me and next parameter redirect support

Files Created (12):
- web/auth/__init__.py, models.py, decorators.py, routes.py
- web/routes/__init__.py, main.py
- web/templates/login.html, setup.html, dashboard.html, scans.html, scan_detail.html
- tests/test_authentication.py (30+ tests)

Files Modified (6):
- web/app.py: Added Flask-Login initialization and main routes
- web/api/scans.py: Protected all endpoints with @api_auth_required
- web/api/settings.py: Protected all endpoints with @api_auth_required
- web/api/schedules.py: Protected all endpoints with @api_auth_required
- web/api/alerts.py: Protected all endpoints with @api_auth_required
- tests/conftest.py: Added authentication test fixtures

Security:
- Session-based authentication for both web UI and API
- Secure password storage with bcrypt
- Protected routes redirect to login page
- Protected API endpoints return 401 Unauthorized
- Health check endpoints remain accessible for monitoring

Testing:
- User model authentication and properties
- Login success/failure flows
- Logout and session management
- Password setup workflow
- API endpoint authentication requirements
- Session persistence and remember me functionality
- Next parameter redirect behavior

Total: ~1,200 lines of code added

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 11:23:46 -06:00
ee0c5a2c3c Phase 2 Step 3: Implement Background Job Queue
Implemented APScheduler integration for background scan execution,
enabling async job processing without blocking HTTP requests.

## Changes

### Background Jobs (web/jobs/)
- scan_job.py - Execute scans in background threads
  - execute_scan() with isolated database sessions
  - Comprehensive error handling and logging
  - Scan status lifecycle tracking
  - Timing and error message storage

### Scheduler Service (web/services/scheduler_service.py)
- SchedulerService class for job management
- APScheduler BackgroundScheduler integration
- ThreadPoolExecutor for concurrent jobs (max 3 workers)
- queue_scan() - Immediate job execution
- Job monitoring: list_jobs(), get_job_status()
- Graceful shutdown handling

### Flask Integration (web/app.py)
- init_scheduler() function
- Scheduler initialization in app factory
- Stored scheduler in app context (app.scheduler)

### Database Schema (migration 003)
- Added scan timing fields:
  - started_at - Scan execution start time
  - completed_at - Scan execution completion time
  - error_message - Error details for failed scans

### Service Layer Updates (web/services/scan_service.py)
- trigger_scan() accepts scheduler parameter
- Queues background jobs after creating scan record
- get_scan_status() includes new timing and error fields
- _save_scan_to_db() sets completed_at timestamp

### API Updates (web/api/scans.py)
- POST /api/scans passes scheduler to trigger_scan()
- Scans now execute in background automatically

### Model Updates (web/models.py)
- Added started_at, completed_at, error_message to Scan model

### Testing (tests/test_background_jobs.py)
- 13 unit tests for background job execution
- Scheduler initialization and configuration tests
- Job queuing and status tracking tests
- Scan timing field tests
- Error handling and storage tests
- Integration test for full workflow (skipped by default)

## Features

- Async scan execution without blocking HTTP requests
- Concurrent scan support (configurable max workers)
- Isolated database sessions per background thread
- Scan lifecycle tracking: created → running → completed/failed
- Error messages captured and stored in database
- Job monitoring and management capabilities
- Graceful shutdown waits for running jobs

## Implementation Notes

- Scanner runs in subprocess from background thread
- Docker provides necessary privileges (--privileged, --network host)
- Each job gets isolated SQLAlchemy session (avoid locking)
- Job IDs follow pattern: scan_{scan_id}
- Background jobs survive across requests
- Failed jobs store error messages in database

## Documentation (docs/ai/PHASE2.md)
- Updated progress: 6/14 days complete (43%)
- Marked Step 3 as complete
- Added detailed implementation notes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 09:24:00 -06:00
6c4905d6c1 Phase 2 Step 2: Implement Scan API Endpoints
Implemented all 5 scan management endpoints with comprehensive error
handling, logging, and integration tests.

## Changes

### API Endpoints (web/api/scans.py)
- POST /api/scans - Trigger new scan with config file validation
- GET /api/scans - List scans with pagination and status filtering
- GET /api/scans/<id> - Retrieve scan details with all relationships
- DELETE /api/scans/<id> - Delete scan and associated files
- GET /api/scans/<id>/status - Poll scan status for long-running scans

### Features
- Comprehensive error handling (400, 404, 500)
- Structured logging with appropriate levels
- Input validation via validators
- Consistent JSON error format
- SQLAlchemy error handling with graceful degradation
- HTTP status codes following REST conventions

### Testing (tests/test_scan_api.py)
- 24 integration tests covering all endpoints
- Empty/populated scan lists
- Pagination with multiple pages
- Status filtering
- Error scenarios (invalid input, not found, etc.)
- Complete workflow integration test

### Test Infrastructure (tests/conftest.py)
- Flask app fixture with test database
- Flask test client fixture
- Database session fixture compatible with app context
- Sample scan fixture for testing

### Documentation (docs/ai/PHASE2.md)
- Updated progress: 4/14 days complete (29%)
- Marked Step 2 as complete
- Added implementation details and testing results

## Implementation Notes

- All endpoints use ScanService for business logic separation
- Scan triggering returns immediately; client polls status endpoint
- Background job execution will be added in Step 3
- Authentication will be added in Step 4

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 09:13:30 -06:00
986c0d3d17 Complete Phase 1: Foundation - Flask web application infrastructure
Implement complete database schema and Flask application structure for
SneakyScan web interface. This establishes the foundation for web-based
scan management, scheduling, and visualization.

Database & ORM:
- Add 11 SQLAlchemy models for comprehensive scan data storage
  (Scan, ScanSite, ScanIP, ScanPort, ScanService, ScanCertificate,
  ScanTLSVersion, Schedule, Alert, AlertRule, Setting)
- Configure Alembic migrations system with initial schema migration
- Add init_db.py script for database initialization and password setup
- Support both migration-based and direct table creation

Settings System:
- Implement SettingsManager with automatic encryption for sensitive values
- Add Fernet encryption for SMTP passwords and API tokens
- Implement PasswordManager with bcrypt password hashing (work factor 12)
- Initialize default settings for SMTP, authentication, and retention

Flask Application:
- Create Flask app factory pattern with scoped session management
- Add 4 API blueprints: scans, schedules, alerts, settings
- Implement functional Settings API (GET/PUT/DELETE endpoints)
- Add CORS support, error handlers, and request/response logging
- Configure development and production logging to file and console

Docker & Deployment:
- Update Dockerfile to install Flask dependencies
- Add docker-compose-web.yml for web application deployment
- Configure volume mounts for database, output, and logs persistence
- Expose port 5000 for Flask web server

Testing & Validation:
- Add validate_phase1.py script to verify all deliverables
- Validate directory structure, Python syntax, models, and endpoints
- All validation checks passing

Documentation:
- Add PHASE1_COMPLETE.md with comprehensive Phase 1 summary
- Update ROADMAP.md with Phase 1 completion status
- Update .gitignore to exclude database files and documentation

Files changed: 21 files
- New: web/ directory with complete Flask app structure
- New: migrations/ with Alembic configuration
- New: requirements-web.txt with Flask dependencies
- Modified: Dockerfile, ROADMAP.md, .gitignore
2025-11-13 23:59:23 -06:00