Files
SneakyScan/web/api/scans.py
Phillip Tarrant 167ab803a6 Phase 2 Step 7: Implement Error Handling & Logging
Comprehensive error handling and logging system with production-ready
features for monitoring, debugging, and user experience.

Enhanced Logging System:
- Implemented RotatingFileHandler (10MB per file, 10 backups, 100MB total)
- Separate error log file for ERROR level messages with detailed tracebacks
- Structured logging with request IDs, timestamps, and module names
- RequestIDLogFilter for automatic request context injection
- Console logging in debug mode with simplified format

Request/Response Middleware:
- Request ID generation using UUID (8-character prefix for readability)
- Request timing with millisecond precision
- User authentication context in all logs
- Response duration tracking and headers (X-Request-ID, X-Request-Duration-Ms)
- Security headers: X-Content-Type-Options, X-Frame-Options, X-XSS-Protection

Database Error Handling:
- Enabled SQLite WAL mode for better concurrency with background jobs
- Busy timeout configuration (15 seconds) for lock handling
- Automatic rollback on request exceptions via teardown handler
- Dedicated SQLAlchemyError handler with explicit rollback
- Connection pooling with pre-ping validation

Comprehensive Error Handlers:
- Content negotiation: JSON responses for API, HTML for web requests
- Error handlers for 400, 401, 403, 404, 405, 500
- Database rollback in all error handlers
- Full exception logging with traceback for debugging

Custom Error Templates:
- Created web/templates/errors/ directory with 6 templates
- Dark theme matching application design (slate colors)
- User-friendly error messages with navigation
- Templates: 400, 401, 403, 404, 405, 500

Testing:
- Comprehensive test suite (320+ lines) in tests/test_error_handling.py
- Tests for JSON vs HTML error responses
- Request ID and duration header verification
- Security header validation
- Log rotation configuration tests
- Structured logging tests

Bug Fix:
- Fixed pagination bug in scans API endpoint
- Changed paginated_result.total_pages to paginated_result.pages
- Resolves AttributeError when listing scans

Files Added:
- tests/test_error_handling.py
- web/templates/errors/400.html
- web/templates/errors/401.html
- web/templates/errors/403.html
- web/templates/errors/404.html
- web/templates/errors/405.html
- web/templates/errors/500.html

Files Modified:
- web/app.py (logging, error handlers, request handlers, database config)
- web/api/scans.py (pagination bug fix)
- docs/ai/PHASE2.md (mark Step 7 complete, update progress to 86%)

Phase 2 Progress: 12/14 days complete (86%)
2025-11-14 12:19:07 -06:00

309 lines
9.0 KiB
Python

"""
Scans API blueprint.
Handles endpoints for triggering scans, listing scan history, and retrieving
scan results.
"""
import logging
from flask import Blueprint, current_app, jsonify, request
from sqlalchemy.exc import SQLAlchemyError
from web.auth.decorators import api_auth_required
from web.services.scan_service import ScanService
from web.utils.validators import validate_config_file
from web.utils.pagination import validate_page_params
bp = Blueprint('scans', __name__)
logger = logging.getLogger(__name__)
@bp.route('', methods=['GET'])
@api_auth_required
def list_scans():
"""
List all scans with pagination.
Query params:
page: Page number (default: 1)
per_page: Items per page (default: 20, max: 100)
status: Filter by status (running, completed, failed)
Returns:
JSON response with scans list and pagination info
"""
try:
# Get and validate query parameters
page = request.args.get('page', 1, type=int)
per_page = request.args.get('per_page', 20, type=int)
status_filter = request.args.get('status', None, type=str)
# Validate pagination params
page, per_page = validate_page_params(page, per_page)
# Get scans from service
scan_service = ScanService(current_app.db_session)
paginated_result = scan_service.list_scans(
page=page,
per_page=per_page,
status_filter=status_filter
)
logger.info(f"Listed scans: page={page}, per_page={per_page}, status={status_filter}, total={paginated_result.total}")
return jsonify({
'scans': paginated_result.items,
'total': paginated_result.total,
'page': paginated_result.page,
'per_page': paginated_result.per_page,
'total_pages': paginated_result.pages,
'has_prev': paginated_result.has_prev,
'has_next': paginated_result.has_next
})
except ValueError as e:
logger.warning(f"Invalid request parameters: {str(e)}")
return jsonify({
'error': 'Invalid request',
'message': str(e)
}), 400
except SQLAlchemyError as e:
logger.error(f"Database error listing scans: {str(e)}")
return jsonify({
'error': 'Database error',
'message': 'Failed to retrieve scans'
}), 500
except Exception as e:
logger.error(f"Unexpected error listing scans: {str(e)}", exc_info=True)
return jsonify({
'error': 'Internal server error',
'message': 'An unexpected error occurred'
}), 500
@bp.route('/<int:scan_id>', methods=['GET'])
@api_auth_required
def get_scan(scan_id):
"""
Get details for a specific scan.
Args:
scan_id: Scan ID
Returns:
JSON response with scan details
"""
try:
# Get scan from service
scan_service = ScanService(current_app.db_session)
scan = scan_service.get_scan(scan_id)
if not scan:
logger.warning(f"Scan not found: {scan_id}")
return jsonify({
'error': 'Not found',
'message': f'Scan with ID {scan_id} not found'
}), 404
logger.info(f"Retrieved scan details: {scan_id}")
return jsonify(scan)
except SQLAlchemyError as e:
logger.error(f"Database error retrieving scan {scan_id}: {str(e)}")
return jsonify({
'error': 'Database error',
'message': 'Failed to retrieve scan'
}), 500
except Exception as e:
logger.error(f"Unexpected error retrieving scan {scan_id}: {str(e)}", exc_info=True)
return jsonify({
'error': 'Internal server error',
'message': 'An unexpected error occurred'
}), 500
@bp.route('', methods=['POST'])
@api_auth_required
def trigger_scan():
"""
Trigger a new scan.
Request body:
config_file: Path to YAML config file
Returns:
JSON response with scan_id and status
"""
try:
# Get request data
data = request.get_json() or {}
config_file = data.get('config_file')
# Validate required fields
if not config_file:
logger.warning("Scan trigger request missing config_file")
return jsonify({
'error': 'Invalid request',
'message': 'config_file is required'
}), 400
# Trigger scan via service
scan_service = ScanService(current_app.db_session)
scan_id = scan_service.trigger_scan(
config_file=config_file,
triggered_by='api',
scheduler=current_app.scheduler
)
logger.info(f"Scan {scan_id} triggered via API: config={config_file}")
return jsonify({
'scan_id': scan_id,
'status': 'running',
'message': 'Scan queued successfully'
}), 201
except ValueError as e:
# Config file validation error
logger.warning(f"Invalid config file: {str(e)}")
return jsonify({
'error': 'Invalid request',
'message': str(e)
}), 400
except SQLAlchemyError as e:
logger.error(f"Database error triggering scan: {str(e)}")
return jsonify({
'error': 'Database error',
'message': 'Failed to create scan'
}), 500
except Exception as e:
logger.error(f"Unexpected error triggering scan: {str(e)}", exc_info=True)
return jsonify({
'error': 'Internal server error',
'message': 'An unexpected error occurred'
}), 500
@bp.route('/<int:scan_id>', methods=['DELETE'])
@api_auth_required
def delete_scan(scan_id):
"""
Delete a scan and its associated files.
Args:
scan_id: Scan ID to delete
Returns:
JSON response with deletion status
"""
try:
# Delete scan via service
scan_service = ScanService(current_app.db_session)
scan_service.delete_scan(scan_id)
logger.info(f"Scan {scan_id} deleted successfully")
return jsonify({
'scan_id': scan_id,
'message': 'Scan deleted successfully'
}), 200
except ValueError as e:
# Scan not found
logger.warning(f"Scan deletion failed: {str(e)}")
return jsonify({
'error': 'Not found',
'message': str(e)
}), 404
except SQLAlchemyError as e:
logger.error(f"Database error deleting scan {scan_id}: {str(e)}")
return jsonify({
'error': 'Database error',
'message': 'Failed to delete scan'
}), 500
except Exception as e:
logger.error(f"Unexpected error deleting scan {scan_id}: {str(e)}", exc_info=True)
return jsonify({
'error': 'Internal server error',
'message': 'An unexpected error occurred'
}), 500
@bp.route('/<int:scan_id>/status', methods=['GET'])
@api_auth_required
def get_scan_status(scan_id):
"""
Get current status of a running scan.
Args:
scan_id: Scan ID
Returns:
JSON response with scan status and progress
"""
try:
# Get scan status from service
scan_service = ScanService(current_app.db_session)
status = scan_service.get_scan_status(scan_id)
if not status:
logger.warning(f"Scan not found for status check: {scan_id}")
return jsonify({
'error': 'Not found',
'message': f'Scan with ID {scan_id} not found'
}), 404
logger.debug(f"Retrieved status for scan {scan_id}: {status['status']}")
return jsonify(status)
except SQLAlchemyError as e:
logger.error(f"Database error retrieving scan status {scan_id}: {str(e)}")
return jsonify({
'error': 'Database error',
'message': 'Failed to retrieve scan status'
}), 500
except Exception as e:
logger.error(f"Unexpected error retrieving scan status {scan_id}: {str(e)}", exc_info=True)
return jsonify({
'error': 'Internal server error',
'message': 'An unexpected error occurred'
}), 500
@bp.route('/<int:scan_id1>/compare/<int:scan_id2>', methods=['GET'])
@api_auth_required
def compare_scans(scan_id1, scan_id2):
"""
Compare two scans and show differences.
Args:
scan_id1: First scan ID
scan_id2: Second scan ID
Returns:
JSON response with comparison results
"""
# TODO: Implement in Phase 4
return jsonify({
'scan_id1': scan_id1,
'scan_id2': scan_id2,
'diff': {},
'message': 'Scan comparison endpoint - to be implemented in Phase 4'
})
# Health check endpoint
@bp.route('/health', methods=['GET'])
def health_check():
"""
Health check endpoint for monitoring.
Returns:
JSON response with API health status
"""
return jsonify({
'status': 'healthy',
'api': 'scans',
'version': '1.0.0-phase1'
})