Files
SneakyScan/web/utils/pagination.py
Phillip Tarrant d7c68a2be8 Phase 2 Step 1: Implement database and service layer
Complete the foundation for Phase 2 by implementing the service layer,
utilities, and comprehensive test suite. This establishes the core
business logic for scan management.

Service Layer:
- Add ScanService class with complete scan lifecycle management
  * trigger_scan() - Create scan record and prepare for execution
  * get_scan() - Retrieve scan with all related data (eager loading)
  * list_scans() - Paginated scan list with status filtering
  * delete_scan() - Remove scan from DB and delete all files
  * get_scan_status() - Poll current scan status and progress
  * _save_scan_to_db() - Persist scan results to database
  * _map_report_to_models() - Complex JSON-to-DB mapping logic

Database Mapping:
- Comprehensive mapping from scanner JSON output to normalized schema
- Handles nested relationships: sites → IPs → ports → services → certs → TLS
- Processes both TCP and UDP ports with expected/actual tracking
- Maps service detection results with HTTP/HTTPS information
- Stores SSL/TLS certificates with expiration tracking
- Records TLS version support and cipher suites
- Links screenshots to services

Utilities:
- Add pagination.py with PaginatedResult class
  * paginate() function for SQLAlchemy queries
  * validate_page_params() for input sanitization
  * Metadata: total, pages, has_prev, has_next, etc.

- Add validators.py with comprehensive validation functions
  * validate_config_file() - YAML structure and required fields
  * validate_scan_status() - Enum validation (running/completed/failed)
  * validate_scan_id() - Positive integer validation
  * validate_port() - Port range validation (1-65535)
  * validate_ip_address() - Basic IPv4 format validation
  * sanitize_filename() - Path traversal prevention

Database Migration:
- Add migration 002 for scan status index
- Optimizes queries filtering by scan status
- Timestamp index already exists from migration 001

Testing:
- Add pytest infrastructure with conftest.py
  * test_db fixture - Temporary SQLite database per test
  * sample_scan_report fixture - Realistic scanner output
  * sample_config_file fixture - Valid YAML config
  * sample_invalid_config_file fixture - For validation tests

- Add comprehensive test_scan_service.py (15 tests)
  * Test scan trigger with valid/invalid configs
  * Test scan retrieval (found/not found cases)
  * Test scan listing with pagination and filtering
  * Test scan deletion with cascade cleanup
  * Test scan status retrieval
  * Test database mapping from JSON to models
  * Test expected vs actual port flagging
  * Test certificate and TLS data mapping
  * Test full scan retrieval with all relationships
  * All tests passing

Files Added:
- web/services/__init__.py
- web/services/scan_service.py (545 lines)
- web/utils/pagination.py (153 lines)
- web/utils/validators.py (245 lines)
- migrations/versions/002_add_scan_indexes.py
- tests/__init__.py
- tests/conftest.py (142 lines)
- tests/test_scan_service.py (374 lines)

Next Steps (Step 2):
- Implement scan API endpoints in web/api/scans.py
- Add authentication decorators
- Integrate ScanService with API routes
- Test API endpoints with integration tests

Phase 2 Step 1 Complete ✓
2025-11-14 00:26:06 -06:00

159 lines
4.3 KiB
Python

"""
Pagination utilities for SneakyScanner web application.
Provides helper functions for paginating SQLAlchemy queries.
"""
from typing import Any, Dict, List
from sqlalchemy.orm import Query
class PaginatedResult:
"""Container for paginated query results."""
def __init__(self, items: List[Any], total: int, page: int, per_page: int):
"""
Initialize paginated result.
Args:
items: List of items for current page
total: Total number of items across all pages
page: Current page number (1-indexed)
per_page: Number of items per page
"""
self.items = items
self.total = total
self.page = page
self.per_page = per_page
@property
def pages(self) -> int:
"""Calculate total number of pages."""
if self.per_page == 0:
return 0
return (self.total + self.per_page - 1) // self.per_page
@property
def has_prev(self) -> bool:
"""Check if there is a previous page."""
return self.page > 1
@property
def has_next(self) -> bool:
"""Check if there is a next page."""
return self.page < self.pages
@property
def prev_page(self) -> int:
"""Get previous page number."""
return self.page - 1 if self.has_prev else None
@property
def next_page(self) -> int:
"""Get next page number."""
return self.page + 1 if self.has_next else None
def to_dict(self) -> Dict[str, Any]:
"""
Convert to dictionary for API responses.
Returns:
Dictionary with pagination metadata and items
"""
return {
'items': self.items,
'total': self.total,
'page': self.page,
'per_page': self.per_page,
'pages': self.pages,
'has_prev': self.has_prev,
'has_next': self.has_next,
'prev_page': self.prev_page,
'next_page': self.next_page,
}
def paginate(query: Query, page: int = 1, per_page: int = 20,
max_per_page: int = 100) -> PaginatedResult:
"""
Paginate a SQLAlchemy query.
Args:
query: SQLAlchemy query to paginate
page: Page number (1-indexed, default: 1)
per_page: Items per page (default: 20)
max_per_page: Maximum items per page (default: 100)
Returns:
PaginatedResult with items and pagination metadata
Examples:
>>> from web.models import Scan
>>> query = db.query(Scan).order_by(Scan.timestamp.desc())
>>> result = paginate(query, page=1, per_page=20)
>>> scans = result.items
>>> total_pages = result.pages
"""
# Validate and sanitize parameters
page = max(1, page) # Page must be at least 1
per_page = max(1, min(per_page, max_per_page)) # Clamp per_page
# Get total count
total = query.count()
# Calculate offset
offset = (page - 1) * per_page
# Execute query with limit and offset
items = query.limit(per_page).offset(offset).all()
return PaginatedResult(
items=items,
total=total,
page=page,
per_page=per_page
)
def validate_page_params(page: Any, per_page: Any,
max_per_page: int = 100) -> tuple[int, int]:
"""
Validate and sanitize pagination parameters.
Args:
page: Page number (any type, will be converted to int)
per_page: Items per page (any type, will be converted to int)
max_per_page: Maximum items per page (default: 100)
Returns:
Tuple of (validated_page, validated_per_page)
Examples:
>>> validate_page_params('2', '50')
(2, 50)
>>> validate_page_params(-1, 200)
(1, 100)
>>> validate_page_params(None, None)
(1, 20)
"""
# Default values
default_page = 1
default_per_page = 20
# Convert to int, use default if invalid
try:
page = int(page) if page is not None else default_page
except (ValueError, TypeError):
page = default_page
try:
per_page = int(per_page) if per_page is not None else default_per_page
except (ValueError, TypeError):
per_page = default_per_page
# Validate ranges
page = max(1, page)
per_page = max(1, min(per_page, max_per_page))
return page, per_page