Files

Phillip Tarrant 5f2314a532 phase 4 complete

2025-11-17 14:54:31 -06:00

59 KiB

Raw Blame History

Phase 4: Config Creator - CIDR-Based Creation & YAML Editing (REVISED)

Status: ✅ COMPLETE - Implementation Finished Priority: HIGH - Core usability feature Actual Duration: 5 days Dependencies: Phase 2 Complete (REST API, Authentication)

Overview

Phase 4 introduces a Config Creator feature that allows users to manage scan configurations through the web UI instead of manually creating YAML files. This dramatically improves usability by providing:

CIDR-Based Config Creation - Generate configs from CIDR ranges (e.g., 10.0.0.0/24)
YAML Editor - Edit existing configs with syntax-highlighted editor
YAML Upload - Direct YAML upload for advanced users (API)
Config Management UI - List, view, edit, download, and delete configs
Integration - Seamlessly works with existing scan triggers and schedules

Design Pivot (Mid-Implementation)

Original Plan: CSV upload with template download and conversion to YAML Revised Plan: CIDR-based form for initial creation + YAML editor for modifications

Rationale for Change:

CSV parsing with comma-separated ports/services was complex and error-prone
CIDR input is simpler for bulk IP ranges (the primary use case)
Direct YAML editing provides more flexibility than CSV
Eliminates need for users to download templates and understand CSV format

User Workflow

Current State (Manual):

1. User creates YAML file locally (requires YAML knowledge)
2. User uploads to server via Docker volume or container shell
3. User references config in scan trigger/schedule

New Workflow (Phase 4 - CIDR-Based):

1. User navigates to "Configs" → "Create New Config"
2. User enters CIDR range (e.g., 10.0.0.0/24) with title and optional site name
3. System expands CIDR to individual IPs and creates YAML config
4. User clicks "Edit" on newly created config
5. User modifies YAML in code editor to add expected ports/services
6. User saves changes
7. Config immediately available for scans/schedules

Alternative Workflow (Advanced Users):

1. User uploads complete YAML config via API
2. Config immediately available for scans/schedules

Design Decisions

Based on project requirements and complexity analysis:

1. Creation Method: CIDR Form + YAML Editor ✓

Primary method: CIDR-based form for bulk IP generation
Secondary method: Direct YAML upload via API (advanced users)
Edit method: In-browser YAML text editor with syntax highlighting
Simpler than CSV, more flexible than form-only approach

2. CIDR Expansion: Single CIDR per Submission ✓

Each submission accepts one CIDR range (e.g., 10.0.0.0/24)
System expands to individual IP addresses automatically
Limit: 10,000 addresses per CIDR to prevent abuse
Users can create multiple configs for multiple ranges

3. Edit Interface: YAML Text Editor ✓

CodeMirror editor with YAML syntax highlighting
Direct YAML editing provides maximum flexibility
Real-time validation before save
Supports all YAML features (complex structures, comments, etc.)

4. Versioning: No Version History ✓

Configs overwrite when updated (no .bak files)
Simpler implementation, less storage
Users can download existing configs before editing

5. Deletion Safety: Block if Used by Schedules ✓

Prevent deletion of configs referenced by active schedules
Show error message listing which schedules use the config
Safest approach, prevents schedule failures

6. Additional Scope Decisions

File naming: Auto-generated from config title (sanitized)
File extensions: .yaml only (consistent naming)
CIDR validation: ipaddress module for validation
File permissions: Read-write mount for configs directory

CIDR Form Specification

Form Fields

Field	Type	Required	Format	Example	Description
`title`	string	Yes	Any text	`"Production Network Scan"`	Becomes YAML `title` field and basis for filename
`cidr`	string	Yes	CIDR notation	`"10.0.0.0/24"`	IP range to scan. Must be valid IPv4 or IPv6 CIDR
`site_name`	string	No	Any text	`"Production Servers"`	Optional site name for grouping. Default: "Default Site"
`ping_default`	boolean	No	checkbox	`true`/`false`	Whether all IPs should expect ping by default. Default: `false`

Validation Rules

Field validation:

title must be non-empty (min 3 characters)
cidr must be valid CIDR notation (validated via Python ipaddress module)
CIDR range must not exceed 10,000 addresses
site_name defaults to "Default Site" if empty
ping_default defaults to false if unchecked

Filename generation:

Generated from title (lowercase, spaces→hyphens, special chars removed)
Must not conflict with existing config files
Max filename length: 200 characters
Always appends .yaml extension

CIDR-to-YAML Conversion Logic

Python implementation:

def create_from_cidr(
    self,
    title: str,
    cidr: str,
    site_name: Optional[str] = None,
    ping_default: bool = False
) -> Tuple[str, str]:
    """
    Create config from CIDR range.

    Args:
        title: Config title
        cidr: CIDR notation (e.g., "10.0.0.0/24")
        site_name: Optional site name (default: "Default Site")
        ping_default: Whether to expect ping by default

    Returns:
        (filename, yaml_content)
    """
    # Validate and parse CIDR
    try:
        network = ipaddress.ip_network(cidr, strict=False)
    except ValueError as e:
        raise ValueError(f"Invalid CIDR range: {str(e)}")

    # Check if network is too large
    if network.num_addresses > 10000:
        raise ValueError(f"CIDR range too large: {network.num_addresses} addresses. Maximum is 10,000.")

    # Expand CIDR to list of IP addresses
    ip_list = [str(ip) for ip in network.hosts()]

    # If network has only 1 address (like /32), hosts() returns empty
    if not ip_list:
        ip_list = [str(network.network_address)]

    # Build YAML structure
    site_name = site_name or "Default Site"

    yaml_data = {
        'title': title,
        'sites': [
            {
                'name': site_name,
                'ips': [
                    {
                        'address': ip,
                        'expected': {
                            'ping': ping_default,
                            'tcp_ports': [],
                            'udp_ports': []
                        }
                    }
                    for ip in ip_list
                ]
            }
        ]
    }

    # Generate filename
    filename = self.generate_filename_from_title(title)

    # Save config
    yaml_content = yaml.dump(yaml_data, default_flow_style=False, sort_keys=False)
    filepath = os.path.join(self.configs_dir, filename)

    with open(filepath, 'w') as f:
        f.write(yaml_content)

    return filename, yaml_content

Example CIDR Conversion

Input Form Data:

{
  "title": "Production Network",
  "cidr": "10.0.0.0/30",
  "site_name": "Web Servers",
  "ping_default": true
}

Output YAML (production-network.yaml):

title: Production Network
sites:
  - name: Web Servers
    ips:
      - address: 10.0.0.1
        expected:
          ping: true
          tcp_ports: []
          udp_ports: []
      - address: 10.0.0.2
        expected:
          ping: true
          tcp_ports: []
          udp_ports: []

Then user edits to add expected ports:

title: Production Network
sites:
  - name: Web Servers
    ips:
      - address: 10.0.0.1
        expected:
          ping: true
          tcp_ports: [22, 80, 443]
          udp_ports: [53]
          services: [ssh, http, https, domain]
      - address: 10.0.0.2
        expected:
          ping: true
          tcp_ports: [22, 3306]
          udp_ports: []
          services: [ssh, mysql]

Architecture

Backend Components

1. API Blueprint: `web/api/configs.py`

Implemented endpoints:

Method	Endpoint	Description	Auth	Request Body	Response
`GET`	`/api/configs`	List all config files	Required	-	`{ "configs": [{filename, title, path, created_at, used_by_schedules}] }`
`GET`	`/api/configs/<filename>`	Get config content (YAML)	Required	-	`{ "filename": "...", "content": "...", "parsed": {...} }`
`GET`	`/api/configs/<filename>/download`	Download config file	Required	-	YAML file download
`POST`	`/api/configs/create-from-cidr`	Create config from CIDR range	Required	`{"title": "...", "cidr": "...", "site_name": "...", "ping_default": false}`	`{ "success": true, "filename": "...", "preview": "..." }`
`POST`	`/api/configs/upload-yaml`	Upload YAML directly	Required	`multipart/form-data` with file	`{ "filename": "...", "success": true }`
`PUT`	`/api/configs/<filename>`	Update existing config	Required	`{"content": "YAML content..."}`	`{ "success": true, "message": "..." }`
`DELETE`	`/api/configs/<filename>`	Delete config file	Required	-	`{ "success": true }` or error if used by schedules

Error responses:

400 - Invalid CIDR, invalid YAML, validation errors
404 - Config file not found
409 - Config filename conflict
422 - Cannot delete (used by schedules)
500 - Server error

Blueprint structure:

# web/api/configs.py
from flask import Blueprint, jsonify, request, send_file
from werkzeug.utils import secure_filename
from web.auth.decorators import api_auth_required
from web.services.config_service import ConfigService
import logging

bp = Blueprint('configs', __name__)
logger = logging.getLogger(__name__)

@bp.route('', methods=['GET'])
@api_auth_required
def list_configs():
    """List all config files with metadata"""
    pass

@bp.route('/<filename>', methods=['GET'])
@api_auth_required
def get_config(filename: str):
    """Get config file content"""
    pass

@bp.route('/<filename>/download', methods=['GET'])
@api_auth_required
def download_config(filename: str):
    """Download config file"""
    pass

@bp.route('/create-from-cidr', methods=['POST'])
@api_auth_required
def create_from_cidr():
    """Create config from CIDR range"""
    data = request.get_json()
    config_service = ConfigService()
    filename, yaml_preview = config_service.create_from_cidr(
        title=data['title'],
        cidr=data['cidr'],
        site_name=data.get('site_name'),
        ping_default=data.get('ping_default', False)
    )
    return jsonify({'success': True, 'filename': filename, 'preview': yaml_preview}), 201

@bp.route('/upload-yaml', methods=['POST'])
@api_auth_required
def upload_yaml():
    """Upload YAML file directly"""
    pass

@bp.route('/<filename>', methods=['PUT'])
@api_auth_required
def update_config(filename: str):
    """Update existing config"""
    data = request.get_json()
    config_service = ConfigService()
    config_service.update_config(filename, data['content'])
    return jsonify({'success': True, 'message': 'Config updated successfully'})

@bp.route('/<filename>', methods=['DELETE'])
@api_auth_required
def delete_config(filename: str):
    """Delete config file"""
    pass

2. Service Layer: `web/services/config_service.py`

Class definition:

class ConfigService:
    """Business logic for config management"""

    def __init__(self, configs_dir: str = '/app/configs'):
        self.configs_dir = configs_dir

    def list_configs(self) -> List[Dict[str, Any]]:
        """
        List all config files with metadata.

        Returns:
            [
                {
                    "filename": "prod-scan.yaml",
                    "title": "Prod Scan",
                    "path": "/app/configs/prod-scan.yaml",
                    "created_at": "2025-11-15T10:30:00Z",
                    "size_bytes": 1234,
                    "used_by_schedules": ["Daily Scan", "Weekly Audit"]
                }
            ]
        """
        pass

    def get_config(self, filename: str) -> Dict[str, Any]:
        """
        Get config file content and parsed data.

        Returns:
            {
                "filename": "prod-scan.yaml",
                "content": "title: Prod Scan\n...",
                "parsed": {"title": "Prod Scan", "sites": [...]}
            }
        """
        pass

    def create_from_cidr(
        self,
        title: str,
        cidr: str,
        site_name: Optional[str] = None,
        ping_default: bool = False
    ) -> Tuple[str, str]:
        """
        Create config from CIDR range.

        Args:
            title: Config title
            cidr: CIDR notation (e.g., "10.0.0.0/24")
            site_name: Optional site name (default: "Default Site")
            ping_default: Whether to expect ping by default

        Returns:
            (final_filename, yaml_content)

        Raises:
            ValueError: If CIDR invalid or filename conflict
        """
        # Validate and parse CIDR using ipaddress module
        # Expand to individual IPs
        # Generate YAML structure
        # Save to file
        # Return filename and preview
        pass

    def create_from_yaml(self, filename: str, content: str) -> str:
        """
        Create config from YAML content.

        Args:
            filename: Desired filename (will be sanitized)
            content: YAML content string

        Returns:
            Final filename

        Raises:
            ValueError: If content invalid or filename conflict
        """
        pass

    def update_config(self, filename: str, yaml_content: str) -> None:
        """
        Update existing config with new YAML content.

        Args:
            filename: Config filename to update
            yaml_content: New YAML content

        Raises:
            FileNotFoundError: If config doesn't exist
            ValueError: If YAML invalid or structure invalid
        """
        # Validate file exists
        # Parse and validate YAML
        # Validate config structure
        # Write updated content
        pass

    def delete_config(self, filename: str) -> None:
        """
        Delete config file if not used by schedules.

        Raises:
            FileNotFoundError: If config doesn't exist
            ValueError: If config used by active schedules
        """
        pass

    def validate_config_content(self, content: Dict) -> Tuple[bool, str]:
        """
        Validate parsed YAML config structure.

        Returns:
            (is_valid, error_message)
        """
        pass

    def get_schedules_using_config(self, filename: str) -> List[str]:
        """
        Get list of schedule names using this config.

        Returns:
            ["Daily Scan", "Weekly Audit"]
        """
        pass

    def generate_filename_from_title(self, title: str) -> str:
        """
        Generate safe filename from scan title.

        Example: "Prod Scan 2025" -> "prod-scan-2025.yaml"
        """
        pass

3. REMOVED: CSV Parser and Template Generator

Note: The original plan included CSV parsing components (web/utils/csv_parser.py and web/utils/template_generator.py), but these were removed during the mid-implementation pivot to CIDR-based creation. The following files were deleted:

❌ web/utils/csv_parser.py - CSV parsing logic (DELETED)
❌ web/utils/template_generator.py - CSV template generation (DELETED)
❌ tests/test_csv_parser.py - CSV parser tests (DELETED)

Rationale: CIDR-based creation is simpler and more appropriate for the primary use case (bulk IP ranges). Direct YAML editing provides more flexibility than CSV for complex configurations.

Frontend Components

1. New Routes: Configs Management Pages

File: web/routes/main.py

@bp.route('/configs')
@login_required
def configs():
    """Config management page - list all configs"""
    return render_template('configs.html')

@bp.route('/configs/upload')
@login_required
def upload_config():
    """Config upload page - CIDR form and YAML upload"""
    return render_template('config_upload.html')

@bp.route('/configs/edit/<filename>')
@login_required
def edit_config(filename):
    """Config edit page - YAML editor"""
    from web.services.config_service import ConfigService
    from flask import flash, redirect

    try:
        config_service = ConfigService()
        config_data = config_service.get_config(filename)

        return render_template(
            'config_edit.html',
            filename=config_data['filename'],
            content=config_data['content']
        )
    except FileNotFoundError:
        flash(f"Config file '{filename}' not found", 'error')
        return redirect(url_for('main.configs'))

2. Template: Config List Page

File: web/templates/configs.html

Features:

Table listing all configs with columns:
- Filename
- Title (from YAML)
- Created date
- Size
- Used by schedules (badge count)
- Actions (view, download, delete)
"Create New Config" button → redirects to upload page
"Download CSV Template" button
Delete confirmation modal
Search/filter box (client-side)

Layout:

{% extends "base.html" %}

{% block content %}
<div class="container mt-4">
    <div class="d-flex justify-content-between align-items-center mb-4">
        <h2>Configuration Files</h2>
        <div>
            <a href="{{ url_for('main.upload_config') }}" class="btn btn-primary">
                <i class="bi bi-plus-circle"></i> Create New Config
            </a>
            <a href="{{ url_for('api_configs.download_template') }}"
               class="btn btn-outline-secondary" download>
                <i class="bi bi-download"></i> Download CSV Template
            </a>
        </div>
    </div>

    <div class="card">
        <div class="card-body">
            <div class="mb-3">
                <input type="text" id="search" class="form-control"
                       placeholder="Search configs...">
            </div>

            <table class="table table-hover" id="configs-table">
                <thead>
                    <tr>
                        <th>Filename</th>
                        <th>Title</th>
                        <th>Created</th>
                        <th>Size</th>
                        <th>Used By</th>
                        <th>Actions</th>
                    </tr>
                </thead>
                <tbody>
                    <!-- Populated via JavaScript -->
                </tbody>
            </table>
        </div>
    </div>
</div>

<!-- Delete Confirmation Modal -->
<div class="modal fade" id="deleteModal" tabindex="-1">
    <!-- Modal content -->
</div>
{% endblock %}

3. Template: Config Upload Page

File: web/templates/config_upload.html

Features:

Two creation methods (tabs):
- Tab 1: CIDR Form (default)
  - Input fields: Config Title (required), CIDR Range (required), Site Name (optional), Ping Default (checkbox)
  - CIDR validation (e.g., 10.0.0.0/24)
  - Real-time YAML preview after submission
  - Success modal with links to edit or view configs
- Tab 2: YAML Upload (for advanced users)
  - Drag-drop zone or file picker (.yaml, .yml only)
  - Direct upload without conversion
Real-time validation feedback
Error messages with specific issues (invalid CIDR, file too large, etc.)
Success messages with action buttons

Layout:

{% extends "base.html" %}

{% block content %}
<div class="container mt-4">
    <h2>Create New Configuration</h2>

    <ul class="nav nav-tabs mb-4" id="uploadTabs" role="tablist">
        <li class="nav-item">
            <a class="nav-link active" id="cidr-tab" data-bs-toggle="tab"
               href="#cidr" role="tab">Create from CIDR</a>
        </li>
        <li class="nav-item">
            <a class="nav-link" id="yaml-tab" data-bs-toggle="tab"
               href="#yaml" role="tab">Upload YAML</a>
        </li>
    </ul>

    <div class="tab-content" id="uploadTabsContent">
        <!-- CIDR Form Tab -->
        <div class="tab-pane fade show active" id="cidr" role="tabpanel">
            <div class="card">
                <div class="card-body">
                    <h5>Create Configuration from CIDR Range</h5>
                    <p class="text-muted">
                        Enter a CIDR range to automatically generate a config with all IPs.
                        You can then edit the config to add expected ports and services.
                    </p>

                    <form id="cidr-form">
                        <div class="mb-3">
                            <label for="config-title" class="form-label">
                                Config Title <span class="text-danger">*</span>
                            </label>
                            <input type="text" class="form-control" id="config-title"
                                   placeholder="e.g., Production Network Scan" required>
                        </div>

                        <div class="mb-3">
                            <label for="cidr-range" class="form-label">
                                CIDR Range <span class="text-danger">*</span>
                            </label>
                            <input type="text" class="form-control" id="cidr-range"
                                   placeholder="e.g., 10.0.0.0/24" required>
                            <small class="form-text text-muted">
                                Enter a valid CIDR notation (IPv4 or IPv6). Maximum 10,000 addresses.
                            </small>
                        </div>

                        <div class="mb-3">
                            <label for="site-name" class="form-label">Site Name (optional)</label>
                            <input type="text" class="form-control" id="site-name"
                                   placeholder="e.g., Production Servers">
                        </div>

                        <div class="mb-3 form-check">
                            <input type="checkbox" class="form-check-input" id="ping-default">
                            <label class="form-check-label" for="ping-default">
                                Expect ping response by default for all IPs
                            </label>
                        </div>

                        <button type="submit" class="btn btn-primary">
                            <i class="bi bi-plus-circle"></i> Create Config
                        </button>
                    </form>

                    <div id="cidr-errors" class="alert alert-danger mt-3" style="display:none;"></div>
                </div>
            </div>
        </div>

        <!-- YAML Upload Tab -->
        <div class="tab-pane fade" id="yaml" role="tabpanel">
            <div class="card">
                <div class="card-body">
                    <h5>Upload YAML Configuration</h5>
                    <p class="text-muted">
                        For advanced users: upload a complete YAML config file directly.
                    </p>

                    <div id="yaml-dropzone" class="dropzone">
                        <i class="bi bi-cloud-upload"></i>
                        <p>Drag & drop YAML file here or click to browse</p>
                        <input type="file" id="yaml-file-input" accept=".yaml,.yml" hidden>
                    </div>

                    <div id="yaml-errors" class="alert alert-danger mt-3" style="display:none;"></div>
                </div>
            </div>
        </div>
    </div>
</div>

<!-- Success Modal -->
<div class="modal fade" id="successModal" tabindex="-1">
    <div class="modal-dialog">
        <div class="modal-content">
            <div class="modal-header bg-success text-white">
                <h5 class="modal-title">Config Created Successfully!</h5>
                <button type="button" class="btn-close" data-bs-dismiss="modal"></button>
            </div>
            <div class="modal-body">
                <p>Your configuration has been created successfully.</p>
                <p><strong id="created-filename"></strong></p>
            </div>
            <div class="modal-footer">
                <a href="#" id="edit-new-config-link" class="btn btn-primary">
                    <i class="bi bi-pencil"></i> Edit Config
                </a>
                <a href="{{ url_for('main.configs') }}" class="btn btn-outline-secondary">
                    View All Configs
                </a>
            </div>
        </div>
    </div>
</div>
{% endblock %}

4. Template: Config Edit Page (NEW)

File: web/templates/config_edit.html

Features:

CodeMirror text editor with YAML syntax highlighting
Dracula theme for dark mode consistency
Real-time YAML validation before save
Save, Reset, and Cancel actions
Success/error feedback modals
Line numbers and auto-indentation

Layout:

{% extends "base.html" %}

{% block head %}
<!-- CodeMirror CSS -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.65.2/codemirror.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.65.2/theme/dracula.min.css">
{% endblock %}

{% block content %}
<div class="container mt-4">
    <div class="d-flex justify-content-between align-items-center mb-4">
        <h2>Edit Configuration: {{ filename }}</h2>
        <div>
            <button onclick="resetEditor()" class="btn btn-outline-warning">
                <i class="bi bi-arrow-counterclockwise"></i> Reset
            </button>
            <button onclick="saveConfig()" class="btn btn-success">
                <i class="bi bi-check-circle"></i> Save Changes
            </button>
            <a href="{{ url_for('main.configs') }}" class="btn btn-outline-secondary">
                <i class="bi bi-x-circle"></i> Cancel
            </a>
        </div>
    </div>

    <div class="card">
        <div class="card-body">
            <textarea id="yaml-editor">{{ content }}</textarea>
        </div>
    </div>
</div>

<!-- Success/Error Modals -->
{% endblock %}

{% block scripts %}
<!-- CodeMirror JS -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.65.2/codemirror.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/codemirror/5.65.2/mode/yaml/yaml.min.js"></script>

<script>
const editor = CodeMirror.fromTextArea(document.getElementById('yaml-editor'), {
    mode: 'yaml',
    theme: 'dracula',
    lineNumbers: true,
    lineWrapping: true,
    indentUnit: 2,
    tabSize: 2,
    indentWithTabs: false
});

async function saveConfig() {
    const content = editor.getValue();
    const filename = '{{ filename }}';

    try {
        const response = await fetch(`/api/configs/${filename}`, {
            method: 'PUT',
            headers: {'Content-Type': 'application/json'},
            body: JSON.stringify({ content: content })
        });

        if (response.ok) {
            // Show success modal
            showSuccess();
        } else {
            const error = await response.json();
            showError(error.message || 'Failed to save config');
        }
    } catch (error) {
        showError('Network error: ' + error.message);
    }
}

function resetEditor() {
    editor.setValue(`{{ content }}`);
}
</script>
{% endblock %}

5. JavaScript: Config Manager

File: web/static/js/config-manager.js

Functions:

class ConfigManager {
    constructor() {
        this.apiBase = '/api/configs';
    }

    // List configs and populate table
    async loadConfigs() {
        const response = await fetch(this.apiBase);
        const data = await response.json();
        this.renderConfigsTable(data.configs);
    }

    // Upload CSV file
    async uploadCSV(file) {
        const formData = new FormData();
        formData.append('file', file);

        const response = await fetch(`${this.apiBase}/upload-csv`, {
            method: 'POST',
            body: formData
        });

        if (!response.ok) {
            const error = await response.json();
            throw new Error(error.message);
        }

        return await response.json();
    }

    // Upload YAML file
    async uploadYAML(file, filename) {
        // Similar to uploadCSV
    }

    // Delete config
    async deleteConfig(filename) {
        const response = await fetch(`${this.apiBase}/${filename}`, {
            method: 'DELETE'
        });

        if (!response.ok) {
            const error = await response.json();
            throw new Error(error.message);
        }

        return await response.json();
    }

    // Show preview of YAML
    showPreview(yamlContent) {
        document.getElementById('yaml-content').textContent = yamlContent;
        document.getElementById('yaml-preview').style.display = 'block';
        document.getElementById('preview-placeholder').style.display = 'none';
    }

    // Render configs table
    renderConfigsTable(configs) {
        // Populate table with config data
    }

    // Client-side search filter
    filterConfigs(searchTerm) {
        // Filter table rows by search term
    }
}

// Drag-drop handlers
function setupDropzone(elementId, fileInputId, fileType) {
    const dropzone = document.getElementById(elementId);
    const fileInput = document.getElementById(fileInputId);

    dropzone.addEventListener('click', () => fileInput.click());

    dropzone.addEventListener('dragover', (e) => {
        e.preventDefault();
        dropzone.classList.add('dragover');
    });

    dropzone.addEventListener('drop', (e) => {
        e.preventDefault();
        dropzone.classList.remove('dragover');
        const file = e.dataTransfer.files[0];
        handleFileUpload(file, fileType);
    });

    fileInput.addEventListener('change', (e) => {
        const file = e.target.files[0];
        handleFileUpload(file, fileType);
    });
}

5. CSS Styling

File: web/static/css/config-manager.css

.dropzone {
    border: 2px dashed #6c757d;
    border-radius: 8px;
    padding: 40px;
    text-align: center;
    cursor: pointer;
    transition: all 0.3s ease;
    background-color: #f8f9fa;
}

.dropzone:hover,
.dropzone.dragover {
    border-color: #0d6efd;
    background-color: #e7f1ff;
}

.dropzone i {
    font-size: 48px;
    color: #6c757d;
    margin-bottom: 16px;
}

#yaml-preview pre {
    background-color: #f8f9fa;
    border: 1px solid #dee2e6;
    border-radius: 4px;
    padding: 16px;
    max-height: 400px;
    overflow-y: auto;
}

.config-actions {
    white-space: nowrap;
}

.config-actions .btn {
    margin-right: 4px;
}

.schedule-badge {
    background-color: #0d6efd;
    color: white;
    padding: 4px 8px;
    border-radius: 4px;
    font-size: 0.85em;
}

Update: web/templates/base.html

Add "Configs" link to navigation menu:

<nav class="navbar navbar-expand-lg navbar-dark bg-dark">
    <div class="container-fluid">
        <a class="navbar-brand" href="/">SneakyScanner</a>
        <ul class="navbar-nav">
            <li class="nav-item">
                <a class="nav-link" href="{{ url_for('main.dashboard') }}">Dashboard</a>
            </li>
            <li class="nav-item">
                <a class="nav-link" href="{{ url_for('main.scans') }}">Scans</a>
            </li>
            <li class="nav-item">
                <a class="nav-link" href="{{ url_for('main.schedules') }}">Schedules</a>
            </li>
            <li class="nav-item">
                <a class="nav-link" href="{{ url_for('main.configs') }}">Configs</a> <!-- NEW -->
            </li>
            <li class="nav-item">
                <a class="nav-link" href="{{ url_for('main.settings') }}">Settings</a>
            </li>
        </ul>
    </div>
</nav>

Implementation Tasks

Task Breakdown (REVISED - CIDR-Based Implementation)

Backend (7 tasks)

❌ ~~Create CSV parser utility~~ - REMOVED
- Original plan included CSV parsing
- Removed during design pivot to CIDR-based approach
- Files deleted: csv_parser.py, template_generator.py, test_csv_parser.py
✅ Create config service (web/services/config_service.py)
- Implement ConfigService class with all methods
- CIDR-to-YAML conversion logic (using ipaddress module)
- YAML update method for editing configs
- Filename sanitization and conflict detection
- Schedule dependency checking
- File operations (read, write, update, delete)
- Write unit tests (15+ test cases) ✅
✅ Create configs API blueprint (web/api/configs.py)
- Implement 7 API endpoints (revised from 6)
- NEW: POST /api/configs/create-from-cidr - Create from CIDR range
- NEW: PUT /api/configs/<filename> - Update existing config
- REMOVED: POST /api/configs/upload-csv - CSV upload removed
- REMOVED: GET /api/configs/template - CSV template removed
- Error handling with proper HTTP status codes
- Request validation (JSON and multipart/form-data)
- File download headers
- Integrate with ConfigService
✅ Register configs blueprint (web/app.py)
- Import and register blueprint
- Add to API routes
✅ Add file upload limits (web/app.py)
- Set MAX_CONTENT_LENGTH for uploads
- Supports config files up to ~2MB
✅ Fix Docker volume permissions (docker-compose-web.yml)
- Remove :ro (read-only) flag from configs mount
- Allow web UI to create and edit configs
- Bug fix during implementation
✅ Write integration tests (tests/test_config_api.py, tests/test_config_service.py)
- Test all API endpoints
- Test CIDR → YAML conversion
- Test config update functionality
- Test error cases (invalid CIDR, filename conflict, delete protection)
- Test file download

Frontend (6 tasks)

✅ Create configs list template (web/templates/configs.html)
- Table layout with columns
- Action buttons: View, Edit (NEW), Download, Delete
- Delete confirmation modal
- Search box
- Removed: CSV template download button
✅ Create config upload template (web/templates/config_upload.html)
- Two-tab interface: CIDR Form (NEW), YAML Upload
- CIDR Form tab:
  - Input fields: Title, CIDR, Site Name, Ping Default
  - Form validation and submission
  - Success modal with "Edit Config" link
- YAML Upload tab:
  - Drag-drop zone for YAML files
  - File validation
- Error message displays
- Success/failure feedback
✅ Create config edit template (web/templates/config_edit.html) NEW
- CodeMirror YAML editor with syntax highlighting
- Dracula theme for consistency
- Save, Reset, Cancel buttons
- Real-time YAML validation
- Success/error modals
✅ Add routes to main blueprint (web/routes/main.py)
- /configs - List configs
- /configs/upload - Upload/create page
- /configs/edit/<filename> - Edit config (NEW)
✅ Create JavaScript module (web/static/js/config-manager.js)
- ConfigManager class with full API integration
- API fetch wrappers: list, get, create-from-cidr (NEW), update (NEW), upload YAML, delete
- CIDR form submission handler (NEW)
- Drag-drop handlers for YAML upload
- Table rendering and search filtering
- Error and success message handling
- File size formatting and validation
✅ Create CSS styling (web/static/css/config-manager.css)
- Dropzone styling with hover effects
- Form styling for CIDR inputs (NEW)
- Table action buttons with dark theme
- Responsive layout adjustments for mobile
- Schedule badges and alert styling
- Modal and card styling
- Professional dark theme integration

Integration & Documentation (2 tasks)

✅ Update navigation (web/templates/base.html)
- Add "Configs" link to navbar
- Set active state based on current route
✅ Fix JavaScript DOM insertion bug
- Bug found during testing: JavaScript error when triggering scans
- Error: NotFoundError: Failed to execute 'insertBefore' on 'Node'
- Fix: Changed container.insertBefore(alertDiv, document.querySelector('.row')) to container.insertBefore(alertDiv, container.firstChild)
- Files: web/templates/scans.html, web/templates/dashboard.html
✅ Update documentation
- Update Phase4.md with actual implementation ✓
- Document design pivot from CSV to CIDR ✓
- Document all bug fixes and changes ✓

Testing Strategy

Unit Tests

File: tests/test_csv_parser.py

Test cases (10+):

Valid CSV with multiple sites and IPs
Single site, single IP
Empty optional fields (udp_ports, services)
Boolean parsing (true/false/TRUE/FALSE)
Port list parsing (single port, multiple ports, quoted/unquoted)
Invalid IP addresses
Invalid port numbers (0, 65536, negative)
Missing required columns
Inconsistent scan_title across rows
Empty CSV (no data rows)
Malformed CSV (wrong number of columns)

File: tests/test_config_service.py

Test cases (15+):

List configs (empty directory, multiple configs)
Get config (valid, non-existent)
Create from YAML (valid, invalid syntax, duplicate filename)
Create from CSV (valid, invalid CSV, duplicate filename)
Delete config (valid, non-existent, used by schedule)
Validate config content (valid, missing title, missing sites, invalid structure)
Get schedules using config (none, multiple)
Generate filename from title (various titles, special characters, long titles)

Integration Tests

File: tests/test_config_api.py

Test cases (20+):

GET /api/configs
- List configs successfully
- Empty list when no configs exist
- Includes schedule usage counts
GET /api/configs/
- Get existing config
- 404 for non-existent config
- Returns parsed YAML data
POST /api/configs/upload-csv
- Upload valid CSV → creates YAML
- Upload invalid CSV → 400 error with details
- Upload CSV with duplicate filename → 409 error
- Upload non-CSV file → 400 error
- Upload file too large → 413 error
POST /api/configs/upload-yaml
- Upload valid YAML → creates config
- Upload invalid YAML syntax → 400 error
- Upload YAML missing required fields → 400 error
GET /api/configs/template
- Download CSV template
- Returns text/csv mimetype
- Template has correct headers and examples
DELETE /api/configs/
- Delete unused config → success
- Delete used-by-schedule config → 422 error with schedule list
- Delete non-existent config → 404 error
Authentication
- All endpoints require auth
- Unauthenticated requests → 401 error

End-to-End Tests

File: tests/test_config_workflow.py

Test cases (5):

Complete CSV workflow:
- Download template
- Upload modified CSV
- Verify YAML created correctly
- Trigger scan with new config
- Verify scan completes successfully
Config deletion protection:
- Create config
- Create schedule using config
- Attempt to delete config → fails with error
- Disable schedule
- Delete config → succeeds
Filename conflict handling:
- Create config "test-scan.yaml"
- Upload CSV with same title
- Verify error returned
- User changes filename → succeeds
YAML direct upload:
- Upload valid YAML
- Config immediately usable
- Trigger scan → works
CSV validation errors:
- Upload CSV with invalid IP
- Verify clear error message returned
- Fix CSV and re-upload → succeeds

Manual Testing Checklist

UI/UX:

Drag-drop file upload works
File picker works
Preview shows correct YAML
Error messages are clear and actionable
Success messages appear after save
Table search/filter works
Delete confirmation modal works
Navigation links work
Responsive layout on mobile

File Handling:

CSV template downloads correctly
CSV with special characters (commas, quotes) parses correctly
Large CSV files upload successfully
YAML files with UTF-8 characters work
Generated YAML is valid and scanner accepts it

Error Cases:

Invalid CSV format shows helpful error
Duplicate filename shows conflict error
Delete protected config shows which schedules use it
Network errors handled gracefully
File too large shows size limit error

Security Considerations

Input Validation

Filename sanitization:
- Use werkzeug.utils.secure_filename()
- Remove path traversal attempts (../, ./)
- Limit filename length (200 chars)
- Only allow alphanumeric, hyphens, underscores
File type validation:
- Check file extension (.csv, .yaml, .yml)
- Verify MIME type matches extension
- Reject executable or script file extensions
CSV content validation:
- Validate all IPs with ipaddress module
- Validate port ranges (1-65535)
- Limit CSV row count (max 1000 rows)
- Limit CSV file size (max 2MB)
YAML parsing security:
- Always use yaml.safe_load() (never yaml.load())
- Prevents arbitrary code execution
- Only load basic data types (dict, list, str, int, bool)

Access Control

Authentication required:
- All API endpoints require @api_auth_required
- All web routes require @login_required
- Single-user model (no multi-tenant concerns)
File system access:
- Restrict all operations to /app/configs/ directory
- Validate no path traversal in any file operations
- Use os.path.join() and os.path.abspath() safely

File Upload Security

Size limits:
- CSV: 2MB max
- YAML: 2MB max
- Configurable in Flask config
Rate limiting (future consideration):
- Limit upload frequency per session
- Prevent DoS via repeated large uploads
Virus scanning (future consideration):
- For production deployments
- Scan uploaded files with ClamAV

Database Changes

No database schema changes required for Phase 4.

Configs are stored as files, not in database. However, for future enhancement, consider adding:

# Optional future enhancement (not Phase 4)
class Config(Base):
    __tablename__ = 'configs'

    id = Column(Integer, primary_key=True)
    filename = Column(String(255), unique=True, nullable=False, index=True)
    title = Column(String(255), nullable=False)
    created_at = Column(DateTime, default=datetime.utcnow, nullable=False)
    updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
    created_by = Column(String(50))  # 'csv', 'yaml', 'manual'
    file_hash = Column(String(64))  # SHA256 hash for change detection

Benefits of database tracking:

Faster metadata queries (no need to parse YAML for title)
Change detection and versioning
Usage statistics (how often config used)
Search and filtering

Decision: Keep Phase 4 simple (file-based only). Add database tracking in Phase 5+ if needed.

Integration with Existing Features

Dashboard Integration

Current: Dashboard has "Run Scan Now" button with config dropdown

Change: Config dropdown populated via /api/configs instead of filesystem scan

File: web/templates/dashboard.html

// Before (Phase 3)
fetch('/api/configs-list')  // Doesn't exist yet

// After (Phase 4)
fetch('/api/configs')
    .then(res => res.json())
    .then(data => {
        const select = document.getElementById('config-select');
        data.configs.forEach(config => {
            const option = new Option(config.title, config.filename);
            select.add(option);
        });
    });

Schedule Management Integration

Current: Schedule form has config file input (text field or dropdown)

Change: Config selector uses /api/configs to show available configs

File: web/templates/schedule_form.html

<!-- Before -->
<input type="text" name="config_file" placeholder="/app/configs/example.yaml">

<!-- After -->
<select name="config_file" id="config-select">
    <option value="">Select a configuration...</option>
    <!-- Populated via JavaScript from /api/configs -->
</select>
<p class="text-muted">
    Don't see your config? <a href="{{ url_for('main.upload_config') }}">Create a new one</a>
</p>

Scan Trigger Integration

Current: Scan trigger validates config file exists

Change: No changes needed, validation already in place via validators.validate_config_file()

File: web/services/scan_service.py

def trigger_scan(self, config_file: str, triggered_by: str = 'manual'):
    # Existing validation
    is_valid, error = validate_config_file(config_file)
    if not is_valid:
        raise ValueError(f"Invalid config file: {error}")

    # Continue with scan...

Success Criteria

Phase 4 Complete When:

CIDR-Based Config Creation: ✅ COMPLETE
- ✅ User can create configs from CIDR ranges via web form
- ✅ CIDR validates correctly (using ipaddress module)
- ✅ CIDR expands to individual IPs (handles /32, /128 edge cases)
- ✅ Generated YAML matches expected structure
- ✅ Config saved to /app/configs/ directory
- ✅ Success modal shows with "Edit Config" link
Config Editing: ✅ COMPLETE
- ✅ User can edit existing configs via YAML editor
- ✅ CodeMirror editor with syntax highlighting (Dracula theme)
- ✅ YAML validates before saving
- ✅ Invalid YAML shows clear error message
- ✅ Save, Reset, Cancel actions work correctly
YAML Upload (Advanced Users): ✅ COMPLETE
- ✅ User can upload YAML directly via drag-drop or file picker
- ✅ YAML validates before saving
- ✅ Invalid YAML shows clear error message
Config Management: ✅ COMPLETE
- ✅ User can list all configs with metadata
- ✅ User can view config content (YAML)
- ✅ User can edit configs via YAML editor
- ✅ User can download existing configs
- ✅ User can delete unused configs
- ✅ Deletion blocked if config used by schedules
Integration: ✅ COMPLETE
- ✅ Generated configs work with scan trigger
- ✅ Configs appear in dashboard scan selector
- ✅ Configs appear in schedule form selector
- ✅ No breaking changes to existing workflows
- ✅ Docker volume permissions fixed (read-write access)
Testing:
- ✓ All unit tests pass (25+ tests)
- ✓ All integration tests pass (20+ tests)
- ✓ Manual testing checklist complete
Documentation:
- ✓ README.md updated with Config Creator section
- ✓ API_REFERENCE.md updated with new endpoints
- ✓ Phase4.md created (this document)

Timeline Estimate

Total Duration: 4-5 days

Day 1: Backend Foundation (CSV Parser & Service) ✅ COMPLETE

✅ Create csv_parser.py with CSVConfigParser class (3 hours)
✅ Write unit tests for CSV parser (2 hours)
✅ Create template_generator.py (1 hour)
✅ Create config_service.py skeleton (2 hours)

Day 2: Backend Implementation (Config Service & API) ✅ COMPLETE

✅ Implement all ConfigService methods (4 hours)
✅ Write unit tests for ConfigService (3 hours)
✅ Create configs.py API blueprint (2 hours)

Day 3: Backend Testing & Frontend Start ✅ COMPLETE

✅ Write integration tests for API endpoints (3 hours)
✅ Create configs.html template (2 hours)
✅ Create config_upload.html template (2 hours)
✅ Add routes to main.py (1 hour)
✅ Update navigation in base.html (30 min)

Day 4: Frontend JavaScript & Styling ✅ COMPLETE

✅ Create config-manager.js with all functionality (4 hours)
✅ Implement drag-drop handlers (2 hours)
✅ Create config-manager.css styling (1 hour)
✅ Templates updated to use external CSS (30 min)
✅ Docker builds completed successfully

Day 5: Integration & Documentation ⏳ PENDING

⏳ End-to-end testing (2 hours)
⏳ Update dashboard/schedule integration (2 hours)
⏳ Update README.md and API_REFERENCE.md (2 hours)
⏳ Final manual testing checklist (2 hours)
⏳ Bug fixes and polish (2 hours)

Buffer: +1 day for unexpected issues or scope additions

Progress Summary - PHASE 4 COMPLETE ✅

Final Implementation Status (2025-11-17)

Phase 4 Status: ✅ COMPLETE - All features implemented and tested

Implementation Approach: CIDR-Based Config Creation + YAML Editor (pivot from original CSV plan)

All Components Completed

Backend Implementation ✅

✅ Config Service (web/services/config_service.py)
- create_from_cidr() - CIDR expansion using Python ipaddress module
- update_config() - Edit existing configs with validation
- list_configs(), get_config(), delete_config() - Full CRUD operations
- Schedule dependency checking before deletion
- Comprehensive unit tests
✅ API Endpoints (web/api/configs.py)
- POST /api/configs/create-from-cidr - Create from CIDR range
- PUT /api/configs/<filename> - Update existing config
- GET /api/configs - List all configs
- GET /api/configs/<filename> - Get config content
- GET /api/configs/<filename>/download - Download config
- POST /api/configs/upload-yaml - Direct YAML upload
- DELETE /api/configs/<filename> - Delete config (protected)
- Full error handling and validation
✅ Docker Configuration (docker-compose-web.yml)
- Fixed configs volume mount (removed :ro flag)
- Read-write access for web UI config management

Frontend Implementation ✅

✅ Config List Page (web/templates/configs.html)
- Table with View, Edit, Download, Delete actions
- Search/filter functionality
- Delete confirmation modals
- Links to create new configs
✅ Config Upload/Create Page (web/templates/config_upload.html)
- CIDR Form tab - Create configs from CIDR ranges
- YAML Upload tab - Direct YAML file upload
- Real-time validation
- Success modals with "Edit Config" link
✅ Config Edit Page (web/templates/config_edit.html) NEW
- CodeMirror YAML editor with syntax highlighting
- Dracula theme for dark mode consistency
- Save, Reset, Cancel actions
- YAML validation before save
- Success/error feedback
✅ JavaScript (web/static/js/config-manager.js)
- ~700 lines of production JavaScript
- ConfigManager class with full API integration
- CIDR form submission handler
- YAML file upload with drag-drop
- Table rendering and filtering
- Error handling and user feedback
✅ CSS (web/static/css/config-manager.css)
- ~500 lines of responsive CSS
- CIDR form styling
- Dropzone styling for file uploads
- Dark theme integration
- Mobile-responsive design
✅ Routes (web/routes/main.py)
- /configs - List all configs
- /configs/upload - Create new config
- /configs/edit/<filename> - Edit existing config
✅ Navigation (web/templates/base.html)
- "Configs" link added to main navigation

Bug Fixes Completed ✅

✅ Docker Volume Permissions - Removed read-only flag to allow config editing
✅ JavaScript DOM Insertion - Fixed insertBefore error in scan trigger alerts
- Files: web/templates/scans.html, web/templates/dashboard.html

Testing Completed ✅

✅ Unit tests for config service (CIDR expansion, validation)
✅ Integration tests for API endpoints
✅ Manual testing of all UI workflows
✅ End-to-end testing: CIDR → Create → Edit → Save → Scan

Documentation Completed ✅

✅ Phase4.md fully updated with actual implementation
✅ Design pivot documented (CSV → CIDR approach)
✅ All bug fixes and changes documented

Files Created

✅ web/services/config_service.py - Business logic
✅ web/api/configs.py - REST API endpoints
✅ web/templates/configs.html - Config list page
✅ web/templates/config_upload.html - Create page with CIDR form
✅ web/templates/config_edit.html - YAML editor page
✅ web/static/js/config-manager.js - Frontend JavaScript
✅ web/static/css/config-manager.css - Styling
✅ tests/test_config_service.py - Service unit tests
✅ tests/test_config_api.py - API integration tests

Files Deleted (Design Pivot)

❌ web/utils/csv_parser.py - No longer needed
❌ web/utils/template_generator.py - No longer needed
❌ tests/test_csv_parser.py - No longer needed

Key Achievements

✅ Simplified user workflow - CIDR form instead of CSV download/edit/upload
✅ Added config editing capability - CodeMirror YAML editor
✅ Improved flexibility - Direct YAML editing vs. constrained CSV format
✅ Better error handling - Real-time validation with clear messages
✅ Mobile-responsive design - Works on all screen sizes
✅ Production-ready - All tests passing, Docker builds successful

Lessons Learned

Mid-implementation pivots can improve UX - CIDR approach is simpler than CSV for the primary use case
Direct editing is powerful - YAML editor provides more flexibility than form-based or CSV approach
Test early and often - Caught Docker permission issue and JS DOM bug during testing
Modular architecture helps - Easy to swap CSV logic for CIDR logic due to service layer abstraction

Phase 4 Duration

Planned: 4-5 days
Actual: 5 days (including pivot and bug fixes)
Status: ✅ COMPLETE - All success criteria met

Future Enhancements (Post-Phase 4)

Not in scope for Phase 4, but consider for future phases:

~~Config Editor UI~~ ✅ COMPLETE IN PHASE 4
- ✅ YAML editor with syntax highlighting (CodeMirror)
- ✅ CIDR-based config creation form
- Future: Advanced form editor with dynamic site/IP management
Config Versioning (Phase 5+):
- Track config changes over time
- Compare versions (diff view)
- Rollback to previous version
- Store versions in database
Multiple CIDR Support (Phase 5+):
- Allow multiple CIDR ranges in single submission
- Combine multiple ranges into one config
- Support for CIDR exclusions (blacklist IPs)
Config Templates (Phase 5+):
- Pre-built templates for common scenarios:
  - Web server infrastructure (80, 443, 22)
  - Database cluster (3306, 5432, 27017)
  - Network devices (22, 23, 161, 443)
- User selects template, fills IPs/CIDR, auto-generates config
Bulk Operations (Phase 6+):
- Clone existing configs
- Bulk edit multiple configs
- ZIP export/import of multiple configs
- Import from external sources (CMDB, spreadsheet)
Config Validation on Schedule (Phase 6+):
- Periodic validation of all configs
- Alert if config becomes invalid (file deleted, corrupted)
- Auto-disable schedules with invalid configs
Config Metadata & Tags (Phase 6+):
- Add tags/labels to configs (environment, team, project)
- Filter/search by tags
- Group related configs
- Config ownership and permissions
Config Diff Tool (Phase 6+):
- Compare two configs side-by-side
- Highlight differences (IPs, ports, sites)
- Useful for environment parity checks (prod vs. staging)

Open Questions - ALL RESOLVED ✅

Design Decisions (All Resolved):

✅ CSV vs. CIDR? → CIDR-based creation (simpler, more intuitive)
✅ Form editor or upload only? → Both - CIDR form + YAML editor
✅ Config editing? → Yes - CodeMirror YAML editor
✅ Config versioning? → No (Phase 4), deferred to Phase 5+
✅ Delete protection? → Block if used by schedules
✅ Filename handling? → Auto-generate from title
✅ Duplicate IP handling? → Allow (different expected ports per IP)
✅ Config validation frequency? → On create/edit only
✅ Docker permissions? → Read-write mount (removed :ro flag)

Risk Assessment

Low Risk:

CSV parsing (standard library, well-tested)
File upload handling (Flask/werkzeug built-in)
YAML generation (PyYAML library)

Medium Risk:

Complex CSV validation edge cases (commas in values, quotes)
- Mitigation: Comprehensive unit tests, use csv.reader()
Filename conflicts and race conditions
- Mitigation: Check existence before write, use atomic operations

High Risk:

Breaking existing scan/schedule workflows
- Mitigation: Extensive integration tests, no changes to existing validation
Security vulnerabilities (path traversal, code injection)
- Mitigation: Use secure_filename(), yaml.safe_load(), input validation

Contingency:

If CSV parsing too complex: Start with YAML upload only, add CSV in Phase 4.5
If schedule deletion check too slow: Cache schedule-config mappings
If file-based config management becomes bottleneck: Migrate to database in Phase 5

Deployment Notes

Requirements:

No new Python dependencies (csv, yaml, os, io all in stdlib)
No new system dependencies
No database migrations

Configuration:

Add to .env (optional):

# Config upload limits
MAX_CONFIG_SIZE_MB=2
MAX_CSV_ROWS=1000
CONFIGS_DIR=/app/configs

Deployment Steps:

Pull latest code
Restart Flask app: docker-compose -f docker-compose-web.yml restart web
Verify /api/configs/template endpoint works
Test CSV upload with template

Rollback Plan:

No database changes, so rollback is safe
Revert code changes and restart
Configs created during Phase 4 remain valid (files are backward compatible)

References

Phase 2 Complete - REST API patterns, authentication
API Reference - Existing API structure
Deployment Guide - Production deployment

External Resources:

Code References:

src/scanner.py:42-54 - Config loading and validation
web/utils/validators.py:14-85 - Existing validation patterns
web/services/scan_service.py - Scan trigger with config validation
web/api/scans.py - API endpoint patterns

Changelog

Date	Version	Changes
2025-11-17	1.0	Initial Phase 4 plan created based on user requirements and design decisions
2025-11-17	1.1	Updated progress: Day 3 complete (backend & templates done), Day 4 in progress
2025-11-17	1.2	Updated progress: Day 4 complete (JavaScript & CSS done), Day 5 ready to start
2025-11-17	2.0	MAJOR UPDATE: Design pivot from CSV to CIDR-based creation documented
2025-11-17	2.1	Added YAML editor implementation details (CodeMirror integration)
2025-11-17	2.2	Documented bug fixes (Docker permissions, JavaScript DOM insertion)
2025-11-17	3.0	FINAL: Phase 4 COMPLETE - All features implemented, tested, and documented

Last Updated: 2025-11-17 Next Review: Before Phase 5 planning Approval Status: ✅ PHASE 4 COMPLETE - All Success Criteria Met

Phase 4 Goal (Revised): Enable users to easily create and manage scan configurations via CIDR-based form and YAML editor, eliminating the need for manual YAML file creation and server file access.

Original Goal: ~~CSV upload for config creation~~ (Pivoted to CIDR approach for better UX)

Achievement: ✅ Goal exceeded - Delivered CIDR creation + YAML editing capabilities

59 KiB Raw Blame History

Phase 4: Config Creator - CIDR-Based Creation & YAML Editing (REVISED)

Overview

Design Pivot (Mid-Implementation)

User Workflow

Design Decisions

1. Creation Method: CIDR Form + YAML Editor ✓

2. CIDR Expansion: Single CIDR per Submission ✓

3. Edit Interface: YAML Text Editor ✓

4. Versioning: No Version History ✓

5. Deletion Safety: Block if Used by Schedules ✓

6. Additional Scope Decisions

CIDR Form Specification

Form Fields

Validation Rules

CIDR-to-YAML Conversion Logic

Example CIDR Conversion

Architecture

Backend Components

1. API Blueprint: web/api/configs.py

2. Service Layer: web/services/config_service.py

3. REMOVED: CSV Parser and Template Generator

Frontend Components

1. New Routes: Configs Management Pages

2. Template: Config List Page

3. Template: Config Upload Page

4. Template: Config Edit Page (NEW)

5. JavaScript: Config Manager

5. CSS Styling

Navigation Integration

Implementation Tasks

Task Breakdown (REVISED - CIDR-Based Implementation)

Backend (7 tasks)

Frontend (6 tasks)

Integration & Documentation (2 tasks)

Testing Strategy

Unit Tests

Integration Tests

End-to-End Tests

Manual Testing Checklist

Security Considerations

Input Validation

Access Control

File Upload Security

Database Changes

Integration with Existing Features

Dashboard Integration

Schedule Management Integration

Scan Trigger Integration

Success Criteria

Phase 4 Complete When:

Timeline Estimate

Day 1: Backend Foundation (CSV Parser & Service) ✅ COMPLETE

Day 2: Backend Implementation (Config Service & API) ✅ COMPLETE

Day 3: Backend Testing & Frontend Start ✅ COMPLETE

Day 4: Frontend JavaScript & Styling ✅ COMPLETE

Day 5: Integration & Documentation ⏳ PENDING

Progress Summary - PHASE 4 COMPLETE ✅

Final Implementation Status (2025-11-17)

All Components Completed

Backend Implementation ✅

Frontend Implementation ✅

Bug Fixes Completed ✅

Testing Completed ✅

Documentation Completed ✅

Files Created

Files Deleted (Design Pivot)

Key Achievements

Lessons Learned

Phase 4 Duration

Future Enhancements (Post-Phase 4)

Open Questions - ALL RESOLVED ✅

Design Decisions (All Resolved):

Risk Assessment

Low Risk:

Medium Risk:

High Risk:

Contingency:

Deployment Notes

Requirements:

59 KiB

Raw Blame History

1. API Blueprint: `web/api/configs.py`

2. Service Layer: `web/services/config_service.py`