feat: on-demand external script analysis + code viewer; refactor form analysis to rule engine

- API: add `POST /api/analyze_script` (app/blueprints/api.py)
  - Fetch one external script to artifacts, run rules, return findings + snippet
  - Uses new ExternalScriptFetcher (results_path aware) and job UUID
  - Returns: { ok, final_url, status_code, bytes, truncated, sha256, artifact_path, findings[], snippet, snippet_len }
  - TODO: document in openapi/openapi.yaml

- Fetcher: update `app/utils/external_fetch.py`
  - Constructed with `results_path` (UUID dir); writes to `<results_path>/scripts/fetched/<index>.js`
  - Loads settings via `get_settings()`, logs via std logging

- UI (results.html):
  - Move “Analyze external script” action into **Content Snippet** column for external rows
  - Clicking replaces button with `<details>` snippet, shows rule matches, and adds “open in viewer” link
  - Robust fetch handler (checks JSON, shows errors); builds viewer URL from absolute artifact path

- Viewer:
  - New route: `GET /view/artifact/<run_uuid>/<path:filename>` (app/blueprints/ui.py)
  - New template: Monaco-based read-only code viewer (viewer.html)
  - Removes SRI on loader to avoid integrity block; loads file via `raw_url` and detects language by extension

- Forms:
  - Refactor `analyze_forms` to mirror scripts analysis:
    - Uses rule engine (`category == "form"`) across regex/function rules
    - Emits rows only when matches exist
    - Includes `content_snippet`, `action`, `method`, `inputs`, `rules`
  - Replace legacy plumbing (`flagged`, `flag_reasons`, `status`) in output
  - Normalize form function rules to canonical returns `(bool, Optional[str])`:
    - `form_action_missing`
    - `form_http_on_https_page`
    - `form_submits_to_different_host`
    - Add minor hardening (lowercasing hosts, no-op actions, clearer reasons)

- CSS: add `.forms-table` to mirror `.scripts-table` (5 columns)
  - Fixed table layout, widths per column, chip/snippet styling, responsive tweaks

- Misc:
  - Fix “working outside app context” issue by avoiding `current_app` at import time (left storage logic inside routes)
  - Add “View Source” link to open page source in viewer

Refs:
- Roadmap: mark “Source code viewer” done; keep TODO to add `/api/analyze_script` to OpenAPI
This commit is contained in:
2025-08-21 15:32:24 -05:00
parent 05cf23ad67
commit 3a24b392f2
15 changed files with 1192 additions and 218 deletions

111
app/templates/viewer.html Normal file
View File

@@ -0,0 +1,111 @@
{% extends "base.html" %}
{% block content %}
<div style="max-width:1100px;margin:0 auto;padding:1rem 1.25rem;">
<header style="display:flex;align-items:center;justify-content:space-between;gap:1rem;flex-wrap:wrap;">
<div>
<h2 style="margin:0;font-size:1.1rem;">Code Viewer</h2>
<div class="text-dim" style="font-size:0.9rem;">
<strong>File:</strong> <span id="fileName">{{ filename }}</span>
</div>
</div>
<div style="display:flex;gap:.5rem;align-items:center;">
<button id="copyBtn" class="btn btn-sm">Copy</button>
<button id="wrapBtn" class="btn btn-sm">Toggle wrap</button>
<a id="openRaw" class="btn btn-sm" href="{{ raw_url }}" target="_blank" rel="noopener">Open raw</a>
<a id="downloadRaw" class="btn btn-sm" href="{{ raw_url }}" download>Download</a>
</div>
</header>
<div id="viewerStatus" class="text-dim" style="margin:.5rem 0 .75rem;"></div>
<div id="editor" style="height:72vh;border:1px solid #1f2a36;border-radius:8px;"></div>
</div>
<!-- Monaco AMD loader (no integrity to avoid mismatch) -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/monaco-editor/0.49.0/min/vs/loader.min.js"
crossorigin="anonymous" referrerpolicy="no-referrer"></script>
<script>
(function () {
const RAW_URL = "{{ raw_url }}";
const FILENAME = "{{ filename }}";
const LANGUAGE = "{{ language|default('', true) }}";
const statusEl = document.getElementById('viewerStatus');
function extToLang(name) {
if (!name) return 'plaintext';
const m = name.toLowerCase().match(/\.([a-z0-9]+)$/);
const ext = m ? m[1] : '';
const map = {js:'javascript',mjs:'javascript',cjs:'javascript',ts:'typescript',json:'json',
html:'html',htm:'html',css:'css',py:'python',sh:'shell',bash:'shell',
yml:'yaml',yaml:'yaml',md:'markdown',txt:'plaintext',log:'plaintext'};
return map[ext] || 'plaintext';
}
// Wait until the AMD loader has defined window.require
function waitForRequire(msLeft = 5000) {
return new Promise((resolve, reject) => {
const t0 = performance.now();
(function poll() {
if (window.require && typeof window.require === 'function') return resolve();
if (performance.now() - t0 > msLeft) return reject(new Error('Monaco loader not available'));
setTimeout(poll, 25);
})();
});
}
function configureMonaco() {
// Point AMD loader at the CDN
require.config({ paths: { 'vs': 'https://cdnjs.cloudflare.com/ajax/libs/monaco-editor/0.49.0/min/vs' } });
// Worker bootstrap
window.MonacoEnvironment = {
getWorkerUrl: function () {
const base = 'https://cdnjs.cloudflare.com/ajax/libs/monaco-editor/0.49.0/min/';
const code = "self.MonacoEnvironment={baseUrl:'" + base + "'};importScripts('" + base + "vs/base/worker/workerMain.js');";
return 'data:text/javascript;charset=utf-8,' + encodeURIComponent(code);
}
};
}
async function main() {
try {
statusEl.textContent = 'Loading file…';
await waitForRequire();
configureMonaco();
const resp = await fetch(RAW_URL, { cache: 'no-store' });
const text = await resp.text();
require(['vs/editor/editor.main'], function () {
const editor = monaco.editor.create(document.getElementById('editor'), {
value: text,
language: LANGUAGE || extToLang(FILENAME),
readOnly: true,
automaticLayout: true,
wordWrap: 'on',
minimap: { enabled: false },
scrollBeyondLastLine: false,
theme: 'vs-dark'
});
// Buttons
document.getElementById('copyBtn')?.addEventListener('click', async () => {
try { await navigator.clipboard.writeText(editor.getValue()); statusEl.textContent = 'Copied.'; }
catch (e) { statusEl.textContent = 'Copy failed: ' + e; }
});
document.getElementById('wrapBtn')?.addEventListener('click', () => {
const opts = editor.getRawOptions();
editor.updateOptions({ wordWrap: opts.wordWrap === 'on' ? 'off' : 'on' });
});
statusEl.textContent = (resp.ok ? '' : `Warning: HTTP ${resp.status}`) + (text.length ? '' : ' (empty file)');
});
} catch (err) {
statusEl.textContent = 'Viewer error: ' + err.message;
}
}
main();
})();
</script>
{% endblock %}