ptarrant/SneakyScope
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: on-demand external script analysis + code viewer; refactor form analysis to rule engine

Browse Source 
- API: add `POST /api/analyze_script` (app/blueprints/api.py)
  - Fetch one external script to artifacts, run rules, return findings + snippet
  - Uses new ExternalScriptFetcher (results_path aware) and job UUID
  - Returns: { ok, final_url, status_code, bytes, truncated, sha256, artifact_path, findings[], snippet, snippet_len }
  - TODO: document in openapi/openapi.yaml

- Fetcher: update `app/utils/external_fetch.py`
  - Constructed with `results_path` (UUID dir); writes to `<results_path>/scripts/fetched/<index>.js`
  - Loads settings via `get_settings()`, logs via std logging

- UI (results.html):
  - Move “Analyze external script” action into **Content Snippet** column for external rows
  - Clicking replaces button with `<details>` snippet, shows rule matches, and adds “open in viewer” link
  - Robust fetch handler (checks JSON, shows errors); builds viewer URL from absolute artifact path

- Viewer:
  - New route: `GET /view/artifact/<run_uuid>/<path:filename>` (app/blueprints/ui.py)
  - New template: Monaco-based read-only code viewer (viewer.html)
  - Removes SRI on loader to avoid integrity block; loads file via `raw_url` and detects language by extension

- Forms:
  - Refactor `analyze_forms` to mirror scripts analysis:
    - Uses rule engine (`category == "form"`) across regex/function rules
    - Emits rows only when matches exist
    - Includes `content_snippet`, `action`, `method`, `inputs`, `rules`
  - Replace legacy plumbing (`flagged`, `flag_reasons`, `status`) in output
  - Normalize form function rules to canonical returns `(bool, Optional[str])`:
    - `form_action_missing`
    - `form_http_on_https_page`
    - `form_submits_to_different_host`
    - Add minor hardening (lowercasing hosts, no-op actions, clearer reasons)

- CSS: add `.forms-table` to mirror `.scripts-table` (5 columns)
  - Fixed table layout, widths per column, chip/snippet styling, responsive tweaks

- Misc:
  - Fix “working outside app context” issue by avoiding `current_app` at import time (left storage logic inside routes)
  - Add “View Source” link to open page source in viewer

Refs:
- Roadmap: mark “Source code viewer” done; keep TODO to add `/api/analyze_script` to OpenAPI
This commit is contained in:
Phillip Tarrant
2025-08-21 15:32:24 -05:00
parent 05cf23ad67
commit 3a24b392f2
15 changed files with 1192 additions and 218 deletions
Download Patch File Download Diff File Expand all files Collapse all files

212
app/blueprints/api.py Normal file
View File

@@ -0,0 +1,212 @@
# app/blueprints/api.py
"""
API blueprint for JSON endpoints.
Endpoints:
POST /api/analyze_script
Body:
{
"job_id": "<uuid>", # or "uuid": "<uuid>"
"url": "https://cdn.example.com/app.js",
"category": "script" # optional, defaults to "script"
}
Response:
{
"ok": true,
"final_url": "...",
"status_code": 200,
"bytes": 12345,
"truncated": false,
"sha256": "...",
"artifact_path": "/abs/path/to/<uuid>/scripts/fetched/<index>.js",
"findings": [ { "name": "...", "description": "...", "severity": "...", "tags": [...], "reason": "..." }, ... ],
"snippet": "<trimmed content>",
"snippet_len": 45678
}
"""
import os
import time
from flask import Blueprint, request, jsonify, current_app, send_file, abort
from pathlib import Path
from app.utils.settings import get_settings
from app.utils.external_fetcher import ExternalScriptFetcher
from werkzeug.exceptions import HTTPException
api_bp = Blueprint("api", __name__, url_prefix="/api")
def _resolve_results_path(job_id: str) -> str:
"""
Compute the absolute results directory for a given job UUID.
Prefers <BASE>/artifacts/<uuid>, falls back to <BASE>/<uuid>.
"""
base_dir = "/data"
candidate_with_artifacts = os.path.join(base_dir, job_id)
if os.path.isdir(candidate_with_artifacts):
return candidate_with_artifacts
fallback = os.path.join(base_dir, job_id)
os.makedirs(fallback, exist_ok=True)
return fallback
def _make_snippet(text: str, max_chars: int = 1200) -> str:
"""Produce a trimmed, safe-to-render snippet of the script contents."""
if not text:
return ""
snippet = text.strip()
return (snippet[:max_chars] + "…") if len(snippet) > max_chars else snippet
@api_bp.errorhandler(400)
@api_bp.errorhandler(403)
@api_bp.errorhandler(404)
@api_bp.errorhandler(405)
def _api_err(err):
"""
Return JSON for common client errors.
"""
if isinstance(err, HTTPException):
code = err.code
name = (err.name or "error").lower()
else:
code = 400
name = "error"
return jsonify({"ok": False, "error": name}), code
@api_bp.errorhandler(500)
def _api_500(err):
"""
Return JSON for server errors and log the exception.
"""
try:
current_app.logger.exception("API 500")
except Exception:
pass
return jsonify({"ok": False, "error": "internal server error"}), 500
@api_bp.post("/analyze_script")
def analyze_script():
"""
Analyze EXACTLY one external script URL for a given job UUID.
Expected JSON body:
{ "job_id": "<uuid>", "url": "https://cdn.example.com/app.js", "category": "script" }
"""
body = request.get_json(silent=True) or {}
job_id_raw = body.get("job_id") or body.get("uuid")
script_url_raw = body.get("url")
category = (body.get("category") or "script").strip() or None # default to "script"
job_id = (job_id_raw or "").strip() if isinstance(job_id_raw, str) else ""
script_url = (script_url_raw or "").strip() if isinstance(script_url_raw, str) else ""
# log this request
current_app.logger.info(f"Got request to analyze {script_url} via API ")
if not job_id or not script_url:
return jsonify({"ok": False, "error": "Missing job_id (or uuid) or url"}), 400
settings = get_settings()
if not settings.external_fetch.enabled:
return jsonify({"ok": False, "error": "Feature disabled"}), 400
# Resolve the UUID-backed results directory for this run.
results_path = _resolve_results_path(job_id)
# Initialize the fetcher; it reads its own settings internally.
fetcher = ExternalScriptFetcher(results_path=results_path)
# Unique index for the saved file name: <results_path>/scripts/fetched/<index>.js
unique_index = int(time.time() * 1000)
outcome = fetcher.fetch_one(script_url=script_url, index=unique_index)
if not outcome.ok or not outcome.saved_path:
return jsonify({
"ok": False,
"error": outcome.reason,
"status_code": outcome.status_code,
"final_url": outcome.final_url
}), 502
# Read bytes and decode to UTF-8 for rules and snippet
try:
with open(outcome.saved_path, "rb") as fh:
js_text = fh.read().decode("utf-8", errors="ignore")
except Exception:
js_text = ""
# Pull the rules engine from the app (prefer attribute, then config).
findings = []
try:
engine = getattr(current_app, "rule_engine", None)
if engine is None:
engine = current_app.config.get("RULE_ENGINE")
except Exception:
engine = None
if engine is not None and hasattr(engine, "run_all"):
try:
# run_all returns PASS/FAIL for each rule; we only surface FAIL (matched) to the UI
all_results = engine.run_all(js_text, category=category)
if isinstance(all_results, list):
matched = []
for r in all_results:
try:
if (r.get("result") == "FAIL"):
matched.append({
"name": r.get("name"),
"description": r.get("description"),
"severity": r.get("severity"),
"tags": r.get("tags") or [],
"reason": r.get("reason"),
"category": r.get("category"),
})
except Exception:
# Ignore malformed entries
continue
findings = matched
except Exception as exc:
try:
current_app.logger.error("Rule engine error", extra={"error": str(exc)})
except Exception:
pass
findings = []
snippet = _make_snippet(js_text, max_chars=settings.ui.snippet_preview_len)
return jsonify({
"ok": True,
"final_url": outcome.final_url,
"status_code": outcome.status_code,
"bytes": outcome.bytes_fetched,
"truncated": outcome.truncated,
"sha256": outcome.sha256_hex,
"artifact_path": outcome.saved_path,
"findings": findings, # only FAILed rules
"snippet": snippet,
"snippet_len": len(js_text)
})
@api_bp.get("/artifacts/<run_uuid>/<filename>")
def get_artifact_raw(run_uuid, filename):
# prevent path traversal
if "/" in filename or ".." in filename:
abort(400)
run_dir = _resolve_results_path(run_uuid)
full_path = Path(run_dir) / filename
# if file is not there, give a 404
if not os.path.isfile(full_path):
abort(404)
# else return file
return send_file(full_path, as_attachment=False)