feat(engine,ui): unify detection in rules engine, add function rules & per-script matches; improve scripts table UX
Core changes
- Centralize detection in the Rules Engine; browser.py now focuses on fetch/extract/persist.
- Add class-based adapters:
- FactAdapter: converts snippets → structured facts.
- FunctionRuleAdapter: wraps dict-based rule functions for engine input (str or dict).
- Register function rules (code-based) alongside YAML rules:
- form_action_missing
- form_http_on_https_page
- form_submits_to_different_host
- script_src_uses_data_or_blob
- script_src_has_dangerous_extension
- script_third_party_host
Rules & YAML
- Expand/normalize YAML rules with severities + tags; tighten patterns.
- Add new regex rules: new_function_usage, unescape_usage, string_timer_usage, long_hex_constants.
- Move iframe rule to `text` category.
- Keep existing script/form/text rules; all compile under IGNORECASE.
Browser / analysis refactor
- browser.py:
- Remove inline heuristics; rely on engine for PASS/FAIL, reason, severity, tags.
- Build page-level overview (`rule_checks`) across categories.
- Analyze forms: add `base_url` + `base_hostname` to snippet so function rules can evaluate; include per-form rule_checks.
- Analyze scripts: **per-script evaluation**:
- Inline -> run regex script rules on inline text.
- External -> run function script rules with a facts dict (src/src_hostname/base_url/base_hostname).
- Only include scripts that matched ≥1 rule; attach severity/tags to matches.
- Persist single source of truth: `/data/<uuid>/results.json`.
- Backward-compat: `fetch_page_artifacts(..., engine=...)` kwarg accepted/ignored.
UI/UX
- Suspicious Scripts table now shows only matched scripts.
- Add severity badges and tag chips; tooltips show rule description.
- Prevent table blowouts:
- Fixed layout + ellipsis + wrapping helpers (`.scripts-table`, `.breakable`, `details pre.code`).
- Shortened inline snippet preview (configurable).
- Minor template niceties (e.g., rel="noopener" on external links where applicable).
Config
- Add `ui.snippet_preview_len` to settings.yaml; default 160.
- Load into `app.config["SNIPPET_PREVIEW_LEN"]` and use in `analyze_scripts`.
Init / wiring
- Import and register function rules as `Rule(...)` objects (not dicts).
- Hook Rules Engine to Flask logger for verbose/diagnostic output.
- Log totals on startup; keep YAML path override via `SNEAKYSCOPE_RULES_FILE`.
Bug fixes
- Fix boot crash: pass `Rule` instances to `engine.add_rule()` instead of dicts.
- Fix “N/A” in scripts table by actually computing per-script matches.
- Ensure form rules fire by including `base_url`/`base_hostname` in form snippets.
Roadmap
- Update roadmap to reflect completed items:
- “Show each check and whether it triggered (pass/fail list per rule)”
- Severity levels + tags in Suspicious Scripts
- Results.json as route source of truth
- Scripts table UX (badges, tooltips, layout fix)
This commit is contained in:
@@ -1,80 +1,138 @@
|
||||
# config/suspicious_rules.yaml
|
||||
# Baseline suspicious rules for SneakyScope
|
||||
# Organized by category: script, form, text
|
||||
# Extend these with more specific rules as needed
|
||||
# Notes:
|
||||
# - Engine compiles regex with IGNORECASE.
|
||||
# - 'severity' is optional: low | medium | high
|
||||
# - 'tags' is optional: list of strings for grouping
|
||||
|
||||
# --- Script Rules ---
|
||||
- name: eval_usage
|
||||
description: "Use of eval() in script"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: "\\beval\\("
|
||||
pattern: '\beval\s*\('
|
||||
severity: high
|
||||
tags: [obfuscation, unsafe-eval]
|
||||
|
||||
- name: new_function_usage
|
||||
description: "Use of Function constructor (new Function)"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: '\bnew\s+Function\s*\('
|
||||
severity: high
|
||||
tags: [obfuscation]
|
||||
|
||||
- name: document_write
|
||||
description: "Use of document.write (often abused in malicious injections)"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: "document\\.write\\("
|
||||
pattern: '\bdocument\s*\.\s*write\s*\('
|
||||
severity: medium
|
||||
tags: [injection, legacy-api]
|
||||
|
||||
- name: inline_event_handler
|
||||
description: "Inline event handler detected (onload, onclick, etc.)"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: "on(load|click|error|mouseover|keydown)\\s*="
|
||||
pattern: '\bon(load|click|error|mouseover|mouseenter|submit|keydown|keyup|change)\s*='
|
||||
severity: medium
|
||||
tags: [inline-handlers, potential-xss]
|
||||
|
||||
- name: obfuscated_encoding
|
||||
description: "Suspicious use of atob() or btoa() (base64 encoding/decoding)"
|
||||
description: "Suspicious use of atob()/btoa() (base64 encode/decode)"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: "\\b(atob|btoa)\\("
|
||||
pattern: '\b(atob|btoa)\s*\('
|
||||
severity: medium
|
||||
tags: [encoding, obfuscation]
|
||||
|
||||
- name: suspicious_iframe
|
||||
description: "Iframe usage in script (possible phishing/malvertising)"
|
||||
- name: unescape_usage
|
||||
description: "Use of unescape() (legacy/obfuscation)"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: "<iframe[^>]*>"
|
||||
pattern: '\bunescape\s*\('
|
||||
severity: low
|
||||
tags: [legacy-api, obfuscation]
|
||||
|
||||
- name: string_timer_usage
|
||||
description: "String passed to setTimeout/setInterval (sink for XSS)"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: '\bset(?:Timeout|Interval)\s*\(\s*[''"`].+[''"`]\s*,'
|
||||
severity: medium
|
||||
tags: [xss-sink]
|
||||
|
||||
- name: long_hex_constants
|
||||
description: "Long hex-like constants (possible obfuscation)"
|
||||
category: script
|
||||
type: regex
|
||||
pattern: '["'']?0x[0-9a-fA-F]{16,}["'']?'
|
||||
severity: low
|
||||
tags: [obfuscation]
|
||||
|
||||
# --- Form Rules ---
|
||||
- name: suspicious_form_action
|
||||
description: "Form action with external URL (potential credential exfiltration)"
|
||||
- name: suspicious_form_action_absolute
|
||||
description: "Form action uses absolute URL (potential credential exfiltration)"
|
||||
category: form
|
||||
type: regex
|
||||
pattern: "<form[^>]*action=['\"]http"
|
||||
pattern: '<form\b[^>]*\baction\s*=\s*[''"]https?://'
|
||||
severity: medium
|
||||
tags: [exfiltration, form]
|
||||
|
||||
- name: hidden_inputs
|
||||
description: "Form with hidden inputs (possible credential harvesting)"
|
||||
description: "Form with hidden inputs (could be used to smuggle data)"
|
||||
category: form
|
||||
type: regex
|
||||
pattern: "<input[^>]*type=['\"]hidden"
|
||||
pattern: '<input\b[^>]*\btype\s*=\s*[''"]hidden[''"]'
|
||||
severity: low
|
||||
tags: [stealth, form]
|
||||
|
||||
- name: password_field
|
||||
description: "Form requesting password field"
|
||||
description: "Form requests a password field"
|
||||
category: form
|
||||
type: regex
|
||||
pattern: "<input[^>]*type=['\"]password"
|
||||
pattern: '<input\b[^>]*\btype\s*=\s*[''"]password[''"]'
|
||||
severity: high
|
||||
tags: [credentials, form]
|
||||
|
||||
# --- Text Rules (Social Engineering / BEC) ---
|
||||
- name: urgent_request
|
||||
description: "Language suggesting urgency (common in phishing/BEC)"
|
||||
category: text
|
||||
type: regex
|
||||
pattern: "(urgent|immediately|asap|action required)"
|
||||
pattern: '\b(urgent|immediately|asap|action\s*required|verify\s*now)\b'
|
||||
severity: medium
|
||||
tags: [bec, urgency]
|
||||
|
||||
- name: account_suspension
|
||||
description: "Threat of account suspension/closure"
|
||||
category: text
|
||||
type: regex
|
||||
pattern: "(account.*suspend|account.*close|verify.*account)"
|
||||
pattern: '\b(account\s*(suspend|closure|close)|verify\s*account)\b'
|
||||
severity: medium
|
||||
tags: [bec, scare-tactics]
|
||||
|
||||
- name: financial_request
|
||||
description: "Request for gift cards, wire transfer, or money"
|
||||
category: text
|
||||
type: regex
|
||||
pattern: "(gift card|wire transfer|bank account|bitcoin|payment required)"
|
||||
pattern: '\b(gift\s*card|wire\s*transfer|bank\s*account|bitcoin|crypto|payment\s*required)\b'
|
||||
severity: high
|
||||
tags: [bec, financial]
|
||||
|
||||
- name: credential_reset
|
||||
description: "Password reset or credential reset wording"
|
||||
category: text
|
||||
type: regex
|
||||
pattern: "(reset password|update credentials|login to verify)"
|
||||
pattern: '\b(reset\s*password|update\s*credentials|log\s*in\s*to\s*verify|password\s*expiry)\b'
|
||||
severity: medium
|
||||
tags: [bec, credentials]
|
||||
|
||||
- name: suspicious_iframe
|
||||
description: "Iframe tag present (possible phishing/malvertising/drive-by)"
|
||||
category: text
|
||||
type: regex
|
||||
pattern: '<iframe\b[^>]*\bsrc\s*=\s*[''"][^''"]+[''"]'
|
||||
severity: medium
|
||||
tags: [iframe, phishing, malvertising]
|
||||
|
||||
Reference in New Issue
Block a user