restructure of dirs, huge docs update
This commit is contained in:
502
docs/CLI_SCANNING.md
Normal file
502
docs/CLI_SCANNING.md
Normal file
@@ -0,0 +1,502 @@
|
||||
# CLI Scanner Guide
|
||||
|
||||
The SneakyScanner CLI provides a standalone scanning tool for quick one-off scans, testing, or CI/CD pipelines without requiring the web application.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Quick Start](#quick-start)
|
||||
2. [Configuration](#configuration)
|
||||
3. [Scan Performance](#scan-performance)
|
||||
4. [Output Formats](#output-formats)
|
||||
5. [Screenshot Capture](#screenshot-capture)
|
||||
6. [HTML Reports](#html-reports)
|
||||
7. [Advanced Usage](#advanced-usage)
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Using Docker Compose (Recommended)
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker compose -f docker-compose-standalone.yml build
|
||||
|
||||
# Run a scan
|
||||
docker compose -f docker-compose-standalone.yml up
|
||||
|
||||
# Results saved to ./output/ directory
|
||||
```
|
||||
|
||||
### Using Docker Directly
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker build -t sneakyscanner .
|
||||
|
||||
# Run a scan
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v $(pwd)/output:/app/output \
|
||||
sneakyscanner /app/configs/your-config.yaml
|
||||
```
|
||||
|
||||
### Requirements
|
||||
|
||||
- Docker
|
||||
- Linux host (required for `--privileged` and `--network host`)
|
||||
- Configuration file in `configs/` directory
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
The YAML configuration file defines scan parameters and expectations.
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
```yaml
|
||||
title: "My Infrastructure Scan"
|
||||
sites:
|
||||
- name: "Web Servers"
|
||||
ips:
|
||||
- address: "192.168.1.10"
|
||||
expected:
|
||||
ping: true
|
||||
tcp_ports: [22, 80, 443]
|
||||
udp_ports: []
|
||||
services: ["ssh", "http", "https"]
|
||||
```
|
||||
|
||||
### CIDR Range Configuration
|
||||
|
||||
```yaml
|
||||
title: "Network Scan"
|
||||
sites:
|
||||
- name: "Production Network"
|
||||
cidr: "192.168.1.0/24"
|
||||
expected_ports:
|
||||
- port: 22
|
||||
protocol: tcp
|
||||
service: "ssh"
|
||||
- port: 80
|
||||
protocol: tcp
|
||||
service: "http"
|
||||
- port: 443
|
||||
protocol: tcp
|
||||
service: "https"
|
||||
ping_expected: true
|
||||
```
|
||||
|
||||
### Configuration Reference
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `title` | string | Yes | Report title |
|
||||
| `sites` | array | Yes | List of sites to scan |
|
||||
| `sites[].name` | string | Yes | Site name |
|
||||
| `sites[].ips` | array | Conditional | List of IP addresses (if not using CIDR) |
|
||||
| `sites[].cidr` | string | Conditional | CIDR range (if not using IPs) |
|
||||
| `sites[].ips[].address` | string | Yes | IP address |
|
||||
| `sites[].ips[].expected.ping` | boolean | No | Expected ping response |
|
||||
| `sites[].ips[].expected.tcp_ports` | array | No | Expected TCP ports |
|
||||
| `sites[].ips[].expected.udp_ports` | array | No | Expected UDP ports |
|
||||
| `sites[].ips[].expected.services` | array | No | Expected service names |
|
||||
| `sites[].expected_ports` | array | No | Expected ports for CIDR range |
|
||||
| `sites[].ping_expected` | boolean | No | Expected ping for CIDR range |
|
||||
|
||||
See `configs/example-site.yaml` for a complete example.
|
||||
|
||||
---
|
||||
|
||||
## Scan Performance
|
||||
|
||||
SneakyScanner uses a five-phase approach for comprehensive scanning:
|
||||
|
||||
1. **Ping Scan** (masscan): ICMP echo detection
|
||||
- Duration: ~1-2 seconds
|
||||
- Tests network reachability
|
||||
|
||||
2. **TCP Port Discovery** (masscan): Scans all 65535 TCP ports
|
||||
- Rate: 10,000 packets/second
|
||||
- Duration: ~13 seconds per 2 IPs
|
||||
|
||||
3. **UDP Port Discovery** (masscan): Scans all 65535 UDP ports
|
||||
- Rate: 10,000 packets/second
|
||||
- Duration: ~13 seconds per 2 IPs
|
||||
|
||||
4. **Service Detection** (nmap): Identifies services on discovered TCP ports
|
||||
- Intensity level: 5 (balanced)
|
||||
- Duration: ~20-60 seconds per IP with open ports
|
||||
- Extracts product names and versions
|
||||
|
||||
5. **HTTP/HTTPS Analysis**: Web protocol detection and SSL/TLS analysis
|
||||
- Screenshot capture (Playwright)
|
||||
- Certificate extraction (sslyze)
|
||||
- TLS version testing
|
||||
- Duration: ~10-20 seconds per web service
|
||||
|
||||
**Example**: Scanning 2 IPs with 10 open ports each (including 2-3 web services) typically takes 2-3 minutes total.
|
||||
|
||||
### Performance Tuning
|
||||
|
||||
Adjust scan rate in the scanner code if needed:
|
||||
- Default: 10,000 pps (packets per second)
|
||||
- Increase for faster scans (may cause network congestion)
|
||||
- Decrease for slower, more reliable scans
|
||||
|
||||
---
|
||||
|
||||
## Output Formats
|
||||
|
||||
After each scan completes, SneakyScanner automatically generates three output formats:
|
||||
|
||||
### 1. JSON Report
|
||||
|
||||
**Filename**: `scan_report_YYYYMMDD_HHMMSS.json`
|
||||
|
||||
Machine-readable scan data with all discovered services, ports, and SSL/TLS information.
|
||||
|
||||
**Structure**:
|
||||
```json
|
||||
{
|
||||
"title": "Scan Title",
|
||||
"scan_time": "2025-01-15T10:30:00Z",
|
||||
"scan_duration": 95.3,
|
||||
"config_file": "/app/configs/example-site.yaml",
|
||||
"sites": [
|
||||
{
|
||||
"name": "Site Name",
|
||||
"ips": [
|
||||
{
|
||||
"address": "192.168.1.10",
|
||||
"expected": {...},
|
||||
"actual": {
|
||||
"ping": true,
|
||||
"tcp_ports": [22, 80, 443],
|
||||
"udp_ports": [],
|
||||
"services": [...]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 2. HTML Report
|
||||
|
||||
**Filename**: `scan_report_YYYYMMDD_HHMMSS.html`
|
||||
|
||||
Human-readable report with dark theme, summary dashboard, and detailed service breakdown.
|
||||
|
||||
Features:
|
||||
- Summary statistics
|
||||
- Drift alerts (unexpected ports/services)
|
||||
- Security warnings (expiring certs, weak TLS)
|
||||
- Site-by-site breakdown
|
||||
- Expandable service details
|
||||
- SSL/TLS certificate information
|
||||
|
||||
See [HTML Reports](#html-reports) section for details.
|
||||
|
||||
### 3. ZIP Archive
|
||||
|
||||
**Filename**: `scan_report_YYYYMMDD_HHMMSS.zip`
|
||||
|
||||
Contains:
|
||||
- JSON report
|
||||
- HTML report
|
||||
- All screenshots (if web services were found)
|
||||
|
||||
Useful for:
|
||||
- Easy sharing
|
||||
- Archival
|
||||
- Compliance documentation
|
||||
|
||||
### 4. Screenshots Directory
|
||||
|
||||
**Directory**: `scan_report_YYYYMMDD_HHMMSS_screenshots/`
|
||||
|
||||
PNG screenshots of all discovered web services:
|
||||
- Filename format: `{ip}_{port}.png` (e.g., `192_168_1_10_443.png`)
|
||||
- Viewport size: 1280x720
|
||||
- Referenced in JSON report under `http_info.screenshot`
|
||||
|
||||
---
|
||||
|
||||
## Screenshot Capture
|
||||
|
||||
SneakyScanner automatically captures webpage screenshots for all discovered HTTP and HTTPS services.
|
||||
|
||||
### Automatic Detection
|
||||
|
||||
Screenshots are captured for services identified as web services based on:
|
||||
- Nmap service detection results (http, https, ssl, http-proxy)
|
||||
- Common web ports (80, 443, 8000, 8006, 8080, 8081, 8443, 8888, 9443)
|
||||
|
||||
### Capture Process
|
||||
|
||||
For each web service:
|
||||
|
||||
1. **Launch Browser**: Headless Chromium (once per scan, reused)
|
||||
2. **Navigate**: To service URL (HTTP or HTTPS)
|
||||
3. **Wait**: For network to be idle (up to 15 seconds)
|
||||
4. **Capture**: Viewport screenshot (1280x720 pixels)
|
||||
5. **Save**: As PNG file in screenshots directory
|
||||
|
||||
### Configuration
|
||||
|
||||
Default settings (configured in `src/screenshot_capture.py`):
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Viewport size | 1280x720 |
|
||||
| Timeout | 15 seconds |
|
||||
| Browser | Chromium (headless) |
|
||||
| SSL handling | Ignores HTTPS errors |
|
||||
| User agent | Mozilla/5.0 (Windows NT 10.0; Win64; x64) |
|
||||
|
||||
### Error Handling
|
||||
|
||||
Screenshots are captured on a best-effort basis:
|
||||
- Failed screenshots are logged but don't stop the scan
|
||||
- Services without screenshots omit the `screenshot` field in JSON
|
||||
- Common errors: timeout, connection refused, invalid SSL
|
||||
|
||||
### Disabling Screenshots
|
||||
|
||||
To disable screenshot capture, modify `src/screenshot_capture.py` or comment out the screenshot phase in `src/scanner.py`.
|
||||
|
||||
---
|
||||
|
||||
## HTML Reports
|
||||
|
||||
SneakyScanner automatically generates comprehensive HTML reports after each scan.
|
||||
|
||||
### Automatic Generation
|
||||
|
||||
HTML reports are created after every scan, along with JSON reports and ZIP archives. All outputs share the same timestamp.
|
||||
|
||||
### Manual Generation
|
||||
|
||||
Generate HTML reports from existing JSON scan data:
|
||||
|
||||
```bash
|
||||
# Generate HTML report (creates report in same directory as JSON)
|
||||
cd app/
|
||||
python3 src/report_generator.py ../output/scan_report_20250115_103000.json
|
||||
|
||||
# Specify custom output path
|
||||
python3 src/report_generator.py ../output/scan_report.json /path/to/custom_report.html
|
||||
```
|
||||
|
||||
### Report Features
|
||||
|
||||
**Summary Dashboard**:
|
||||
- **Scan Statistics**: Total IPs, TCP/UDP ports, services, web services, screenshots
|
||||
- **Drift Alerts**: Unexpected ports, missing services, new services
|
||||
- **Security Warnings**: Expiring certificates (<30 days), weak TLS (1.0/1.1), self-signed certs, high ports (>10000)
|
||||
|
||||
**Site-by-Site Breakdown**:
|
||||
- Organized by logical site grouping from configuration
|
||||
- Per-IP sections with status badges (ping, port drift)
|
||||
- Service tables with expandable details (click to expand)
|
||||
- Visual badges: green (expected), red (unexpected), yellow (missing/warning)
|
||||
|
||||
**Service Details** (expandable):
|
||||
- Product name, version, extra information, OS type
|
||||
- HTTP/HTTPS protocol detection
|
||||
- Screenshot links for web services
|
||||
- SSL/TLS certificate details:
|
||||
- Subject, issuer, validity dates, serial number
|
||||
- Days until expiration (color-coded warnings)
|
||||
- Subject Alternative Names (SANs)
|
||||
- TLS version support (1.0, 1.1, 1.2, 1.3) with cipher suites
|
||||
- Weak TLS and self-signed certificate warnings
|
||||
|
||||
**UDP Port Handling**:
|
||||
- Expected UDP ports: green "Expected" badge
|
||||
- Unexpected UDP ports: red "Unexpected" badge
|
||||
- Missing UDP ports: yellow "Missing" badge
|
||||
- Note: Service detection not available for UDP (nmap limitation)
|
||||
|
||||
**Design**:
|
||||
- Dark theme with slate/grey color scheme
|
||||
- Responsive layout
|
||||
- No external dependencies (single HTML file)
|
||||
- Minimal JavaScript for expand/collapse
|
||||
- Optimized hover effects
|
||||
|
||||
### Report Portability
|
||||
|
||||
The HTML report is a standalone file that can be:
|
||||
- Opened in any web browser (Chrome, Firefox, Safari, Edge)
|
||||
- Shared via email or file transfer
|
||||
- Archived for compliance or historical comparison
|
||||
- Viewed without internet connection
|
||||
|
||||
**Note**: Screenshot links use relative paths, so keep the report and screenshot directory together.
|
||||
|
||||
---
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Running on Remote Targets
|
||||
|
||||
```bash
|
||||
# Scan remote network via Docker host
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v $(pwd)/output:/app/output \
|
||||
sneakyscanner /app/configs/remote-network.yaml
|
||||
```
|
||||
|
||||
**Note**: The Docker host must have network access to the target network.
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
```yaml
|
||||
# Example GitLab CI pipeline
|
||||
scan-infrastructure:
|
||||
stage: test
|
||||
image: docker:latest
|
||||
services:
|
||||
- docker:dind
|
||||
script:
|
||||
- docker build -t sneakyscanner .
|
||||
- docker run --rm --privileged --network host \
|
||||
-v $PWD/configs:/app/configs:ro \
|
||||
-v $PWD/output:/app/output \
|
||||
sneakyscanner /app/configs/production.yaml
|
||||
artifacts:
|
||||
paths:
|
||||
- output/
|
||||
expire_in: 30 days
|
||||
```
|
||||
|
||||
### Batch Scanning
|
||||
|
||||
```bash
|
||||
# Scan multiple configs sequentially
|
||||
for config in configs/*.yaml; do
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v $(pwd)/output:/app/output \
|
||||
sneakyscanner "/app/configs/$(basename $config)"
|
||||
done
|
||||
```
|
||||
|
||||
### Custom Output Directory
|
||||
|
||||
```bash
|
||||
# Use custom output directory
|
||||
mkdir -p /path/to/custom/output
|
||||
|
||||
docker run --rm --privileged --network host \
|
||||
-v $(pwd)/configs:/app/configs:ro \
|
||||
-v /path/to/custom/output:/app/output \
|
||||
sneakyscanner /app/configs/config.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Permission Denied Errors
|
||||
|
||||
**Problem**: masscan or nmap fails with permission denied
|
||||
|
||||
**Solution**: Ensure Docker is running with `--privileged` flag:
|
||||
```bash
|
||||
docker run --rm --privileged --network host ...
|
||||
```
|
||||
|
||||
### No Ports Found
|
||||
|
||||
**Problem**: Scan completes but finds no open ports
|
||||
|
||||
**Possible Causes**:
|
||||
- Firewall blocking scans
|
||||
- Wrong network (ensure `--network host`)
|
||||
- Target hosts are down
|
||||
- Incorrect IP addresses in config
|
||||
|
||||
**Debug**:
|
||||
```bash
|
||||
# Test ping manually
|
||||
ping 192.168.1.10
|
||||
|
||||
# Check Docker network mode
|
||||
docker inspect <container-id> | grep NetworkMode
|
||||
```
|
||||
|
||||
### Screenshots Failing
|
||||
|
||||
**Problem**: Screenshots not being captured
|
||||
|
||||
**Possible Causes**:
|
||||
- Chromium not installed (check Dockerfile)
|
||||
- Timeout too short (increase in screenshot_capture.py)
|
||||
- Web service requires authentication
|
||||
- SSL certificate errors
|
||||
|
||||
**Debug**: Check scan logs for screenshot errors
|
||||
|
||||
### Scan Takes Too Long
|
||||
|
||||
**Problem**: Scan runs for 30+ minutes
|
||||
|
||||
**Solutions**:
|
||||
- Reduce scan rate (edit scanner.py)
|
||||
- Limit port range (edit scanner.py to scan specific ports)
|
||||
- Reduce number of IPs in config
|
||||
- Disable UDP scanning if not needed
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Privileged Mode
|
||||
|
||||
The CLI scanner requires `--privileged` flag for:
|
||||
- Raw socket access (masscan, nmap)
|
||||
- ICMP echo requests (ping)
|
||||
|
||||
**Security implications**:
|
||||
- Container has extensive host capabilities
|
||||
- Only run on trusted networks
|
||||
- Don't expose to public networks
|
||||
|
||||
### Network Mode: Host
|
||||
|
||||
The scanner uses `--network host` for:
|
||||
- Direct network access without NAT
|
||||
- Raw packet sending
|
||||
- Accurate service detection
|
||||
|
||||
**Security implications**:
|
||||
- Container shares host network namespace
|
||||
- Can access all host network interfaces
|
||||
- Bypass Docker network isolation
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. **Only scan authorized networks**
|
||||
2. **Run on dedicated scan server** (not production)
|
||||
3. **Limit network access** with firewall rules
|
||||
4. **Review scan configs** before running
|
||||
5. **Store results securely** (may contain sensitive data)
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
- **Deployment Guide**: [docs/DEPLOYMENT.md](DEPLOYMENT.md)
|
||||
- **API Reference**: [docs/API_REFERENCE.md](API_REFERENCE.md)
|
||||
- **Roadmap**: [docs/ROADMAP.md](ROADMAP.md)
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-11-17
|
||||
**Version**: Phase 4 Complete
|
||||
Reference in New Issue
Block a user