skills-ws
Version:
85 agent skills for AI coding assistants — marketing, growth, web3, dev, design & operations. Built for OpenClaw, Claude Code, Cursor, and Codex.
651 lines (518 loc) • 31.1 kB
Markdown
name: security-pentester
description: "Active offensive testing — OWASP Top 10 exploitation, white-box source-aware scans, CI/CD security gates, vuln report interpretation, remediation. Use when running pentests or attacking your own app. Paired with security-hardening (defensive code) and security-sentinel (runtime threat intel)."
# Security Pentester
> Disambiguation: this skill = active offensive testing. For defensive code patterns see `security-hardening`. For runtime threat intel / URL+wallet scam scanning see `security-sentinel`.
Autonomous web application penetration testing driven by an LLM-agent pipeline (Shannon), backed by manual validation. Source-aware analysis combines code reading with live exploitation attempts and prioritizes findings that come with a reproducible proof-of-concept.
This skill is tool-agnostic in principle: Shannon is the reference automated driver, but the workflow (recon → analyze → exploit → triage → remediate → regression-test) and every remediation playbook below apply to any pentest engagement (manual, Burp/ZAP-driven, or other agents).
## Core Principle
**Evidence over assertion.** Prefer findings that ship a reproducible proof-of-concept (PoC) over unproven "potential" findings. A PoC sharply reduces false positives but does NOT guarantee correctness: an LLM-driven pipeline can still hallucinate impact, mis-read a response, or fire in a test-only configuration. Treat every automated finding as a *lead*, not a verdict.
**Manual validation is mandatory for any finding that drives a security decision** — not only Critical/High. Before you file a ticket, block a deploy, or tell anyone "you are vulnerable," reproduce the PoC yourself against the in-scope target and confirm real impact (see section 4 — False Positive Identification).
## 1. Vulnerability Coverage
### OWASP Top 10 Testing Matrix
| Category | What Shannon Tests | Techniques |
|----------|-------------------|------------|
| **SQL Injection** | Union-based, blind (boolean/time), error-based, second-order | Payload fuzzing, source-guided parameter discovery |
| **Command Injection** | OS command injection via user input | Backtick, pipe, semicolon, `$()` injection patterns |
| **XSS** | Reflected, stored, DOM-based | Context-aware payload generation, filter bypass |
| **SSRF** | Internal network access, cloud metadata | `http://169.254.169.254`, internal service probing |
| **Broken Authentication** | Credential stuffing, session fixation, JWT attacks | Brute force, token manipulation, 2FA bypass — **rate-limit & isolate, see §8** |
| **Broken Authorization** | IDOR, privilege escalation, role bypass | Horizontal/vertical access control testing |
> The high-volume auth tests above (brute force, credential stuffing, 2FA enumeration) and the SSRF metadata probes are **noisy and side-effecting**: they can lock real accounts, blow rate budgets, trigger SMS/email/billing, and page on-call. Do not run them against any environment without the safe-test controls in **§8 — Safe Testing Practices**.
### OWASP Web Security Testing Guide (WSTG) Coverage
```
WSTG-INFO — Information Gathering ✓ Automated
WSTG-CONF — Configuration Management ✓ Automated
WSTG-IDNT — Identity Management ✓ Automated
WSTG-ATHN — Authentication Testing ✓ Automated
WSTG-ATHZ — Authorization Testing ✓ Automated
WSTG-SESS — Session Management ✓ Automated
WSTG-INPV — Input Validation ✓ Automated
WSTG-ERRH — Error Handling ✓ Automated
WSTG-CRYP — Cryptography ◐ Partial (TLS config, weak hashing)
WSTG-BUSN — Business Logic ✗ Manual (no automated tool reliably models domain rules — see §7)
WSTG-CLNT — Client-Side Testing ✓ Automated (DOM XSS, open redirects)
WSTG-APIS — API Testing ✓ Automated (REST, limited GraphQL)
```
## 2. Running a Pentest
### Quick Start
```bash
# Clone Shannon
git clone https://github.com/KeygraphHQ/shannon.git
cd shannon
# Set API key (use >> to append if .env already exists)
echo "ANTHROPIC_API_KEY=your-key-here" >> .env
# Run against a target (black-box)
./shannon start URL=https://target-app.example.com REPO=my-app
# Run with source code (white-box — recommended, finds more vulns)
./shannon start URL=https://target-app.example.com REPO=my-app
# Place source code in workspaces/my-app/repo/ before running
```
### Configuration (shannon.yaml)
```yaml
# Authentication config — tell Shannon how to log in
auth:
login_url: /login
credentials:
- username: testuser@example.com
password: TestPass123!
role: user
- username: admin@example.com
password: AdminPass456!
role: admin
# Scope rules
rules:
avoid:
- /api/admin/delete-all # Don't hit destructive endpoints
- /api/billing/* # Skip billing endpoints
- /logout # Don't log yourself out
focus:
- /api/* # Prioritize API endpoints
- /dashboard/* # Focus on authenticated surfaces
# 2FA support (if app uses TOTP)
totp:
secret: JBSWY3DPEHPK3PXP # PLACEHOLDER — replace with your test account's actual TOTP secret
```
### CLI Commands
> Flag names, subcommands, and report file paths change between releases. Verify against the current `KeygraphHQ/shannon` README (`./shannon --help`) before scripting against them — the names below are the reference set as of Jun 2026.
```bash
./shannon start URL=<url> REPO=<name> # Start full pentest
./shannon start URL=<url> REPO=<name> CONFIG=shannon.yaml # With config (always use in CI)
./shannon workspaces # List all workspaces
./shannon logs ID=<workflow-id> # Tail live logs
./shannon query ID=<workflow-id> # Check progress
./shannon stop # Stop containers (preserves data — safe)
```
**Destructive cleanup — guard it.** `./shannon stop CLEAN=true` deletes ALL workspace data: reports, PoCs, recon output, and logs. There is no undo. Never put it in an unattended script or CI job. Always export first and require an explicit confirmation:
```bash
# 1. Export everything you might need before destroying it
WS="workspaces/<name>"
mkdir -p "./shannon-archive/<name>-$(date +%Y%m%d)"
cp -r "$WS"/*.md "$WS"/*.json "./shannon-archive/<name>-$(date +%Y%m%d)/" 2>/dev/null
# 2. Confirm interactively before the irreversible step
read -r -p "Archived. DELETE all Shannon workspace data now? type 'DELETE': " ok
[ "$ok" = "DELETE" ] && ./shannon stop CLEAN=true || echo "Aborted — data preserved."
```
## 3. Understanding the Pipeline
### 4-Phase Architecture
```
Phase 1: RECONNAISSANCE
├── Pre-Recon (source code analysis with configured LLM)
│ └── Outputs: code_analysis_deliverable.md
└── Recon (attack surface mapping with Playwright + Nmap)
└── Outputs: recon_deliverable.md
Phase 2: VULNERABILITY ANALYSIS (5 parallel agents)
├── Injection Analysis → injection_analysis.md + exploitation_queue.json
├── XSS Analysis → xss_analysis.md + exploitation_queue.json
├── Auth Analysis → auth_analysis.md + exploitation_queue.json
├── SSRF Analysis → ssrf_analysis.md + exploitation_queue.json
└── AuthZ Analysis → authz_analysis.md + exploitation_queue.json
Phase 3: EXPLOITATION (5 parallel agents, conditional)
├── Injection Exploit → injection_exploitation_evidence.md
├── XSS Exploit → xss_exploitation_evidence.md
├── Auth Exploit → auth_exploitation_evidence.md
├── SSRF Exploit → ssrf_exploitation_evidence.md
└── AuthZ Exploit → authz_exploitation_evidence.md
Phase 4: REPORTING
└── comprehensive_security_assessment_report.md
```
### What Each Phase Does
**Pre-Recon** reads source code to understand the application architecture, identify entry points, map data flows, and find potential vulnerability patterns before any network interaction.
**Recon** maps the live attack surface: crawls the app with a headless browser, enumerates API endpoints, identifies technologies, scans for open ports.
**Vulnerability Analysis** agents work in parallel, each specializing in one category. They combine source code knowledge with recon data to hypothesize specific vulnerabilities and create exploitation queues.
**Exploitation** agents receive the queues and attempt real attacks using browser automation (Playwright) and HTTP requests. Only proven exploits are included in the final report.
## 4. Interpreting Reports
### Severity Levels
| Severity | Definition | Action |
|----------|-----------|--------|
| **Critical** | Direct data breach, RCE, full authentication bypass | Fix immediately, consider taking app offline |
| **High** | Significant data exposure, privilege escalation, stored XSS | Fix within 24-48 hours |
| **Medium** | Limited data exposure, CSRF, reflected XSS, information disclosure | Fix within 1-2 weeks |
| **Low** | Minor information leaks, missing headers, verbose errors | Fix in next sprint |
### Reading a Finding
Each finding in the report includes:
```markdown
## [CRITICAL] SQL Injection in /api/users/search
**Endpoint:** GET /api/users/search?q=
**Parameter:** q
**Type:** Union-based SQL injection
### Proof of Concept
GET /api/users/search?q=' UNION SELECT username,password,NULL FROM users--
### Response Evidence
HTTP/1.1 200 OK
[{"username":"admin","password":"$2b$12$...","3":null}]
### Source Code Reference
File: src/routes/users.ts:42
const results = await db.query(`SELECT * FROM users WHERE name LIKE '%${req.query.q}%'`);
### Remediation
Use parameterized queries:
const results = await db.query('SELECT * FROM users WHERE name LIKE $1', [`%${req.query.q}%`]);
```
### False Positive Identification
Shannon's "no exploit, no report" policy minimizes false positives, but review for:
- **Environment-specific**: Exploit only works in test environment (different DB, debug mode)
- **Already mitigated**: WAF or middleware blocks the attack in production but not staging
- **Intended behavior**: Feature that looks like a vulnerability (e.g., admin search returns all users by design)
- **LLM hallucination**: Report claims a vulnerability but the PoC doesn't actually demonstrate impact
Always verify the PoC manually for Critical/High findings before filing tickets.
## 4a. Remediation Playbooks
A finding is only closed when the secure pattern is in place *and* a regression test proves the PoC no longer works. Below are framework-specific fixes for each class Shannon covers. Pair each fix with a test (see §6 — Regression Testing).
### SQL / NoSQL Injection
- **Root cause:** untrusted input concatenated into a query.
- **Fix:** always parameterize; never build query strings. Prefer a query builder / ORM with bound parameters.
```js
// ❌ const r = await db.query(`SELECT * FROM users WHERE name LIKE '%${q}%'`);
// ✅ Node + pg
const r = await db.query('SELECT * FROM users WHERE name LIKE $1', [`%${q}%`]);
// ✅ Mongo: never pass req.body/req.query straight into a filter — cast & whitelist:
await User.find({ name: String(q) }); // reject objects so {$ne:null} can't slip in
```
- Also: least-privilege DB user (no DDL), reject `$`/`.` keys in JSON filters, validate types at the edge (zod/Joi).
### Cross-Site Scripting (XSS)
- **Root cause:** untrusted data rendered into HTML/JS/attribute/URL context without context-correct encoding.
- **Fix:** rely on the framework's auto-escaping; never bypass it with raw-HTML sinks on untrusted data.
```jsx
// ✅ React/Vue/Svelte auto-escape {value}. The danger is the escape hatch:
// ❌ <div dangerouslySetInnerHTML={{ __html: userInput }} />
// ✅ If you MUST render HTML, sanitize first:
import DOMPurify from 'dompurify';
<div dangerouslySetInnerHTML={{ __html: DOMPurify.sanitize(userInput) }} />
```
- Defense in depth: a strict `Content-Security-Policy` (no `unsafe-inline`; use nonces/hashes), `HttpOnly`+`Secure`+`SameSite` cookies, and URL-scheme allowlists (`https:` only) to block `javascript:` sinks.
### SSRF (Server-Side Request Forgery)
- **Root cause:** server fetches a user-supplied URL.
- **Fix:** allowlist destinations; resolve the host and reject private/link-local ranges; disable redirects to new hosts.
```js
import dns from 'node:dns/promises';
import ipaddr from 'ipaddr.js';
async function assertPublicUrl(raw) {
const u = new URL(raw);
if (!['http:', 'https:'].includes(u.protocol)) throw new Error('scheme');
const { address } = (await dns.lookup(u.hostname));
const r = ipaddr.parse(address).range(); // 'private' | 'loopback' | 'linkLocal' | ...
if (['private','loopback','linkLocal','uniqueLocal','reserved'].includes(r)) throw new Error('blocked');
return u; // fetch with redirect: 'manual', re-check each hop
}
```
- Cloud: enforce **IMDSv2** so a basic SSRF can't read instance credentials; egress-firewall the service.
### Command Injection
- **Root cause:** user input reaches a shell.
- **Fix:** never invoke a shell with interpolated input; pass an argv array to `execFile`/`spawn` with `shell:false`.
```js
// ❌ exec(`convert ${file} out.png`); // shell metacharacters → RCE
import { execFile } from 'node:child_process';
execFile('convert', [file, 'out.png'], { shell: false }, cb); // ✅ args never parsed by a shell
```
- Allowlist the binary and validate args (e.g. filename matches `^[\w.-]+$`).
### Broken Authorization (IDOR / privilege escalation)
- **Root cause:** the handler trusts a client-supplied id/role without checking the *current* user owns or may access it.
- **Fix:** enforce object-level authz on every read/write, server-side, from the session — not from a request field.
```js
// ❌ const doc = await Doc.findById(req.params.id); // any id → anyone's doc
// ✅ scope the query to the authenticated principal
const doc = await Doc.findOne({ _id: req.params.id, ownerId: req.user.id });
if (!doc) return res.sendStatus(404); // 404, not 403 (don't confirm existence)
// role checks come from the verified session/JWT claims, never from req.body.role
```
- Use centralized policy (e.g. CASL/OPA), deny-by-default, and avoid sequential/guessable ids (use UUIDs).
### Broken Authentication (sessions, JWT, brute force)
- **Fix:** verify JWTs with a pinned algorithm (`algorithms:['RS256']`) and reject `alg:none`; rotate the session id on login (kills fixation); short-lived access tokens + rotating refresh tokens; bcrypt/argon2 for passwords; rate-limit + lockout/backoff on login.
```js
jwt.verify(token, PUBLIC_KEY, { algorithms: ['RS256'] }); // never accept attacker-chosen alg
req.session.regenerate(() => {/* set new session after successful auth */});
```
### Insecure Deserialization
- **Root cause:** untrusted bytes turned into objects that can execute code on construct.
- **Fix:** don't deserialize untrusted data into rich objects. Use a data-only format (JSON) and validate against a schema; never `node-serialize`/Java native `readObject`/Python `pickle.loads`/PHP `unserialize` on user input.
```js
const data = JSON.parse(body); // data only, no behavior
const safe = MySchema.parse(data); // zod: reject unexpected shape/types
```
### HTTP Request Smuggling
- **Root cause:** front-end and back-end disagree on request boundaries (`Content-Length` vs `Transfer-Encoding`).
- **Fix:** mostly an infra fix — use HTTP/2 end-to-end or a proxy that normalizes/rejects ambiguous framing; reject requests containing both `Content-Length` and `Transfer-Encoding`; keep proxy and origin on the same HTTP version and patched. Validate with `smuggler`/`h2csmuggler` (see §7).
### Security Misconfiguration / headers
- **Fix:** ship secure defaults — `helmet()` (Express) or framework equivalents — setting HSTS, `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY` (or CSP `frame-ancestors`), a strict CSP, and disabling stack traces / `X-Powered-By` in production.
## 5. CI/CD Integration
> **Never run a full active pentest in CI without a committed scope config.** The job below requires `shannon.yaml` (`CONFIG=`) so the destructive-endpoint denylist, rate limits, and isolated test accounts always apply. A scan with no scope rules can hammer auth endpoints, trigger SMS/email/billing, and lock accounts even in a "test" stack.
**Networking note (Linux runners).** `host.docker.internal` does **not** resolve by default on GitHub-hosted `ubuntu-latest`. There are two robust options:
- **App published to the runner host** (e.g. `docker compose ... up -d` mapping `3000:3000`): target `http://localhost:3000` and run Shannon directly on the host. This is what the workflow below does.
- **Shannon itself running in Docker**, needing to reach the host: start that container with `--add-host=host.docker.internal:host-gateway` (Docker ≥ 20.10), then target `http://host.docker.internal:3000`. Put app + Shannon on a shared user-defined network and address the app by its service name instead, when possible.
### CI scope config (commit this next to the workflow)
```yaml
# .github/shannon-ci.yaml — mandatory scope for automated runs
auth:
login_url: /login
credentials:
# Disposable accounts seeded ONLY in the ephemeral CI database.
# Real user accounts must never appear here.
- username: ci-user@test.local
password: ${CI_TEST_USER_PW} # injected from CI secret, not committed
role: user
rules:
avoid:
- /api/admin/** # privileged / destructive admin actions
- /api/billing/** # never trigger real charges/refunds
- /api/payments/**
- "**/delete*" # bulk-delete style endpoints
- "**/export*" # data-exfil heavy endpoints
- /logout # don't log the test session out
- /api/notifications/** # don't fan out email/SMS/push
focus:
- /api/**
- /dashboard/**
limits:
max_requests_per_second: 5 # cap noise/cost; keep under app rate limits
max_agents: 3
max_steps: 40
```
### Pre-Deploy Security Gate
```yaml
# .github/workflows/security.yml
name: Security Pentest
on:
pull_request:
branches: [main]
schedule:
- cron: '0 2 * * 1' # Weekly Monday 2am
jobs:
pentest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Start isolated test application
run: docker compose -f docker-compose.test.yml up -d
# docker-compose.test.yml maps "3000:3000" and seeds a throwaway DB (see §8)
- name: Wait for app
run: |
for i in $(seq 1 30); do
curl -fsS http://localhost:3000/health && break
sleep 2
done
- name: Run Shannon pentest (scoped)
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
CI_TEST_USER_PW: ${{ secrets.CI_TEST_USER_PW }}
run: |
git clone --depth 1 https://github.com/KeygraphHQ/shannon.git /tmp/shannon
cp "$GITHUB_WORKSPACE/.github/shannon-ci.yaml" /tmp/shannon/shannon.yaml
cd /tmp/shannon
# localhost works because the app is published on the runner host (see networking note above).
# CONFIG is REQUIRED — never run an unscoped active pentest in CI.
./shannon start URL=http://localhost:3000 REPO=pr-${{ github.event.pull_request.number }} CONFIG=shannon.yaml
- name: Reset / tear down test data
if: always()
run: docker compose -f docker-compose.test.yml down -v # -v drops the throwaway DB volume
- name: Check for critical findings
run: |
# Confirm the actual report filename against the current release if this path changes.
REPORT="/tmp/shannon/workspaces/pr-${{ github.event.pull_request.number }}/comprehensive_security_assessment_report.md"
if [ ! -f "$REPORT" ]; then
echo "::error::Security report not found at $REPORT — pentest may have failed. Blocking deploy."
exit 1
fi
# Count severity headings (format: ## [CRITICAL] or ## [HIGH])
CRITICAL_COUNT=$(grep -c '^##.*\[CRITICAL\]' "$REPORT" || true)
HIGH_COUNT=$(grep -c '^##.*\[HIGH\]' "$REPORT" || true)
if [ "$CRITICAL_COUNT" -gt 0 ]; then
echo "::error::$CRITICAL_COUNT critical findings — review and manually validate the report before merging."
cat "$REPORT"
exit 1
fi
if [ "$HIGH_COUNT" -gt 0 ]; then
echo "::warning::$HIGH_COUNT high-severity findings. Manual validation required (findings are leads, not verdicts)."
fi
- name: Upload report
if: always()
uses: actions/upload-artifact@v4
with:
name: security-report
path: /tmp/shannon/workspaces/pr-*/comprehensive_security_assessment_report.md
```
### Integration Patterns
| Pattern | When | Relative cost | Coverage |
|---------|------|---------------|----------|
| **Full pentest on PR** | Every pull request to main | High (5 categories × full pipeline) | Complete |
| **Weekly scheduled** | Cron job on staging | High × runs/month | Complete |
| **Quick single-category** | Pre-merge for risky changes | Low (one category) | One vuln type |
| **Pre-release gate** | Before production deploy | High | Complete |
### Cost Management
These runs are LLM-token-billed, so the dollar cost is whatever your provider charges times the tokens consumed — it moves with model choice and provider pricing and is **not** a fixed per-run number. Estimate it from the drivers, then read the actual spend off your provider dashboard after the first run and calibrate.
**Cost ≈ Σ over agents of `(input + output tokens) × model price/token`, scaled by retries.** The knobs that move tokens:
| Driver | Effect on cost | Lever |
|--------|---------------|-------|
| **Model** | Dominant — frontier models cost multiples of small/fast ones per token | Pick the cheapest model that still finds real bugs; verify the live per-token price on the provider's pricing page |
| **Endpoints in scope** | ~linear | `focus`/`avoid` rules in `shannon.yaml` |
| **Vuln categories** | ~linear (5 parallel agents at full coverage) | Run a single category for targeted checks |
| **Max agents / max steps** | ~linear | `limits.max_agents`, `limits.max_steps` |
| **Retries / re-runs** | multiplies the above | Use named workspaces to resume, not restart |
```bash
# Estimate BEFORE a big run: dry-cost a single category on a few endpoints first,
# read the spend from your provider dashboard, then extrapolate:
# est_full ≈ pilot_cost × (total_endpoints / pilot_endpoints) × (categories / 1)
# Pull current per-token prices from the provider's pricing page — never hardcode them.
```
Cost-reduction strategies:
1. Narrow scope with `CONFIG` (`focus`/`avoid` rules) — biggest lever.
2. Run single-category scans for targeted, post-change checks.
3. Cap `max_agents` / `max_steps` / requests-per-second in `shannon.yaml`.
4. Use named workspaces to resume interrupted scans instead of paying for a full re-run.
5. Schedule full scans weekly; run quick single-category scans on PRs.
## 6. Post-Pentest Workflow
### Triage → Fix → Verify
```
1. TRIAGE (Day 0)
├── Read the full report
├── Verify all Critical/High PoCs manually
├── Create tickets with severity labels
├── Assign owners and deadlines
└── Notify stakeholders for Critical findings
2. FIX (Day 1-14, based on severity)
├── Critical: same day
├── High: within 48 hours
├── Medium: within 2 weeks
└── Low: next sprint
3. VERIFY (After fix)
├── Re-run Shannon against the same workspace
│ └── ./shannon start URL=<url> REPO=<same-name> WORKSPACE=verify
├── Completed agents are skipped (resumable)
├── Confirm the PoC no longer works
└── Update ticket status
4. DOCUMENT
├── Archive the report
├── Update security runbook with new patterns
├── Add regression tests for each finding
└── Schedule next pentest
```
### Regression Testing
For each finding, create a permanent test:
```javascript
// tests/security/sql-injection.test.ts
describe('SQL Injection regression', () => {
it('should not be vulnerable to union-based injection in /api/users/search', async () => {
const res = await request(app)
.get("/api/users/search")
.query({ q: "' UNION SELECT username,password,NULL FROM users--" });
// Should NOT return other users' data
expect(res.body).not.toEqual(
expect.arrayContaining([
expect.objectContaining({ username: 'admin' })
])
);
});
it('should use parameterized queries', async () => {
const res = await request(app)
.get("/api/users/search")
.query({ q: "test" });
expect(res.status).toBe(200);
// Normal search should still work
});
});
```
## 7. What Shannon Doesn't Cover
Supplement with manual testing or other tools:
| Gap | Alternative |
|-----|------------|
| Business logic flaws | Manual review, threat modeling |
| Mobile app testing | OWASP MAS, Frida, Objection |
| Infrastructure/cloud | ScoutSuite, Prowler, CloudSploit |
| Container security | Trivy, Grype, Docker Bench |
| API rate limiting | Custom load testing (k6, Artillery) |
| GraphQL deep testing | InQL, graphql-cop |
| WebSocket testing | OWASP ZAP WebSocket plugin |
| Dependency vulnerabilities | npm audit, Snyk, Socket.dev |
| Secrets in source code | TruffleHog, GitLeaks, detect-secrets |
| CORS misconfiguration | CORScanner, manual review |
| HTTP request smuggling | smuggler, h2csmuggler |
| Race conditions / TOCTOU | Turbo Intruder, manual testing |
| Cache poisoning | Web Cache Deception Scanner |
| Host header injection | Manual review of password reset flows |
### Complementary Tool Stack
```bash
# Run alongside Shannon for full coverage:
# Dependency scanning (production deps only)
npm audit --omit=dev # `--production` is deprecated; use `--omit=dev`
pnpm audit --prod # pnpm equivalent
yarn npm audit --environment production # Yarn Berry (v2+); classic: `yarn audit --groups dependencies`
npx snyk test
# Secret detection
trufflehog git file://. --only-verified
# Container scanning
trivy image myapp:latest
# Infrastructure
prowler aws --severity critical high
# API fuzzing
schemathesis run http://localhost:3000/openapi.json
```
## 8. Safe Testing Practices
### Rules of Engagement
```
DO:
✓ Only test applications you own or have written authorization to test
✓ Use staging/test environments, never production
✓ Create dedicated test accounts with known credentials
✓ Set scope rules to avoid destructive endpoints
✓ Review reports before sharing (may contain sensitive data)
✓ Keep API keys secure (Shannon uses significant API credits)
DON'T:
✗ Point Shannon at production systems
✗ Test third-party services without explicit written permission
✗ Share reports containing valid credentials or PII
✗ Run without scope rules on apps with destructive endpoints
✗ Ignore the cost — monitor API spend during runs
```
### Safe-test controls for side-effecting attacks
Some attack classes have real-world blast radius even in staging. Apply these controls *before* enabling them — and prefer the scope `avoid` rules in §5 when in doubt.
| Attack class | Hazard | Required controls |
|--------------|--------|-------------------|
| **Brute force / password spraying** | Locks accounts; floods auth; triggers WAF/SIEM alerts | Use disposable accounts you can re-create; cap attempts per account *below* the lockout threshold (e.g. 3 if lockout is 5); cap requests/sec (`limits.max_requests_per_second`); raise or disable lockout for the dedicated test users only; never spray real usernames |
| **Credential stuffing** | Lateral lockouts; alerts the real users whose emails are tried | Test ONLY against seeded fake accounts in an isolated DB; never load a real breach corpus against a shared environment; disable any "new device" email on the test tenant |
| **2FA / OTP bypass & enumeration** | Burns SMS/email budget; spams real recipients; locks 2FA | Use TOTP test secrets you control (not SMS) — see the `totp:` block in §2; if SMS/email is unavoidable, route it to a catch-all mailbox / SMS sandbox and rate-limit; never enumerate against real phone numbers |
| **SSRF / cloud-metadata probing** | Can pivot into real internal services or live cloud creds | Run only in an isolated network with NO route to production VPCs or `169.254.169.254`; in cloud CI, enforce IMDSv2 and scope the runner's IAM role to nothing; assert the metadata endpoint is unreachable from the test host before probing |
| **Email / SMS / push triggers** (signup, reset, invite, notify) | Real messages to real people; sender-reputation damage | Add `/api/notifications/**`, invite, and reset flows to `avoid`, OR point the test env's mail/SMS provider at a sandbox (e.g. a catch-all inbox); verify NODE_ENV/test config routes nothing to the real provider |
| **Payment / billing / refund endpoints** | Real charges, refunds, payouts, webhooks | Always `avoid` these unless the env uses the payment provider's *test mode* keys with test cards; assert the publishable key is a test key before running; never test billing against live keys |
**Pre-run assertions (fail closed).** Bake these checks into the test harness so a misconfigured target aborts the run instead of doing damage:
```bash
# Refuse to run unless we're clearly NOT in production
[ "$NODE_ENV" = "test" ] || { echo "Refusing: NODE_ENV is not 'test'"; exit 1; }
case "$TARGET_URL" in *prod*|*www.*) echo "Refusing: target looks like production"; exit 1;; esac
# Cloud metadata must be unreachable from the test host before SSRF probing
curl -s --max-time 2 http://169.254.169.254/ >/dev/null \
&& { echo "Refusing: cloud metadata endpoint is reachable from test host"; exit 1; } || true
```
### Test Environment Setup
```yaml
# docker-compose.test.yml — isolated test environment
services:
app:
build: .
environment:
- NODE_ENV=test
- DATABASE_URL=postgres://test:test@db:5432/testdb
ports:
- "3000:3000"
networks:
- pentest-net
db:
image: postgres:16
environment:
- POSTGRES_DB=testdb
- POSTGRES_USER=test
- POSTGRES_PASSWORD=test
networks:
- pentest-net
networks:
pentest-net:
driver: bridge
# Isolated network — no access to host or internet
```