paqad-ai/runtime/capabilities/security/skills/pentest/SKILL.md

Spec-driven development framework — AI agents that think before they type

---
name: pentest
description: Runs the whole pentest workflow in order, from context and project scripts to a Markdown report with a JSON sidecar, and retests an earlier report. Use when the user asks to pentest an app or repo, run a penetration test against any build, or do a full security audit or security scan, or to retest earlier pentest findings; the workflow is pentest or pentest-retest.
metadata:
  model_tier: reasoning
  cacheable: 'false'
  output_format: markdown
  paqad-meta: skill.meta.yaml
---

## What It Does

Define the exact steps to follow when a pentest is requested so the AI executes the full structured workflow rather than ad-hoc checks.

It runs the security helper skills in the order below and gathers their output into one report. A retest of an earlier report is the Retest mode of this skill.

## Use This When

Run this workflow whenever the user says anything equivalent to:

- "run pentest", "run a penetration test", "pentest the app", "security audit", "run security scan"

Do **not** improvise a pentest. Always follow the steps below in order.

Use the Retest mode below when the workflow is `pentest-retest`.

## Inputs

- The request text, and a target URL when one is available (from the request text, `.env` APP_URL, or `project-profile.yaml`).
- Module docs under `docs/modules/` and tests under `tests/` for the relevant modules.
- The five `pentest-*.sh` scripts in `scripts/` after onboarding.
- For the Retest mode: the source report under `docs/pentest/`. A `.md` path maps to its `.json` sidecar.

## Procedure

Progress is tracked in `.paqad/pentest/runs/<run_id>/progress.json`. The workflow is resumable: if a step is already marked `completed` with the same input hash, skip it.

### Step 1: collect-context

- Load all module documentation from `docs/modules/` for the relevant modules.
- Load test files from `tests/` for the relevant modules.
- Write a `docs-summary.json` and `tests-summary.json` artefact to `.paqad/pentest/runs/<run_id>/artifacts/`.
- Skills (run **in this order**: STRIDE must run first so all other skills can reference the threat inventory):
  1. `stride-threat-model`: enumerate threats per STRIDE category across all modules; write `stride-threats.json`
  2. `permission-boundary-review`: BOLA, function-level auth, mass assignment, resource consumption
  3. `business-logic-abuse-review`: race conditions, workflow abuse, financial manipulation, GraphQL abuse
  4. `input-validation-review`: SSRF, IDOR, mass assignment, injection, file upload, prototype pollution, ReDoS
  5. `auth-mechanism-review`: JWT, session, OAuth/OIDC, brute-force, password storage
  6. `cryptographic-review`: encryption at rest/transit, key management, hashing, PRNG
  7. `logging-monitoring-review`: audit trail, log injection, sensitive data in logs, alerting
  8. `rate-limiting-review`: missing rate limits on auth, bulk, and expensive endpoints

### Step 2: run-project-scripts

Run the following five scripts in parallel (they live in `scripts/` after onboarding):

1. `pentest-audit-dependencies.sh`: scan lockfile for known vulnerabilities via native audit + OSV; check for typosquatting indicators, dependency confusion, and abandoned packages
2. `pentest-scan-secrets.sh`: scan source **and** git history for leaked credentials or secrets (including deleted files and squashed commits)
3. `pentest-enumerate-surface.sh`: enumerate routes, controllers, exposed endpoints; include `/swagger`, `/api-docs`, `/openapi.json`, `/graphql` (introspection query), and WebSocket endpoints
4. `pentest-db-readonly.sh`: verify database interactions are read-only where expected
5. `pentest-scan-git-history.sh`: scan the git object database for secrets in all commits (not just the working tree), covering deleted branches, squashed commits, and amended history

Write all artefacts to `.paqad/pentest/runs/<run_id>/artifacts/`.

- Skills: `dependency-advisory-triage`

### Step 2.5: generate-local-attack-playbook

For each suspected finding category identified in Steps 1–2, generate `docs/pentest/<timestamp>-local-playbook.md` with a concrete local test command per category. This is the **"break locally first"** step: every finding gets a curl command or script template the developer can run against `localhost` to confirm or refute it.

Example playbook entries:

```bash
# IDOR: swap user ID in path
# Expected: 403 Forbidden. If 200 with another user's data → confirmed IDOR (WSTG-AUTHZ-04).
curl -s -H "Authorization: Bearer $LOW_PRIV_TOKEN" \
  "http://localhost:8000/api/users/2/profile"

# Race condition: concurrent requests (use HTTP/2 single-packet for reliable timing)
# Expected: only 1 success. Multiple 200s → race condition confirmed (WSTG-BUSL-09).
for i in {1..20}; do
  curl -s -X POST -H "Authorization: Bearer $TOKEN" \
    "http://localhost:8000/api/vouchers/PROMO10/redeem" &
done; wait
# For tighter timing: use Burp Turbo Intruder or send all requests in one TCP packet (HTTP/2).

# Mass assignment: inject privileged fields
# Expected: extra fields ignored or 400. If role/is_admin changes → confirmed (WSTG-INPV-06).
curl -s -X PATCH -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"name":"test","is_admin":true,"role":"admin","balance":99999}' \
  "http://localhost:8000/api/profile"

# SSRF: AWS metadata probe
# Expected: blocked or error. If metadata returned → SSRF confirmed (WSTG-INPV-17).
curl -s -X POST -H "Content-Type: application/json" \
  -d '{"url":"http://169.254.169.254/latest/meta-data/"}' \
  "http://localhost:8000/api/preview"

# JWT alg:none bypass
# 1. Split your valid JWT: header.payload.signature
# 2. Re-encode header with "alg":"none" (base64url, no padding)
# 3. Send: <new_header>.<original_payload>. (empty signature: note trailing dot)
# Expected: 401. If 200 → alg:none not rejected (WSTG-SESS-10).
curl -s -H "Authorization: Bearer <forged_alg_none_token>" \
  "http://localhost:8000/api/me"

# GraphQL introspection
# Expected: disabled in production config. If full schema returned → information disclosure.
curl -s -X POST -H "Content-Type: application/json" \
  -d '{"query":"{ __schema { types { name fields { name } } } }"}' \
  "http://localhost:8000/graphql"

# Open redirect
# Expected: only relative paths or trusted domains accepted. If external redirect → confirmed.
curl -sI "http://localhost:8000/auth/callback?redirect=https://evil.com"

# Security headers check
# Expected: all security headers present.
curl -sI "http://localhost:8000/" | grep -iE "strict-transport|x-frame|content-security|x-content-type"
```

**Important:** This playbook is **generated only: never auto-executed.** All commands are templates; substitute `$TOKEN`, `$LOW_PRIV_TOKEN`, and endpoint paths with values from the target project. Never run these against a production system.

### Step 3: live-validation

- If a target URL is available (from the request text, `.env` APP_URL, or `project-profile.yaml`), run `pentest-runtime-checks.sh` against it.
- The runtime checks script must cover:
  - Base reachability (HTTP 200 on home/health)
  - Security header audit (HSTS, X-Frame-Options, CSP, X-Content-Type-Options, Referrer-Policy)
  - CORS header probe on at least one authenticated endpoint
  - Error disclosure probe (trigger a 404 and a malformed request; check for stack traces in response)
  - Open redirect probe on known redirect/callback parameters
  - Sensitive file exposure (`/.env`, `/.git/config`, `/phpinfo.php`)
  - GraphQL introspection probe (`{ __schema { types { name } } }`)
  - Rate limit probe on login endpoint (10 rapid requests; check for 429 response)
- If no URL is found, record the check as blocked and continue.
- Skills: `runtime-surface-probing`

### Step 4: synthesize-findings

Normalise all evidence from steps 1–3 into structured findings:

- **Dependency findings**: from lockfile audit and OSV results
- **Module findings**: from doc and test coverage gaps (all 8 skill categories)
- **Secret findings**: from secret scan and git history scan output
- **Runtime findings**: from live-validation results (if available)

Assign stable `PT-XXXX` IDs. Write a `finding-index.json` to `.paqad/pentest/runs/<run_id>/`.

- Skills: `finding-normalizer`

### Step 5: write-report

- Write a Markdown report to `docs/pentest/<timestamp>.md`
- Write a JSON sidecar to `docs/pentest/<timestamp>.json`
- Include:
  - Stack summary and capability profile
  - **OWASP WSTG Coverage Matrix** (see template below)
  - Ordered findings (critical → high → medium → low)
  - Blocked checks with explanation
  - Remediation priorities
  - Methodology notes
  - Link to the local attack playbook generated in Step 2.5

#### OWASP WSTG Coverage Matrix Template

| WSTG Category     | Test ID          | Description                       | Status            |
| ----------------- | ---------------- | --------------------------------- | ----------------- |
| WSTG-AUTHZ-04     | IDOR             | Insecure Direct Object Reference  | covered / blocked |
| WSTG-INPV-17      | SSRF             | Server-Side Request Forgery       | covered / blocked |
| WSTG-INPV-06      | Mass Assignment  | Broken Object Property Level Auth | covered / blocked |
| WSTG-SESS-10      | JWT              | JSON Web Token Security           | covered / blocked |
| WSTG-SESS-03      | Session Fixation | Session Fixation Attack           | covered / blocked |
| WSTG-ATHN-03      | Brute Force      | Testing for Account Lockout       | covered / blocked |
| WSTG-BUSL-01..12  | Business Logic   | Workflow abuse, race conditions   | covered / blocked |
| WSTG-CRYP-01..03  | Cryptography     | Weak encryption / hashing         | covered / blocked |
| WSTG-INPV-13      | Rate Limiting    | Resource exhaustion / DoS         | covered / blocked |
| WSTG-AUTHZ-01..11 | Authorization    | Privilege escalation suite        | covered / blocked |
| WSTG-ERRH-02      | Logging          | Security logging failures         | covered / blocked |

### Retest mode

Use this mode for the `pentest-retest` workflow. It re-checks every finding of an earlier report against fresh evidence and keeps the original finding IDs. Progress is tracked in `.paqad/pentest/runs/<run_id>/progress.json` in the same way, and a step already marked `completed` with the same input hash is skipped.

1. load-source-report: read the source sidecar named in the request and copy it to `.paqad/pentest/runs/<run_id>/artifacts/source-report.json`. Hand off to the `retest-verification` skill, which loads the source findings.
2. rerun-evidence: run `pentest-retest.sh`, `pentest-audit-dependencies.sh`, `pentest-scan-secrets.sh` and `pentest-runtime-checks.sh` in parallel against the source report. Record a script that is unavailable in `blocked_checks` and continue.
3. evaluate-source-findings: with `retest-verification`, give each source finding one status, `fixed`, `still-open` or `needs-manual-verification`, from the fresh evidence, and write `finding-index.json` to `.paqad/pentest/runs/<run_id>/`. A new gap that is not in the source report belongs in a separate report.
4. write-report: write the retest report to `docs/pentest/retests/<timestamp>-<source>.md` and its JSON sidecar beside it, with the remediation priorities for every finding that is not fixed.

## Output Contract

- A pentest run writes `docs/pentest/<timestamp>.md` and the JSON sidecar `docs/pentest/<timestamp>.json`, with stable `PT-XXXX` finding IDs and the OWASP WSTG coverage matrix from Step 5.
- The local attack playbook is `docs/pentest/<timestamp>-local-playbook.md`.
- Run artefacts stay under `.paqad/pentest/runs/<run_id>/`.
- A retest writes `docs/pentest/retests/<timestamp>-<source>.md` and its `.json` sidecar, and keeps the source finding IDs.

## Escalate / Stop Conditions

- Never skip steps. If a script is unavailable, record it in `blocked_checks` and proceed.
- Do not run destructive operations (no writes to the live database, no mutations via the API).
- Always write both the `.md` report and the `.json` sidecar; downstream retest workflows depend on the sidecar.
- The local attack playbook (Step 2.5) is **generated only**: never execute it automatically.
- For a retest of an existing report, use the `pentest-retest` workflow instead, referencing the original sidecar path to preserve finding IDs.

## Resources

- Helper skills, in the order the Procedure calls them: `stride-threat-model`, `permission-boundary-review`, `business-logic-abuse-review`, `input-validation-review`, `auth-mechanism-review`, `cryptographic-review`, `logging-monitoring-review`, `rate-limiting-review`, `dependency-advisory-triage`, `runtime-surface-probing`, `finding-normalizer` and, for the Retest mode, `retest-verification`.
- `references/helper-map.md`: read it when you need to see which helper a step calls.
- `agents/openai.yaml`