skills-ws/skills/security-pentester/references/5-ci-cd-integration.md

87 agent skills for AI coding assistants — marketing, growth, web3, dev, design & operations. Built for OpenClaw, Claude Code, Cursor, and Codex.

## Contents

- 5. CI/CD Integration
- CI scope config (commit this next to the workflow)
- Pre-Deploy Security Gate
- Integration Patterns
- Cost Management

## 5. CI/CD Integration

> **Never run a full active pentest in CI without a committed scope config.** The job below requires `shannon.yaml` (`-c`) so the destructive-endpoint denylist, rate limits, and isolated test accounts always apply. A scan with no scope rules can hammer auth endpoints, trigger SMS/email/billing, and lock accounts even in a "test" stack.

**Networking note (Linux runners).** `host.docker.internal` does **not** resolve by default on GitHub-hosted `ubuntu-latest`. There are two robust options:
- **App published to the runner host** (e.g. `docker compose ... up -d` mapping `3000:3000`): target `http://localhost:3000` and run Shannon directly on the host. This is what the workflow below does.
- **Shannon itself running in Docker**, needing to reach the host: start that container with `--add-host=host.docker.internal:host-gateway` (Docker ≥ 20.10), then target `http://host.docker.internal:3000`. Put app + Shannon on a shared user-defined network and address the app by its service name instead, when possible.

### CI scope config (commit this next to the workflow)

```yaml
# .github/shannon-ci.yaml — mandatory scope for automated runs
auth:
  login_url: /login
  credentials:
    # Disposable accounts seeded ONLY in the ephemeral CI database.
    # Real user accounts must never appear here.
    - username: ci-user@test.local
      password: ${CI_TEST_USER_PW}      # injected from CI secret, not committed
      role: user
rules:
  avoid:
    - /api/admin/**            # privileged / destructive admin actions
    - /api/billing/**          # never trigger real charges/refunds
    - /api/payments/**
    - "**/delete*"             # bulk-delete style endpoints
    - "**/export*"             # data-exfil heavy endpoints
    - /logout                  # don't log the test session out
    - /api/notifications/**    # don't fan out email/SMS/push
  focus:
    - /api/**
    - /dashboard/**
limits:
  max_requests_per_second: 5   # cap noise/cost; keep under app rate limits
  max_agents: 3
  max_steps: 40
```

### Pre-Deploy Security Gate

```yaml
# .github/workflows/security.yml
name: Security Pentest
on:
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 2 * * 1'  # Weekly Monday 2am

jobs:
  pentest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Start isolated test application
        run: docker compose -f docker-compose.test.yml up -d
        # docker-compose.test.yml maps "3000:3000" and seeds a throwaway DB (see §8)

      - name: Wait for app
        run: |
          for i in $(seq 1 30); do
            curl -fsS http://localhost:3000/health && break
            sleep 2
          done

      - name: Run Shannon pentest (scoped)
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          CI_TEST_USER_PW: ${{ secrets.CI_TEST_USER_PW }}
        run: |
          # ANTHROPIC_API_KEY is read straight from the environment (set in env: above),
          # so no interactive `setup` step is needed in CI.
          # localhost works because the app is published on the runner host (see networking note above).
          # -c is REQUIRED: never run an unscoped active pentest in CI. The repo is mounted read-only.
          npx @keygraph/shannon start \
            -u http://localhost:3000 \
            -r "$GITHUB_WORKSPACE" \
            -w pr-${{ github.event.pull_request.number }} \
            -c "$GITHUB_WORKSPACE/.github/shannon-ci.yaml"

      - name: Reset / tear down test data
        if: always()
        run: docker compose -f docker-compose.test.yml down -v   # -v drops the throwaway DB volume

      - name: Check for critical findings
        run: |
          # Confirm the actual report filename against the current release if this path changes.
          # npx mode writes workspaces under ~/.shannon/workspaces/<name>/; the final report is at the workspace root.
          REPORT="$HOME/.shannon/workspaces/pr-${{ github.event.pull_request.number }}/Security-Assessment-Report.md"
          if [ ! -f "$REPORT" ]; then
            echo "::error::Security report not found at $REPORT — pentest may have failed. Blocking deploy."
            exit 1
          fi
          # Count severity headings (format: ## [CRITICAL] or ## [HIGH])
          CRITICAL_COUNT=$(grep -c '^##.*\[CRITICAL\]' "$REPORT" || true)
          HIGH_COUNT=$(grep -c '^##.*\[HIGH\]' "$REPORT" || true)
          if [ "$CRITICAL_COUNT" -gt 0 ]; then
            echo "::error::$CRITICAL_COUNT critical findings — review and manually validate the report before merging."
            cat "$REPORT"
            exit 1
          fi
          if [ "$HIGH_COUNT" -gt 0 ]; then
            echo "::warning::$HIGH_COUNT high-severity findings. Manual validation required (findings are leads, not verdicts)."
          fi

      - name: Upload report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: security-report
          path: ~/.shannon/workspaces/pr-*/Security-Assessment-Report.md
```

### Integration Patterns

| Pattern | When | Relative cost | Coverage |
|---------|------|---------------|----------|
| **Full pentest on PR** | Every pull request to main | High (5 categories × full pipeline) | Complete |
| **Weekly scheduled** | Cron job on staging | High × runs/month | Complete |
| **Quick single-category** | Pre-merge for risky changes | Low (one category) | One vuln type |
| **Pre-release gate** | Before production deploy | High | Complete |

### Cost Management

These runs are LLM-token-billed, so the dollar cost is whatever your provider charges times the tokens consumed — it moves with model choice and provider pricing and is **not** a fixed per-run number. Estimate it from the drivers, then read the actual spend off your provider dashboard after the first run and calibrate.

**Cost ≈ Σ over agents of `(input + output tokens) × model price/token`, scaled by retries.** The knobs that move tokens:

| Driver | Effect on cost | Lever |
|--------|---------------|-------|
| **Model** | Dominant — frontier models cost multiples of small/fast ones per token | Pick the cheapest model that still finds real bugs; verify the live per-token price on the provider's pricing page |
| **Endpoints in scope** | ~linear | `focus`/`avoid` rules in `shannon.yaml` |
| **Vuln categories** | ~linear (5 parallel agents at full coverage) | Run a single category for targeted checks |
| **Max agents / max steps** | ~linear | `limits.max_agents`, `limits.max_steps` |
| **Retries / re-runs** | multiplies the above | Use named workspaces to resume, not restart |

```bash
# Estimate BEFORE a big run: dry-cost a single category on a few endpoints first,
# read the spend from your provider dashboard, then extrapolate:
#   est_full ≈ pilot_cost × (total_endpoints / pilot_endpoints) × (categories / 1)
# Pull current per-token prices from the provider's pricing page — never hardcode them.
```

Cost-reduction strategies:
1. Narrow scope with `CONFIG` (`focus`/`avoid` rules) — biggest lever.
2. Run single-category scans for targeted, post-change checks.
3. Cap `max_agents` / `max_steps` / requests-per-second in `shannon.yaml`.
4. Use named workspaces to resume interrupted scans instead of paying for a full re-run.
5. Schedule full scans weekly; run quick single-category scans on PRs.

---