claude-code-templates

# Security Validation System Comprehensive security validation for Claude Code Templates components based on industry standards (npm 2025, PyPI, GitHub Package Registry, SLSA Framework). ## Table of Contents - [Overview](#overview) - [Quick Start](#quick-start) - [Validation Tiers](#validation-tiers) - [CLI Usage](#cli-usage) - [CI/CD Integration](#cicd-integration) - [Architecture](#architecture) - [Error Codes](#error-codes) - [Best Practices](#best-practices) ## Overview The Security Validation System ensures that all components in the repository are safe, trustworthy, and meet quality standards before distribution. It validates components across 5 tiers: 1. **Structural Validation** - YAML frontmatter, file format, encoding 2. **Integrity Validation** - SHA256 hashing, tamper detection 3. **Semantic Validation** - Prompt injection, jailbreaks, malicious patterns 4. **Reference Validation** - URLs, external links, SSRF prevention 5. **Provenance Validation** - Author metadata, repository info, Git history ### Key Features - ✅ **372+ components validated** - Agents, commands, settings, hooks - 🔒 **10+ security patterns detected** - Jailbreaks, prompt injection, XSS - 📊 **Trust scoring (0-100)** - Overall security score per component - 🔗 **Hash tracking** - SHA256 hashes for tamper detection - ⚡ **Fast validation** - Completes in ~20 seconds for 372 components - 🤖 **GitHub Actions integration** - Automatic PR validation ## Quick Start ### Run Security Audit Locally ```bash # From repository root cd cli-tool npm run security-audit # With verbose output npm run security-audit:verbose # Generate JSON report npm run security-audit:json ``` ### Example Output ``` 🔒 Claude Code Templates - Security Audit ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 📁 Scanning components directory... Found 372 components 🔍 Running security validation... 📊 Validation Summary: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Total components: 372 ✅ Passed: 123 ❌ Failed: 249 ⚠️ Warnings: 938 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` ## Validation Tiers ### 1. Structural Validation Validates component structure, format, and metadata. **Checks:** - ✅ Valid YAML frontmatter - ✅ Required fields present (name, description, tools) - ✅ File size limits (< 100KB) - ✅ UTF-8 encoding - ✅ Valid tool names (Read, Write, Edit, Bash, Glob, Grep, *) - ✅ Valid model names (sonnet, opus, haiku) - ✅ Section count (< 20 sections) **Error Codes:** `STRUCT_E001` - `STRUCT_E010` **Warning Codes:** `STRUCT_W001` - `STRUCT_W011` ### 2. Integrity Validation Ensures components haven't been tampered with. **Checks:** - ✅ Generate SHA256 hash - ✅ Track hash in registry (`.claude/security/component-hashes.json`) - ✅ Detect content changes - ✅ Validate semantic versioning **Error Codes:** `INT_E001` - `INT_E003` **Info Codes:** `INT_I001` - `INT_I005` ### 3. Semantic Validation Detects malicious content and prompt injection attempts. **Dangerous Patterns Detected:** - 🚫 **Prompt Injection:** - "ignore previous instructions" - "ignore all previous instructions" - "disregard all prior directives" - 🚫 **Jailbreaks:** - "you are now a..." - "system prompt" - "developer instructions" - 🚫 **Code Execution:** - "execute the following code" - "run this command" - 🚫 **Credential Harvesting:** - "fetch...token/key/password" - Hardcoded API keys, passwords - 🚫 **XSS & Injection:** - `<script>`, `<iframe>` - `javascript:`, `data:` - Event handlers (onclick, onerror) - 🚫 **Dangerous Commands:** - `rm -rf /` - Fork bombs: `:(){ :|:& };:` - `dd if=/dev/zero of=/dev/sda` **Error Codes:** `SEM_E001` - `SEM_E008` **Warning Codes:** `SEM_W001` - `SEM_W004` ### 4. Reference Validation Validates external URLs and prevents SSRF attacks. **Checks:** - ✅ Extract all URLs from markdown - ✅ Block dangerous protocols (file://, javascript:, data:) - ✅ Detect private IP addresses (127.0.0.1, 10.x.x.x, 192.168.x.x) - ✅ Flag suspicious TLDs (.tk, .ml, .ga, .zip) - ✅ HTTPS enforcement (optional strict mode) **Error Codes:** `REF_E001` - `REF_E005` **Warning Codes:** `REF_W001` - `REF_W003` ### 5. Provenance Validation Validates component authorship and origin. **Checks:** - ✅ Author metadata in frontmatter - ✅ Repository information - ✅ Version tracking - ✅ Git commit SHA, author, date - ✅ Repository platform validation (github.com, gitlab.com) **Error Codes:** `PROV_E001` - `PROV_E003` **Warning Codes:** `PROV_W001` - `PROV_W003` ## CLI Usage ### Basic Commands ```bash # Run validation on all components npm run security-audit # Verbose output with detailed errors npm run security-audit:verbose # Generate JSON report npm run security-audit:json # CI mode (fails on errors) npm run security-audit:ci ``` ### Direct Execution ```bash # From cli-tool directory node src/security-audit.js [options] # Options: # --ci Exit with code 1 on errors (for CI/CD) # --verbose, -v Show detailed validation results # --json Output results as JSON # --output=FILE Save JSON report to file ``` ### Examples ```bash # Local development - show all details npm run security-audit:verbose # Generate report for review npm run security-audit:json > my-report.json # CI/CD pipeline - fail on errors npm run security-audit:ci ``` ## CI/CD Integration ### GitHub Actions Workflow The repository includes a GitHub Actions workflow that automatically validates components on: - ✅ Pull requests modifying components - ✅ Pushes to main branch **File:** `.github/workflows/component-security-validation.yml` ```yaml name: Component Security Validation on: pull_request: paths: - 'cli-tool/components/**' push: branches: - main paths: - 'cli-tool/components/**' jobs: validate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' - name: Install dependencies working-directory: cli-tool run: npm install - name: Run security validation working-directory: cli-tool run: npm run security-audit:ci - name: Upload security report if: always() uses: actions/upload-artifact@v4 with: name: security-report path: cli-tool/security-report.json retention-days: 30 ``` ### PR Comment Bot The workflow automatically posts validation results as PR comments: ```markdown ## 🔒 Security Validation Results **Status:** ❌ Failed ### Summary - Total Components: 15 - Passed: 12 - Failed: 3 - Warnings: 8 ### Failed Components 1. agents/security/penetration-tester.md - Error: Dangerous command detected: `rm -rf /tmp` - Score: 50/100 [View Full Report](artifacts/security-report.json) ``` ## Architecture ### Module Structure ``` cli-tool/src/validation/ ├── ARCHITECTURE.md # Detailed architecture documentation ├── README.md # This file ├── BaseValidator.js # Abstract base class for validators ├── ValidationOrchestrator.js # Coordinates all validators └── validators/ ├── StructuralValidator.js ├── IntegrityValidator.js ├── SemanticValidator.js ├── ReferenceValidator.js └── ProvenanceValidator.js ``` ### Validator Lifecycle ``` 1. Component Scanning ↓ 2. Content Reading ↓ 3. Parallel Validation ├── StructuralValidator ├── IntegrityValidator ├── SemanticValidator ├── ReferenceValidator └── ProvenanceValidator ↓ 4. Result Aggregation ↓ 5. Report Generation ``` ### Scoring Algorithm ```javascript componentScore = ( structuralScore * 0.25 + integrityScore * 0.20 + semanticScore * 0.30 + referencesScore * 0.15 + provenanceScore * 0.10 ) validatorScore = max(0, 100 - (errors * 25) - (warnings * 5)) ``` ## Error Codes ### Structural (STRUCT_*) | Code | Type | Message | |------|------|---------| | `STRUCT_E001` | Error | No YAML frontmatter found | | `STRUCT_E002` | Error | Invalid YAML syntax | | `STRUCT_E003` | Error | Missing required field: {field} | | `STRUCT_E004` | Error | File too large (> 100KB) | | `STRUCT_E010` | Error | Invalid UTF-8 encoding | | `STRUCT_W006` | Warning | Unknown tools specified | | `STRUCT_W011` | Warning | Too many sections (>20) | ### Integrity (INT_*) | Code | Type | Message | |------|------|---------| | `INT_E001` | Error | Content changed since last validation | | `INT_E002` | Error | Invalid semantic version format | | `INT_I001` | Info | Generated SHA256 hash | | `INT_I005` | Info | Component not in registry (new) | ### Semantic (SEM_*) | Code | Type | Message | |------|------|---------| | `SEM_E001` | Error | Prompt injection detected: {pattern} | | `SEM_E002` | Error | Jailbreak attempt detected: {pattern} | | `SEM_E003` | Error | Code execution pattern detected | | `SEM_E004` | Error | Credential harvesting detected | | `SEM_E005` | Error | HTML injection detected: {tag} | | `SEM_E006` | Error | Hardcoded credentials detected | | `SEM_E007` | Error | Dangerous command detected: {command} | ### References (REF_*) | Code | Type | Message | |------|------|---------| | `REF_E001` | Error | Dangerous protocol: {protocol} | | `REF_E002` | Error | Private IP address detected | | `REF_E003` | Error | SSRF attempt via private network | | `REF_W001` | Warning | Suspicious TLD: {tld} | | `REF_W002` | Warning | HTTP URL (should use HTTPS) | ### Provenance (PROV_*) | Code | Type | Message | |------|------|---------| | `PROV_E001` | Error | Missing author information | | `PROV_W001` | Warning | No repository information | | `PROV_W002` | Warning | No version specified | ## Best Practices ### For Component Authors 1. **Include Complete Frontmatter:** ```yaml --- name: my-agent description: Clear description author: Your Name version: 1.0.0 repository: https://github.com/user/repo tools: - Read - Write model: sonnet --- ``` 2. **Avoid Dangerous Patterns:** - Don't include executable shell commands - No hardcoded credentials or API keys - Avoid instructing Claude to ignore security rules - Use HTTPS URLs only 3. **Keep Components Concise:** - File size < 50KB (max 100KB) - Sections < 15 (max 20) - Clear, focused instructions 4. **Test Before Submitting:** ```bash npm run security-audit:verbose ``` ### For Repository Maintainers 1. **Review PR Validation Results:** - Check GitHub Actions status - Review security scores - Investigate failed components 2. **Monitor Hash Changes:** - Unexpected hash changes indicate tampering - Review `.claude/security/component-hashes.json` 3. **Update Validation Rules:** - Add new dangerous patterns to SemanticValidator - Adjust scoring thresholds if needed - Keep validators up to date 4. **Regular Audits:** ```bash npm run security-audit:json # Review security-report.json ``` ## Integration with Component Generation The security validation system integrates with `scripts/generate_components_json.py`: 1. **Automatic Validation** - Runs before generating components.json 2. **Metadata Inclusion** - Security scores, hashes, validation status 3. **Download Statistics** - Combined with Supabase analytics **Generated metadata:** ```json { "name": "frontend-developer", "security": { "validated": true, "valid": true, "score": 96, "errorCount": 0, "warningCount": 2, "lastValidated": "2025-10-15T20:07:37.583Z", "hash": "eb5f9f4978b71ba06f129c73f5573a58...", "validators": { "structural": true, "integrity": true, "semantic": true, "references": true, "provenance": true } } } ``` ## Troubleshooting ### Common Issues **Issue:** Components directory not found ```bash # Solution: Run from correct directory cd cli-tool npm run security-audit ``` **Issue:** Hash registry conflicts ```bash # Solution: Clear and regenerate rm -rf .claude/security/component-hashes.json npm run security-audit ``` **Issue:** False positives in semantic validation ```bash # Solution: Review SemanticValidator.js patterns # Adjust regex patterns if needed ``` ### Performance Optimization - Validation of 372 components: ~20 seconds - Parallel validation with async/await - Cached hash registry for faster checks - JSON report generation optimized ## Contributing ### Adding New Validators 1. Extend `BaseValidator` class 2. Implement `validate(component)` method 3. Add to `ValidationOrchestrator` 4. Write comprehensive tests ```javascript class CustomValidator extends BaseValidator { async validate(component) { // Your validation logic if (issue) { this.addError('CUSTOM_E001', 'Description', metadata); } return this.getResults(); } } ``` ### Updating Error Codes 1. Document in `ARCHITECTURE.md` 2. Add to error code tables above 3. Include in test coverage 4. Update this README ## Resources - [ARCHITECTURE.md](./ARCHITECTURE.md) - Detailed architecture - [SLSA Framework](https://slsa.dev/) - Supply chain security - [npm 2025 Security](https://docs.npmjs.com/security) - Package security - [OWASP Top 10](https://owasp.org/www-project-top-ten/) - Security risks ## License MIT - See repository LICENSE file