semgrep-s3-scanner/README.md

Run semgrep scans using rules stored in S3

# Semgrep S3 Scanner

A command-line utility that integrates with your projects to run semgrep scans using rules stored in S3. This tool is designed to be easily installed via npm and configured in your project's `package.json`.

## Installation

```bash
npm install -g semgrep-s3-scanner
```

## Prerequisites

1. Install semgrep:
```bash
# For macOS
brew install semgrep

# For other platforms, see: https://semgrep.dev/docs/getting-started/installation/
```

2. Configure AWS credentials (for S3 access):
```bash
export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key
```

## Usage

Basic usage:
```bash
semgrep-s3-scanner scan
```

With options:
```bash
semgrep-s3-scanner scan \
  --bucket your-bucket-name \
  --prefix rules/ \
  --target ./src \
  --output report.json \
  --format json
```

### Options

- `-b, --bucket <bucket>`: S3 bucket name (default: 'semgrep-rules')
- `-p, --prefix <prefix>`: Rules prefix in S3 (default: 'rules/')
- `-t, --target <target>`: Target directory or file to scan (default: '.')
- `-o, --output <output>`: Output file for the report (default: 'semgrep-report.json')
- `-f, --format <format>`: Output format (json or sarif) (default: 'json')

## Integration with CI/CD

Add to your CI pipeline:

```yaml
steps:
  - name: Run Semgrep Scan
    run: |
      npm install -g semgrep-s3-scanner
      semgrep-s3-scanner scan --output scan-results.json
```

## Development

1. Clone the repository
2. Install dependencies:
```bash
npm install
```

3. Build the project:
```bash
npm run build
```

4. Run tests:
```bash
npm test
```

## License

MIT

## Command Line Options

- `-d, --directory <path>`: Directory to scan (required)
- `-b, --bucket <name>`: S3 bucket name (required, can be set via SEMGREP_S3_BUCKET env variable)
- `-r, --rules-path <path>`: Path to rules in S3 bucket (default: 'rules/')
- `-o, --output <path>`: Output path for the report (default: 'semgrep-report.md')

## S3 Rules Format

The rules should be stored in your S3 bucket as a YAML file. Example:

```yaml
rules:
  - id: example-rule
    pattern: $X = $Y
    message: "Found assignment"
    languages: [python]
    severity: WARNING
```

## Output

The tool generates two files:
1. `semgrep-results.json` - Raw semgrep results
2. `semgrep-report.md` - Human-readable markdown report

## Requirements

- Node.js 14+
- AWS credentials configured
- semgrep CLI installed