@dreamhorizonorg/sentinel/docs/PROVIDERS.md

Open-source, zero-dependency tool that blocks compromised packages BEFORE download. Built to counter supply chain and credential theft attacks like Shai-Hulud.

# Vulnerability Providers

Sentinel Package Manager integrates with established security providers to check packages against comprehensive vulnerability databases **before installation**.

## 📑 Table of Contents

- [Why Providers?](#-why-providers) - Benefits of using providers
- [Available Providers](#-available-providers) - OSV, GitHub, Snyk
- [Configuration](#-configuration) - How to configure providers
- [Provider Priority](#-provider-priority) - How providers are checked
- [Troubleshooting](#-troubleshooting) - Common issues

> **Related:** See [Data Sources Guide](DATA_SOURCES.md) for all data source types.  
> **Quick Start:** See [Usage Guide](USAGE.md) for basic setup.

---

## 🎯 **Why Providers?**

While the local blacklist covers known compromised packages (like Shai-Hulud), providers give you access to:
- ✅ **Real-time vulnerability data** from industry-leading sources
- ✅ **CVE databases** (Common Vulnerabilities and Exposures)
- ✅ **Comprehensive coverage** beyond custom blacklists
- ✅ **Automatic updates** as new vulnerabilities are discovered

---

## 📊 **Available Providers**

### **1. OSV (Open Source Vulnerabilities)** ✅ Enabled by Default

**Google's comprehensive vulnerability database**

- ✅ **Free** - No API key required
- ✅ **Comprehensive** - Aggregates data from multiple sources
- ✅ **Fast** - Low latency API
- ✅ **Real-time** - Updated continuously

**Configuration:**
```json
{
  "providers": {
    "osv": {
      "enabled": true,
      "timeout": 5000
    }
  }
}
```

**API:** https://osv.dev/docs/

---

### **2. GitHub Security Advisories** ✅ Enabled by Default

**GitHub's security advisory database**

- ✅ **Free** - Public API can be queried without token
- ⚠️ **Token recommended** - Unauthenticated requests may return incomplete results or be rate-limited (60 requests/hour)
- ✅ **Authenticated access** - With token: 5000 requests/hour, complete results
- ✅ **GitHub ecosystem** - Covers npm packages on GitHub
- ⚠️ **Token recommended** - Unauthenticated API may have incomplete data or rate limits (60 req/hr)

**Configuration:**
```json
{
  "providers": {
    "github": {
      "enabled": true,
      "timeout": 5000,
      "token": null  // Optional: GitHub token for higher rate limits
    }
  }
}
```

**API:** https://docs.github.com/en/rest/security-advisories

**To get a GitHub token:**
1. Go to https://github.com/settings/tokens
2. Generate a new token (classic)
3. Add `public_repo` scope
4. Add token to config: `"token": "ghp_..."`

---

### **3. Snyk** ⚠️ Disabled by Default

**Snyk's comprehensive vulnerability database**

- ⚠️ **REQUIRES API token** - Cannot work without token
- ⚠️ **Disabled by default** - Must be explicitly enabled with token
- ✅ **Enterprise-grade** - Used by major companies
- ✅ **Detailed reports** - Severity, CVSS scores, remediation
- ✅ **High rate limits** - Suitable for enterprise use

**Configuration:**
```json
{
  "providers": {
    "snyk": {
      "enabled": true,
      "token": "your-snyk-api-token"
    }
  }
}
```

**To get a Snyk token:**
1. Sign up at https://snyk.io
2. Go to Account Settings → API Token
3. Generate a new token
4. Add token to config: `"token": "..."`

**API:** https://snyk.io/api

---

## ⚙️ **Configuration**

### **Full Example Config**

```json
{
  "dataSourcePath": "./config/compromised-packages.json",
  "skipNpmAudit": false,
  "logMode": "normal",
  "providers": {
    "osv": {
      "enabled": true,
      "timeout": 5000
    },
    "github": {
      "enabled": true,
      "timeout": 5000,
      "token": null
    },
    "snyk": {
      "enabled": false,
      "token": null
    }
  }
}
```

### **Disable All Providers**

```json
{
  "providers": {
    "osv": { "enabled": false },
    "github": { "enabled": false }
  }
}
```

### **Enable Only OSV**

```json
{
  "providers": {
    "osv": { "enabled": true },
    "github": { "enabled": false }
  }
}
```

---

## 🔄 **How It Works**

### **Validation Flow:**

```
1. Check local blacklist (Shai-Hulud, custom)
   ↓
2. Check providers (OSV, GitHub, Snyk) in parallel
   ↓
3. If vulnerability found → BLOCK installation
   ↓
4. Fallback to npm audit (if providers didn't find anything)
```

### **Provider Priority:**

1. **OSV** - Checked first (fastest, most comprehensive)
2. **GitHub** - Checked second (npm ecosystem focus)
3. **Snyk** - Checked third (if enabled, enterprise-grade)

**Note:** Providers run in **parallel** for speed. The first vulnerability found blocks installation.

---

## 🚀 **Usage**

### **Out of the Box (Default)**

**Providers run automatically - no configuration needed!**

```bash
npm install express
# ✅ Automatically checks OSV + GitHub Advisories (no config needed)
```

**What runs by default:**
- ✅ **OSV** - Enabled, no token required
- ✅ **GitHub Advisories** - Enabled, no token required
- ❌ **Snyk** - Disabled (requires token)

### **With CLI Arguments**

```bash
# Disable a provider
sentinel scan package-name --enableOsv=false

# Enable Snyk with token
sentinel scan package-name --enableSnyk=true --snykToken="your-token"

# Add GitHub token for higher rate limits
sentinel scan package-name --githubToken="ghp_..."
```

### **With Config File**

```bash
# Create config
sentinel init

# Edit sentinel.config.json
# Add provider configs

# Use normally
npm install express
```

---

## 📊 **Provider Comparison**

| Provider | Free | Token Required | Default | Rate Limit | Coverage | Speed |
|----------|------|----------------|---------|------------|----------|-------|
| **OSV** | ✅ | ❌ No | ✅ Enabled | High | Comprehensive | Fast |
| **GitHub** | ✅ | ⚠️ Optional | ✅ Enabled | 60/hour (public)<br>5000/hour (with token) | npm ecosystem | Fast |
| **Snyk** | ⚠️ | ✅ Required | ❌ Disabled | High | Enterprise | Fast |

---

## 🔧 **Troubleshooting**

### **Provider Timeout**

If providers are slow, increase timeout:

```json
{
  "providers": {
    "osv": { "timeout": 10000 },
    "github": { "timeout": 10000 }
  }
}
```

### **Rate Limit Errors**

**GitHub:** Add a token for higher limits (5000/hour)

```json
{
  "providers": {
    "github": {
      "token": "ghp_your_token_here"
    }
  }
}
```

**Snyk:** Check your plan's rate limits

### **Provider Down or Timeout**

If a provider is down or times out, Sentinel:
- ✅ **Fails gracefully** (doesn't block installation) - fail-open behavior
- ✅ **Continues with other sources** (local blacklist, other providers, npm audit)
- ✅ **Logs error** in verbose mode only
- ⚠️ **npm audit fallback** - Only works when scanning projects with lockfiles, not standalone packages

**To disable a problematic provider:**

```json
{
  "providers": {
    "github": { "enabled": false }
  }
}
```

---

## 🎯 **Best Practices**

1. **Enable OSV** - Free, fast, comprehensive
2. **Enable GitHub** - Good npm ecosystem coverage
3. **Enable Snyk** - If you have enterprise needs
4. **Use tokens** - For higher rate limits (GitHub, Snyk)
5. **Monitor logs** - Use `--logMode=verbose` to see provider activity

---

## 📚 **More Information**

- [OSV Documentation](https://osv.dev/docs/)
- [GitHub Security Advisories API](https://docs.github.com/en/rest/security-advisories)
- [Snyk API Documentation](https://snyk.io/api)

---

## ✅ **Summary**

**Providers give you:**
- ✅ Real-time vulnerability data
- ✅ Industry-leading security databases
- ✅ Automatic updates
- ✅ Comprehensive coverage

**Out of the box (default):**
- ✅ **OSV enabled** - Free, no token, no config needed
- ✅ **GitHub enabled** - Works without token, but token recommended for complete results
- ❌ **Snyk disabled** - Requires token (must enable manually)

**Token Requirements:**
- **OSV:** ❌ No token needed
- **GitHub:** ⚠️ Token **recommended** (unauthenticated may have incomplete results, 60 req/hr limit)
- **Snyk:** ✅ Token is **required** (cannot work without it)

**Just install and use - OSV works automatically! GitHub works without a token but a token is recommended for complete results.** 🎉

---

## 🔧 Provider Behavior & Failure Handling

### **Failure Behavior (Fail-Open)**

Sentinel uses a **fail-open** approach for providers:

| Failure Scenario | Behavior | Configurable? |
|------------------|----------|--------------|
| **Provider timeout** | Warn (verbose mode) and continue with other sources | Yes (timeout config) |
| **Provider API error** | Warn (verbose mode) and continue | No |
| **Provider rate limit** | Warn (verbose mode) and continue | No |
| **Network failure** | Warn (verbose mode) and continue | No |
| **Blacklist file missing** | Warn and continue (does not block) | No |

**Rationale:** If a provider is down, Sentinel doesn't block legitimate installs. It continues checking other sources (local blacklist, other providers, npm audit).

### **Priority & Conflict Resolution**

> **📖 For complete priority and conflict resolution details**, see [Data Sources Guide](DATA_SOURCES.md#priority--conflict-resolution) which covers all data source types (local files, API endpoints, and providers).

### **Version Matching Logic**

Sentinel matches packages using:
- **Package name**: Exact match (case-sensitive)
- **Version**: Exact version match (e.g., `1.2.3` matches `1.2.3`)
- **Version ranges**: Not supported - only exact versions in blacklist
- **Scoped packages**: Supported (e.g., `@scope/package`)

**Example:**
- Blacklist has `"express": ["4.18.2"]` → Blocks `express@4.18.2` only
- Blacklist has `"express": []` → Blocks all versions of `express`

### **Caching**

Sentinel does **not** cache provider responses. Each validation checks providers in real-time to ensure up-to-date vulnerability data.

### **No Telemetry**

Sentinel sends **zero telemetry** and never uploads:
- Your dependency graph
- Package names or versions you're installing
- Any information to external services (except provider API calls for vulnerability checks)

**CLI Arguments:**
```bash
# Disable providers
--enableOsv=false
--enableGitHub=false

# Enable Snyk
--enableSnyk=true --snykToken="..."

# Add tokens
--githubToken="ghp_..."  # Optional (recommended for complete results)
--snykToken="..."        # Required
```

---

## ⚠️ Important Limitations

### **GitHub Advisories API**

- Unauthenticated requests may return incomplete results (60 requests/hour limit)
- Token recommended for production use (5000 requests/hour, complete data)
- API may have temporary issues during peak hours

### **npm audit Fallback**

- Only works when scanning directories/repositories with `package.json` + lockfile
- Cannot check standalone packages (e.g., `sentinel scan react@18.0.0`)
- Requires npm to be installed and accessible

### **Version Matching**

- Only exact version matching supported (e.g., `1.2.3` matches `1.2.3`)
- Semver ranges not supported in blacklist (e.g., `>=1.2.0` won't match)
- Use empty array `[]` in blacklist to block all versions of a package