@boilerbuilder/deps-analyzer/README.md

CLI tool to analyze dependency evolution and release frequency

# Dependencies Evolution Analyzer

📊 Advanced CLI tool to analyze dependency evolution, release frequency, and semver patterns in JavaScript/Node.js projects.

## Features

- 🔍 **Auto-discovery**: Finds package.json files in directories or glob patterns
- 📊 **NPM Analysis**: Fetches release data from NPM registry with smart filtering
- 🏷️ **Semver Analysis**: Detailed breakdown of major/minor/patch releases
- 📅 **Time-based Analysis**: Compares releases from different time periods (12m vs 12-24m)
- 📄 **Multiple Outputs**: Generates both JSON and Markdown reports
- 🚀 **Multiple Projects**: Analyze multiple patterns/projects in one command
- ⚡ **Fast**: Built-in NPM CLI integration and efficient processing
- 🛡️ **Robust**: Handles errors gracefully and continues processing
- 🔧 **Smart Filtering**: Automatically filters dev/beta/alpha/nightly versions

## Installation

### Local Development

```bash
git clone <repository-url>
cd deps-analyzer
npm install
```

### Using npm link for testing

```bash
npm run link          # Link globally for testing
deps-analyzer --help  # Test the command
npm run unlink        # Unlink when done
```

## 📚 Documentation

For detailed information on specific topics:

- **[Libyear Algorithm](docs/libyears.md)** - Understanding drift/pulse metrics and calculations
- **[Max-Deps Configuration](docs/max-deps.md)** - Strategic version control and constraints  
- **[NPM Client](docs/npm-client.md)** - Data collection strategies and filtering
- **[Report Interpretation](docs/reporter.md)** - How to read and act on analysis results

## Usage

### Basic Usage

```bash
# Analyze templates directory
deps-analyzer ./templates

# Analyze single project
deps-analyzer ./package.json

# Use glob patterns
deps-analyzer "projects/**/package.json"
```

### Multiple Projects

```bash
# Analyze specific projects
deps-analyzer templates/next-app templates/react-spa-vite --output selected-projects

# Mix directories and files
deps-analyzer ./frontend ./backend/package.json --output fullstack-analysis
```

### Advanced Options

```bash
# Custom analysis period
deps-analyzer ./templates --months 18

# Custom output name
deps-analyzer ./templates --output my-report

# Skip NPM analysis (discovery only)
deps-analyzer ./templates --no-npm

# Use HTTP instead of npm CLI (slower but more compatible)
deps-analyzer ./templates --use-http

# Advanced filtering for internal libraries
deps-analyzer ./templates --internal-libs "@akad/*,design-system*"

# Skip specific version filters
deps-analyzer ./templates --ignore-filter alpha,beta,rc

# Combined enterprise usage
deps-analyzer ./templates --internal-libs "@company/*" --ignore-filter canary,nightly

# Custom max-deps configuration file
deps-analyzer ./templates --max-deps-config ./my-constraints.json

# Strategic version control example
echo '{"react": "^18.0.0", "typescript": "^5.0.0"}' > max-deps.json
deps-analyzer ./project --max-deps-config max-deps.json

# Help and version
deps-analyzer --help
deps-analyzer --version
```

### Scope Filtering

Analyze only internal or external dependencies using `--scope`:

```bash
# Analyze only internal dependencies
deps-analyzer ./templates --scope=internal --internal-libs "@company/*"

# Analyze only external dependencies (excludes internal)
deps-analyzer ./templates --scope=external --internal-libs "@company/*"

# Analyze all dependencies (default behavior)
deps-analyzer ./templates --scope=all --internal-libs "@company/*"
```

### Max-Deps Configuration Examples

Strategic version control with custom configuration files:

```bash
# Production environment constraints
deps-analyzer ./templates --max-deps-config ./configs/production.json

# Shared team constraints
deps-analyzer ./templates --max-deps-config /shared/team-constraints.json

# Benchmark analysis (zero-drift simulation)
deps-analyzer ./templates --max-deps-config ./benchmarks/zero-drift.json --output benchmark

# Enterprise compliance analysis
deps-analyzer ./microservices --max-deps-config ./security/approved-versions.json
```

## Output

The tool generates two files with comprehensive analysis:

### JSON Report (`<output>.json`)
```json
{
  "metadata": {
    "generatedAt": "2025-08-14T00:26:31.877Z",
    "period": "24 months",
    "totalProjects": 2,
    "totalDependencies": 52,
    "uniqueDependencies": 36,
    "globalUniqueReleases": {
      "totalLast12m": 1130,
      "totalLast12to24m": 983,
      "avgPerMonthLast12m": 94.17,
      "avgPerMonthLast12to24m": 81.92,
      "semverBreakdown": {
        "majorLast12m": 26,
        "minorLast12m": 165,
        "patchLast12m": 597,
        "avgMajorPerMonthLast12m": 2.17,
        "avgMinorPerMonthLast12m": 13.75,
        "avgPatchPerMonthLast12m": 49.75
      }
    }
  },
  "projects": {
    "react-spa-vite": {
      "packagePath": "/path/to/package.json",
      "dependencyCount": 32,
      "dependencies": {
        "react": {
          "currentVersion": "19.1.1",
          "last12Months": 3,
          "months12to24": 2,
          "totalReleases": 345,
          "avgReleasesPerMonthLast12m": 0.25,
          "avgReleasesPerMonthLast12to24m": 0.17,
          "avgDaysBetweenReleases": 14.6,
          "semverBreakdown": {
            "majorLast12m": 1,
            "minorLast12m": 1,
            "patchLast12m": 1,
            "majorLast12to24m": 0,
            "minorLast12to24m": 1,
            "patchLast12to24m": 1
          },
          "monthlyBreakdown": {
            "2024-12": ["19.0.0"],
            "2025-03": ["19.1.0"], 
            "2025-07": ["19.1.1"]
          },
          "mostActiveMonth": "2024-12 (1)"
        }
      },
      "metadata": {
        "totalReleasesLast12m": 672,
        "totalReleasesLast12to24m": 655,
        "totalReleases": 6429,
        "avgReleasesPerMonthLast12m": 56,
        "avgReleasesPerMonthLast12to24m": 54.58,
        "semverBreakdown": {
          "totalMajorLast12m": 22,
          "totalMinorLast12m": 140,
          "totalPatchLast12m": 419,
          "avgMajorPerMonthLast12m": 1.83,
          "avgMinorPerMonthLast12m": 11.67,
          "avgPatchPerMonthLast12m": 34.92
        }
      }
    }
  }
}
```

### Markdown Report (`<output>.md`)
```markdown
# Dependencies Evolution Report

## Summary
- **Generated at:** 2025-08-14 00:26:31 UTC
- **Analysis period:** 24 months
- **Projects analyzed:** 2
- **Total dependencies:** 52
- **Unique dependencies:** 36

## Global Statistics
- **Total releases (12m):** 1,130 (avg: 94.17/month)
- **Total releases (12-24m):** 983 (avg: 81.92/month)
- **Semver breakdown (12m):** 26 major, 165 minor, 597 patch

## Project: react-spa-vite (32 dependencies)

### Dependencies Release Activity (Last 12m vs 12-24m)
| Package | Current | 12m | 12-24m | Major | Minor | Patch | Most Active Month |
|---------|---------|-----|--------|-------|-------|-------|-------------------|
| react   | 19.1.1  | 3   | 2      | 1     | 1     | 1     | 2024-12 (1)      |
```

## API

### Programmatic Usage

```javascript
const { main } = require('deps-analyzer');

// Basic analysis
const result = await main(['./templates']);

// Multiple patterns
const result = await main([
  './templates/next-app',
  './templates/react-spa-vite'
]);

// With options
const result = await main(['./templates'], {
  months: 18,
  output: 'custom-report',
  analyzeNpm: true,
  useNpmCli: true
});
```

## Options Reference

| Option | Default | Description |
|--------|---------|-------------|
| `--months <number>` | `24` | Analysis period in months |
| `--output <path>, -o` | `dependency-report` | Output file base name |
| `--scope <type>` | `all` | Filter dependencies by scope: internal, external, all |
| `--no-npm` | `false` | Skip NPM registry analysis (discovery only) |
| `--use-http` | `false` | Use HTTP requests instead of npm CLI (slower) |
| `--ignore-filter <list>` | `[]` | Skip filtering specific version types (alpha,beta,rc,etc) |
| `--internal-libs <list>` | `[]` | Libraries to include alpha/beta versions (supports wildcards: @akad/*,design-system*) |
| `--max-deps-config <path>` | `null` | Custom max-deps.json file path for strategic version control |
| `--help, -h` | - | Show help information |
| `--version, -v` | - | Show version |

## Examples

### Template Repository Analysis
```bash
# Analyze all templates
deps-analyzer ./templates --months 12 --output templates-report

# Analyze specific templates
deps-analyzer templates/next-app templates/react-spa-vite --output selected-templates
```

### Monorepo Analysis
```bash
# Analyze all packages
deps-analyzer "packages/*/package.json" --months 18

# Analyze specific workspaces
deps-analyzer packages/frontend packages/backend packages/shared --output monorepo-analysis
```

### Quick Dependency Discovery
```bash
# Skip NPM analysis for faster results
deps-analyzer ./project --no-npm

# Discovery only with HTTP fallback
deps-analyzer ./project --no-npm --use-http
```

### Enterprise Usage
```bash
# Comprehensive analysis with custom period
deps-analyzer ./microservices ./shared-libs --months 36 --output enterprise-deps
```

## Advanced Filtering

### Filtering Strategy Hierarchy

The tool applies filtering in a specific order to ensure predictable results:

1. **Scope Filtering** (`--scope`): First, filter dependencies by scope (internal, external, all)
2. **Internal Library Classification** (`--internal-libs`): Classify remaining dependencies as internal/external  
3. **Selective Version Filtering**: Apply different filtering rules based on classification
4. **Filter Overrides** (`--ignore-filter`): Override specific version type filtering

### Internal vs External Dependencies

The tool implements **selective filtering** that treats internal and external dependencies differently:

- **External Dependencies**: Automatically filters out pre-release versions (alpha, beta, rc, etc.)
- **Internal Dependencies**: Can include pre-release versions when specified via `--internal-libs`

### Scope Filtering Behavior

- **`--scope=internal`**: Analyze only dependencies matching `--internal-libs` patterns
- **`--scope=external`**: Analyze only dependencies NOT matching `--internal-libs` patterns  
- **`--scope=all`**: Analyze all dependencies (default), but still apply selective filtering logic

### Pattern Matching

The `--internal-libs` flag supports wildcard patterns:

```bash
# Exact match
deps-analyzer ./project --internal-libs "my-package"

# Scoped packages with wildcard
deps-analyzer ./project --internal-libs "@company/*"

# Multiple patterns
deps-analyzer ./project --internal-libs "@company/*,design-system*,@internal/*"
```

### Filtering Override

Use `--ignore-filter` to skip specific version type filtering:

```bash
# Include alpha versions for ALL dependencies
deps-analyzer ./project --ignore-filter alpha

# Include multiple pre-release types
deps-analyzer ./project --ignore-filter alpha,beta,rc

# Combined usage: include alpha/beta for internal libs only
deps-analyzer ./project --internal-libs "@company/*" --ignore-filter canary,nightly
```

### Practical Examples

#### Enterprise Monorepo Analysis
```bash
# Analyze with company-specific internal libraries
deps-analyzer ./packages/* \
  --internal-libs "@acme/*,design-tokens*" \
  --ignore-filter canary \
  --output enterprise-analysis
```

#### Mixed Public/Private Dependencies
```bash
# Allow alpha/beta for internal libs, strict filtering for external
deps-analyzer ./apps/frontend ./apps/backend \
  --internal-libs "@company/*,internal-*" \
  --output mixed-dependencies
```

#### Historical Analysis with Custom Filters
```bash
# Skip all pre-release filtering for specific analysis
deps-analyzer ./legacy-projects \
  --ignore-filter alpha,beta,rc,canary,experimental \
  --output legacy-comprehensive
```

## Understanding the Output

### Portfolio vs Project Metrics

The tool provides **two different types of drift/pulse calculations** that serve different purposes:

#### 🏢 Global/Portfolio Metrics (Unique Dependencies)
- **What it measures**: Drift/pulse for each dependency **once per portfolio**
- **Calculation**: Sums drift only from unique dependencies across all projects
- **Example**: If `react` appears in 4 projects, its drift is counted **only once**
- **Use case**: Strategic decisions - "Which dependencies should we prioritize updating?"

#### 📋 Project Metrics (Total Across Projects)  
- **What it measures**: Drift/pulse for **every occurrence** of each dependency
- **Calculation**: Sums drift from all dependencies in all projects
- **Example**: If `react` appears in 4 projects, its drift is counted **4 times**
- **Use case**: Effort estimation - "How much total work is needed to update everything?"

#### 💡 Example: 4 Projects with React (2y drift)

**Scenario:**
- 4 projects each use `react` with 2 years of drift
- Each project also has `lodash` with 1 year of drift

**Global Drift (Unique Dependencies):**
- `react`: 2 years (counted once)
- `lodash`: 1 year (counted once)
- **Total: 3 years**

**Project Drift (Sum Across Projects):**
- Project 1: `react` (2y) + `lodash` (1y) = 3y
- Project 2: `react` (2y) + `lodash` (1y) = 3y  
- Project 3: `react` (2y) + `lodash` (1y) = 3y
- Project 4: `react` (2y) + `lodash` (1y) = 3y
- **Total: 12 years**

#### 🎯 When to Use Each Metric

| Scenario | Use Global Metrics | Use Project Metrics |
|----------|-------------------|-------------------|
| Prioritizing which deps to update | ✅ Focus on unique deps | ❌ May mislead priorities |
| Estimating update effort/time | ❌ Underestimates work | ✅ Shows true scope |
| Executive reporting | ✅ Clear strategic view | ❌ May seem inflated |
| Sprint planning | ❌ Doesn't show full work | ✅ Shows actual tasks |
| Ecosystem health assessment | ✅ True portfolio state | ❌ Duplicated concerns |

### Semver Analysis
The tool performs intelligent semver analysis:
- **Major**: Breaking changes (1.0.0 → 2.0.0)
- **Minor**: New features (1.0.0 → 1.1.0)
- **Patch**: Bug fixes (1.0.0 → 1.0.1)

### Smart Filtering
Automatically filters out:
- Development versions (`-dev`, `-nightly`)
- Alpha/Beta releases (`-alpha`, `-beta`)
- Release candidates (`-rc`)
- Experimental builds (`-experimental`, `-canary`)
- Malformed versions (`0.0.0-*`)

### Backport Detection
The tool detects version backports/downgrades:
- Total releases include all stable versions
- Semver breakdown only counts progressive releases
- Difference represents maintenance releases for older versions

## Technical Details

- **Rate Limiting**: Intelligent batching with NPM CLI
- **Error Handling**: Continues processing even if some packages fail
- **Ignores**: `node_modules/`, `public/`, `dist/`, `build/` directories
- **Node Version**: Requires Node.js >= 16.0.0
- **NPM Version**: Works with npm >= 7.0.0

## Development

### Project Structure
```
deps-analyzer/
├── package.json
├── README.md
├── bin/
│   └── deps-analyzer.js    # CLI entry point
├── lib/
│   ├── index.js            # Core logic & discovery
│   ├── npm-client.js       # NPM registry client
│   ├── libyear.js          # Drift/pulse calculations
│   └── reporter.js         # Report generation
└── utils/                  # Utility functions
    ├── chunkArray.js       # Array chunking
    ├── cleanVersion.js     # Version cleaning
    ├── filterDependenciesByScope.js  # Dependency filtering
    ├── findMaxAllowedVersion.js      # Version constraint parsing
    ├── findWorkspaceRoot.js          # Workspace detection
    ├── isInternalLib.js              # Internal library matching
    ├── parsePnpmLock.js              # PNPM lock file parsing
    ├── parseYarnLock.js              # Yarn lock file parsing
    └── versionSatisfiesConstraint.js # Semver constraint checking
```

### Scripts
```bash
npm run start      # Run with default args
npm run test       # Test with templates
npm run dev        # Run with debugging
npm run link       # Link globally
npm run unlink     # Unlink globally
```

### Debugging
```bash
# Enable detailed error information
DEBUG=1 deps-analyzer ./templates

# Use HTTP instead of npm CLI if issues
deps-analyzer ./templates --use-http
```

## Troubleshooting

### Common Issues

**npm CLI not found**
```bash
# Use HTTP fallback
deps-analyzer ./templates --use-http
```

**Permission errors**
```bash
# Check npm permissions
npm config get prefix
npm config get cache
```

**Network timeouts**
```bash
# Increase npm timeout
npm config set timeout 60000
deps-analyzer ./templates
```

**Memory issues with large projects**
```bash
# Analyze projects separately
deps-analyzer ./project1 --output report1
deps-analyzer ./project2 --output report2
```

## Performance

### Benchmarks
- **Small project** (10 deps): ~2-5 seconds
- **Medium project** (50 deps): ~10-20 seconds  
- **Large monorepo** (200+ deps): ~1-3 minutes

### Optimization Tips
- Use npm CLI mode (default) for better performance
- Analyze fewer projects simultaneously for large codebases
- Use `--no-npm` for quick dependency discovery

## License

MIT

## Contributing

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Create a Pull Request

### Development Guidelines
- Maintain backward compatibility
- Add tests for new features
- Update documentation
- Follow existing code style

---

Built with ❤️ for better dependency management and DevOps insights