veriqa-test-advisor/README.md

AI-powered regression test case advisor CLI tool with integrated Manual QA support

# 🧠 VeriQA Test Advisor

> **AI-Powered Regression Test Case Advisor CLI Tool** 🇮🇳 Made in India

[![npm version](https://badge.fury.io/js/veriqa-test-advisor.svg)](https://badge.fury.io/js/veriqa-test-advisor)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**VeriQA** intelligently suggests which test cases to run based on:
- 📊 **Git commits/diffs** analysis
- 🤖 **AI (Perplexity)** suggestions  
- 🗺️ **Module-to-Test** mapping
- ⚡ **Custom CLI** execution
- 📋 **Enhanced Reporting** (PDF & CSV formats) ✨ **NEW**

## ✨ Latest Updates

### 🎉 Enhanced Reporting System (v2.0)
**Complete integration of simplified enhanced reporting supporting PDF and CSV formats only:**

- **📊 Multi-Format Reports**: Generate both PDF and CSV reports simultaneously
- **🏢 Enterprise Features**: Context-specific messaging for different audiences
- **🧹 Smart Cleanup**: Automatic maintenance of latest 5 reports per format
- **⚡ CLI Integration**: Seamlessly integrated into main CLI without code duplication

```bash
# Generate CSV report for TCS submissions
veriqa signoff --format csv

# Generate both PDF and CSV formats
veriqa signoff --multi-format

# Clean up old reports with statistics
veriqa cleanup --stats
```

## 🎯 Why VeriQA?

| Problem | VeriQA Solution |
|---------|----------------|
| Test suites are growing large; running full regression is costly | Use Git diff + module mapping to detect impacted test cases |
| Manual mapping of modules to tests is tedious and error-prone | Automatically generate mapping JSON using directory structures |
| QA/devs may not know what test cases are relevant to a change | Use AI to suggest intelligent test cases based on commit message |
| Integration with CI tools is often manual | Create a CLI that can be easily integrated into any CI/CD pipeline |
| **Need comprehensive reporting for stakeholders and TCS** | **Enhanced PDF/CSV reporting with enterprise messaging** ✨ |

## 🚀 Quick Start

### Installation

```bash
# Install globally (recommended)
npm install -g veriqa-test-advisor

# Or install locally in your project
npm install veriqa-test-advisor --save-dev
```

### 🎯 Zero-Config Setup

VeriQA provides **automatic project setup** for new users:

```bash
# Install and auto-setup your project
npm install -g veriqa-test-advisor
npx veriqa-setup

# This creates:
# ✅ Sample test files with @module: tags
# ✅ Configuration files (.env, playwright.config.js)  
# ✅ Project structure (tests/auth, tests/product)
# ✅ Ready-to-run examples
```

### Basic Usage

```bash
# Get AI suggestions based on latest commit
veriqa-test-advisor --ai

# Run suggested test cases
veriqa-test-advisor --run

# Filter by tags and environment
veriqa-test-advisor --run --tag smoke --env staging

# Dry run (show what would be executed)
veriqa-test-advisor --dry-run --tag regression
```

### 📦 NPM Scripts (Auto-Added)

After setup, use these convenient commands:

```bash
npm run test:ai          # AI-powered test suggestions
npm run test:smoke       # Run smoke tests
npm run test:regression  # Run regression tests
npm run test:dry         # Dry run mode

# 🎯 Interactive Demos & Setup
npm run demo:simple      # Interactive VeriQA demo (4 modes)
npm run setup:github     # Enhanced GitHub Copilot setup
```

### 🎭 Interactive Demo Modes

```bash
# Quick overview (30 seconds)
npm run demo:simple

# Select from 4 demo modes:
# 1️⃣ Quick Overview - Essential commands  
# 2️⃣ Interactive Demo - Live project analysis
# 3️⃣ Advanced Features - Complete feature tour
# 4️⃣ Installation Guide - Step-by-step setup
```

## 📖 Command Reference

| Command | Description |
|---------|-------------|
| `veriqa-test-advisor` | Suggest test cases based on code changes |
| `--ai` | Use AI to suggest test cases via commit message |
| `--run` | Run only suggested test cases |
| `--tag <tag>` | Filter tests by tag (@smoke, @regression, etc.) |
| `--env <env>` | Pass environment variable or profile |
| `--dry-run` | Show test plan without execution |
| `--verbose` | Show internal debug information |
| `--version` | Show version information |
| `--help` | Show help message with installed dependencies |

## ⚙️ Setup & Configuration

### 1. Test Structure

Organize your tests with module annotations:

```javascript
// tests/auth/login.test.js
// @module: auth

const { test, expect } = require('@playwright/test');

test('TC_Login_01 @smoke @regression - should login', async ({ page }) => {
  await page.goto('/login');
  await page.fill('#username', 'testuser');
  await page.fill('#password', 'password123');
  await page.click('#login-btn');
  await expect(page.locator('text=Welcome')).toBeVisible();
});
```

### 2. Environment Variables

Create a `.env` file:

```env
# Recommended: Use your GitHub Copilot license!
GITHUB_TOKEN=your_github_token_here

# Alternative: Perplexity AI (free option)  
PERPLEXITY_API_KEY=your_perplexity_api_key
OPENAI_MODEL=sonar-pro

# VeriQA Settings
VERIQA_FRAMEWORK=playwright
VERIQA_ENV=staging
```

### 3. Supported Frameworks

VeriQA works with:
- ✅ **Playwright** (`@playwright/test`)
- ✅ **CodeceptJS** with Playwright helper
- 📊 **Allure** reporting (automatic)

## 🤖 AI Integration

VeriQA supports multiple AI providers for intelligent test suggestions:

### 🚀 **GitHub Copilot Integration** (Recommended)
Use your existing GitHub Copilot license for enhanced AI suggestions:

```bash
# Setup: Add GITHUB_TOKEN to .env (with repo, copilot scopes)
# Get token: https://github.com/settings/tokens

# AI will use GitHub Copilot to analyze your commits
veriqa-test-advisor --ai
```

### 🔄 **Perplexity AI** (Free Alternative)
Free option using Perplexity AI:

```bash
# Setup: Add PERPLEXITY_API_KEY to .env
# Get free key: https://www.perplexity.ai/

# Example output:
# 🤖 AI Suggested Regression Test Cases:
# - TC_Login_01.spec.js (reason: login functionality changes)
# - TC_2FA_01.spec.js (reason: authentication flow affected)
# - TC_Registration_test.js (reason: user management updates)
```

## 🗺️ Module Mapping

VeriQA automatically generates `mappings/module_test_map.json`:

```json
{
  "auth": [
    "tests/auth/TC_Login_01.spec.js",
    "tests/auth/TC_2FA_01.spec.js"
  ],
  "product": [
    "tests/product/TC_Search.spec.js"
  ]
}
```

## 📊 Advanced Features

### ✨ Enhanced Reporting System (NEW)

VeriQA now includes comprehensive enterprise-ready reporting with PDF and CSV formats:

```bash
# Generate CSV report for TCS submissions and Excel analysis
veriqa signoff --format csv

# Generate PDF report for stakeholders and management
veriqa signoff --format pdf

# Generate both formats simultaneously
veriqa signoff --multi-format

# Custom filename with date
veriqa signoff --format csv --filename "tcs-submission-$(date +%Y%m%d)"
```

#### 🏢 Enterprise Features
- **📊 Smart Multi-Format**: Generate PDF and CSV simultaneously with `--multi-format`
- **🧹 Automatic Cleanup**: Maintains latest 5 reports per format automatically
- **🎯 Context Messaging**: Format-specific messages for different audiences
- **📈 Statistics**: View report statistics with `--stats` option

#### 📋 Report Commands
```bash
# Report cleanup with statistics
veriqa cleanup --stats --max 3

# Preview cleanup without deletion
veriqa cleanup --dry-run

# Generate quarterly review reports
veriqa signoff --multi-format --filename "Q1-2024-review"
```

#### 📊 Report Formats

| Format | Use Case | Audience | Command |
|--------|----------|----------|---------|
| **CSV** | TCS submissions, Excel analysis | Analysts, QA Teams | `--format csv` |
| **PDF** | Stakeholder presentations | Management, Executives | `--format pdf` |
| **JSON** | CI/CD integration, automation | Development Teams | `--format json` |
| **Console** | Daily development workflow | Developers | `--format console` |

### K6-Style Test Summary

After test execution, get detailed metrics:

```
running (veriqa-advisor) ...
wall time:         2m 34.2s
tests:             25
✓ passed:          23 (92.0%)
✗ failed:          2 (8.0%)
avg duration:      3.4s
p50 | p90:         2.1s | 8.7s
p95 | p99:         12.3s | 18.9s
RPS:               0.16

by framework:
  playwright       20 (80.0%)
  codeceptjs       5 (20.0%)

by tag:
  smoke           15 (60.0%)
  regression      10 (40.0%)
```

### CI/CD Integration

```yaml
# GitHub Actions example
- name: Run Smart Regression Tests
  run: |
    npx veriqa-test-advisor --run --tag smoke
    
# GitLab CI example  
test:regression:
  script:
    - npx veriqa-test-advisor --ai --verbose
    - npx veriqa-test-advisor --run --env $CI_ENVIRONMENT_NAME
```

## 🎨 Examples

### Basic Workflow

```bash
# 1. Make code changes
git add . && git commit -m "fix: update login validation logic"

# 2. Get AI suggestions
veriqa-test-advisor --ai

# 3. Run suggested tests
veriqa-test-advisor --run --tag smoke

# 4. Run full regression if needed
veriqa-test-advisor --run --tag regression
```

### Advanced Usage

```bash
# Debug mode with verbose output
veriqa-test-advisor --ai --verbose

# Dry run to see what would be executed
veriqa-test-advisor --dry-run --tag "smoke|regression"

# Environment-specific testing
VERIQA_ENV=staging veriqa-test-advisor --run --tag smoke
```

## 🔧 Configuration Files

### Playwright Config
```javascript
// playwright.config.js
module.exports = {
  testDir: 'tests',
  reporter: [
    ['list'],
    ['allure-playwright']  // Required for VeriQA metrics
  ]
};
```

### CodeceptJS Config
```javascript
// codecept.conf.js  
exports.config = {
  tests: 'tests/**/*.js',
  plugins: {
    allure: {
      enabled: true,
      require: '@codeceptjs/allure-legacy'
    }
  }
};
```

## 📈 Business Value

- 📉 **Reduce CI/CD costs** by 60-80% (run only relevant tests)
- ⚡ **Faster feedback** for developers (smart test selection)
- 🎯 **Higher test coverage** confidence (AI-driven suggestions)
- 🔄 **Easy integration** with existing pipelines
- 📊 **Better insights** with detailed test analytics

## 🛠️ Troubleshooting

### Quick Diagnostics
```bash
# Comprehensive project analysis & diagnosis
npx veriqa-troubleshoot

# Auto-fix detected issues (config naming, imports, etc.)
npx veriqa-troubleshoot --fix

# Validate complete installation
npx veriqa-validate
```

### Advanced Troubleshooting
Our enhanced troubleshoot system automatically detects and fixes:

✅ **Config File Naming Issues**
- `playwright.config` → `playwright.config.js`
- `codecept.config.js` → `codecept.conf.js`  
- `cypress.config` → `cypress.config.js`
- `Package.json` → `package.json`

✅ **Framework Compatibility**
- Multi-framework conflict detection
- Port conflict warnings
- Dependency validation

✅ **Environment Configuration**
- Missing API keys detection
- `.env` file validation
- Smart recommendations

### Common Issues

**❌ "ReferenceError: devices is not defined"**
```bash
# Fix Playwright config devices import
npx veriqa-troubleshoot --fix
# Or manually add: const { defineConfig, devices } = require('@playwright/test');
```

**❌ "Command not found: veriqa-test-advisor"**
```bash
# Reinstall globally
npm uninstall -g veriqa-test-advisor
npm install -g veriqa-test-advisor@latest
```

**❌ "No tests found"**
```bash
# Check project setup
npx veriqa-validate
# Regenerate mapping
veriqa-test-advisor --run --verbose
```

**❌ "AI suggestions failed"**  
```bash
# Check API key setup
cat .env | grep PERPLEXITY_API_KEY
# Get your free API key at: https://perplexity.ai
```

### Get Help
- 📖 [Installation Guide](INSTALLATION.md)
- 🔧 [Troubleshooting Tool](scripts/troubleshoot.js)
- 🐛 [Report Issues](https://github.com/kapil971390/VeriQA-Regression-Advisor/issues)

## 📈 Business Value

- 📉 **Reduce CI/CD costs** by 60-80% (run only relevant tests)
- ⚡ **Faster feedback** for developers (smart test selection)
- 🎯 **Higher test coverage** confidence (AI-driven suggestions)
- 🔄 **Easy integration** with existing pipelines
- 📊 **Better insights** with detailed test analytics

## 🤝 Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md).

## 📄 License

MIT © [Kapil Rathore](https://github.com/kapilrathore)

## 🔗 Links

- 📦 [npm package](https://www.npmjs.com/package/veriqa-test-advisor)
- 🐙 [GitHub repository](https://github.com/kapilrathore/veriqa-test-advisor)
- 📖 [Documentation](https://veriqa.dev/docs)
- 💬 [Discord Community](https://discord.gg/veriqa)

## 👨‍💻 Author

**Kapil Rathore** - *Creator & Maintainer*
- 🐙 GitHub: [@kapil971390](https://github.com/kapil971390)
- 📧 Email: kapil9713@gmail.com
- 💼 LinkedIn: [Connect with me](https://www.linkedin.com/in/kapil-rathore)

## 🔄 Version Management

VeriQA includes automatic version checking and update notifications:

### Version Commands
```bash
# Check for updates
veriqa-version --check

# Show detailed version info
veriqa-version --info

# View version history
npm run version:info
```

### Automatic Updates
- 🔍 **Auto-check**: Checks for updates every 24 hours (non-intrusive)
- 📢 **Smart notifications**: Shows update alerts only once per version
- 📊 **Usage analytics**: Anonymous usage tracking for better development
- 🎯 **Update urgency**: Color-coded notifications (🚨 Major, ✨ Minor, 🔧 Patch)

### Update Service (Optional)
```bash
# Start background update service
veriqa-update-service --start

# Check service status
veriqa-update-service --status

# Manual update check
veriqa-update-service --check
```

### Privacy & Analytics
VeriQA collects anonymous usage data to improve the tool:
- ✅ **Anonymous**: No personal information collected
- ✅ **Local**: Data stored locally on your machine
- ✅ **Minimal**: Only feature usage and version information
- ✅ **GDPR Compliant**: Clear and export your data anytime

```bash
# View your analytics data
node -e "console.log(JSON.stringify(require('./node_modules/veriqa-test-advisor/src/versionAnalytics').prototype.exportAnalytics(), null, 2))"

# Clear analytics data
node -e "new (require('./node_modules/veriqa-test-advisor/src/versionAnalytics'))().clearAnalytics()"
```

## 🤝 Contributing

Contributions are welcome! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

**Copyright (c) 2025 [Kapil Rathore](https://github.com/kapil971390)**

---

<p align="center">
  <b>🇮🇳 Made with ❤️ in India</b><br>
  <sub>Empowering QA teams worldwide with intelligent test automation</sub><br><br>
  <a href="https://github.com/kapil971390">
    <img src="https://img.shields.io/badge/GitHub-kapil971390-blue?style=flat-square&logo=github" alt="GitHub Profile">
  </a>
</p>