@vizzly-testing/cli/docs/tdd-mode.md

Visual review platform for UI developers and designers

# TDD Mode Guide

TDD Mode enables test-driven visual development with an interactive dashboard for rapid iteration.

## What is TDD Mode?

TDD Mode transforms your visual testing workflow with:

- **Interactive Dashboard** - Real-time visual feedback as tests run
- **Local Comparison** - Compares screenshots on your machine using `honeydiff`
- **Live Updates** - See comparisons instantly in the browser
- **Baseline Management** - Accept/reject changes directly from the UI
- **Settings Editor** - Adjust comparison threshold, ports, and more without touching config files
- **Project Tools** - Login and link directories to cloud projects from the dashboard
- **Fast Feedback** - No network uploads during development
- **No Token Required** - Visual testing works entirely offline for local development

## Quick Start

### 1. Start the TDD Server

Start the interactive TDD server:

```bash
npx vizzly tdd start
```

🐻 **TDD server starts:**
- Opens at `http://localhost:47392` (or custom `--port`)
- Dashboard shows empty state ready for comparisons
- Runs in the background and returns your terminal immediately
- Settings and Projects tabs available for convenient configuration

### 2. Run Your Tests in Watch Mode

In the same terminal or a new one, run your tests in watch mode:

```bash
npm test -- --watch
```

🐻 **As tests run:**
- Screenshots sent to dashboard in real-time
- Comparisons appear instantly
- Live pass/fail statistics update
- Filter by status (all/changed/identical/new)

### 3. Review Changes in the Dashboard

Open your browser to `http://localhost:47392`:

- **Visual Diff Modes** - Overlay, side-by-side, onion skin, toggle
- **Accept Baselines** - Click to accept individual or all changes
- **Test Statistics** - Total tests, pass rate, change detection
- **Dark Theme** - Easy on the eyes during development

### 4. Accept Changes (Update Baseline)

Accept changes directly in the dashboard UI, or via CLI:

```bash
npx vizzly tdd run "npm test" --set-baseline
```

🐻 **Baseline update behavior:**
- Skips all comparisons
- Sets current screenshots as new baselines
- All tests pass (baseline accepted)
- Future runs use updated baselines

### 5. Manage Settings & Projects (Optional)

Use the dashboard tabs for convenient configuration:

- **Settings** - Edit comparison thresholds, ports, and build settings without touching files
- **Projects** - Login to Vizzly cloud and link directories to projects
- **Comparisons** - View visual diffs (main view)
- **Stats** - See test metrics and trends

These tools are conveniences - you can always edit `vizzly.config.js` directly or use `vizzly login`/`vizzly project:select` CLI commands instead.

### 6. Stop the TDD Server

When done developing:

```bash
npx vizzly tdd stop
```

Or press `Ctrl+C` if running in foreground.

## How It Works

TDD Mode provides two workflows:

### Interactive TDD Workflow

1. **Start TDD server** - `vizzly tdd start` launches persistent server with dashboard
2. **Run tests in watch** - Tests run continuously as you code
3. **Live updates** - Screenshots compared and displayed in real-time
4. **Review in browser** - Visual diff modes help analyze changes
5. **Manage settings (optional)** - Use Settings/Projects tabs for quick config changes
6. **Accept baselines** - Click to update baselines from UI

### Single-Shot Workflow

1. **Run tests** - `vizzly tdd run "npm test"` executes once
2. **Compares locally** - Uses `honeydiff` for high-performance comparison
3. **Generates report** - Creates self-contained HTML report with React UI
4. **Exit with status** - Fails if differences exceed threshold
5. **Server auto-stops** - Ephemeral server cleans up automatically

## Directory Structure

TDD Mode creates a `.vizzly/` directory:

```
.vizzly/
├── baselines/           # Baseline images (local or downloaded)
│   ├── homepage.png
│   ├── dashboard.png
│   └── metadata.json    # Baseline build information
├── current/             # Current test screenshots
│   ├── homepage.png
│   └── dashboard.png
├── report/              # Interactive HTML report
│   └── index.html       # Visual comparison interface
└── diffs/               # Visual diff images (when differences found)
    ├── homepage.png     # Red overlay showing differences
    └── dashboard.png
```

**Important**: Add `.vizzly/` to your `.gitignore` file as it contains local development artifacts.

## Interactive Dashboard

The TDD dashboard provides real-time visual comparison feedback as you develop.

### 🐻 **Dashboard Features**

**Two-View Navigation**
- **Comparisons View** - Main view showing all screenshot comparisons with visual diffs
- **Statistics View** - Overview of test runs, pass/fail metrics, and baseline management
- Switch between views using the navigation tabs at the top

**Live Updates**
- Screenshots appear as tests run
- Comparisons processed in real-time
- No page refresh needed
- Auto-refreshes every 2 seconds to show latest results

**Visual Diff Modes** (in Comparisons view)
- **Overlay** - Toggle diff overlay on/off
- **Side-by-Side** - Compare baseline and current horizontally
- **Onion Skin** - Drag to reveal baseline underneath
- **Toggle** - Click to switch between baseline and current

**Baseline Management**
- **Accept Individual** - Click accept on any comparison to update that baseline
- **Accept All Changes** - Bulk accept all failed/new screenshots at once (shown when changes detected)
  - Shows count of failed and new baselines
  - Prominent button appears in Comparisons view when changes exist
- **Reset Baselines** - Delete all baselines and comparison history (in Statistics view)
  - Useful for starting fresh or fixing corrupted state
  - Requires confirmation before executing

**Filtering & Search** (in Comparisons view)
- Filter by status: All, Failed, Passed, New
- Search by screenshot name
- Filter by browser type
- Filter by viewport size
- Sort by name, status, or diff percentage

### 🐻 **Dashboard UI**

```bash
# Start the dashboard
npx vizzly tdd start

# Opens at http://localhost:47392
# Shows real-time comparisons as tests run
# Dark theme optimized for development
# Navigate between Comparisons and Statistics views
```

**Dashboard Views:**

1. **Comparisons View** (`/`)
   - Lists all screenshot comparisons with visual diffs
   - Filter, search, and sort capabilities
   - "Accept All" button appears when changes are detected
   - Individual accept/reject actions per comparison
   - Multiple visual diff modes for detailed inspection

2. **Statistics View** (`/stats`)
   - Overview of test runs and baseline status
   - Total pass/fail/new screenshot counts
   - Current baseline information (build name, creation date)
   - "Accept All Changes" button for bulk baseline updates
   - "Reset Baselines" button to clear all baselines and start fresh

### 🐻 **Static HTML Report**

When using `vizzly tdd run`, a static HTML report is generated at `.vizzly/report/index.html`:

```bash
# Report path shown after each run
🐻 View detailed report: file:///path/to/.vizzly/report/index.html

# Click the link in your terminal, or open manually
open .vizzly/report/index.html  # macOS
```

## Command Options

### TDD Subcommands

**Start Dashboard Server**
```bash
vizzly tdd start [options]
```

Starts the dashboard server in the background as a detached process and returns your terminal immediately.

Options:
- `--port <port>` - Server port (default: 47392)
- `--threshold <number>` - Comparison threshold in CIEDE2000 Delta E (default: 2.0)
- `--baseline-build <id>` - Use specific build as baseline
- `--open` - Auto-open dashboard in browser

**Run Tests (Single-Shot)**
```bash
vizzly tdd run "npm test" [options]
```

Options:
- `--set-baseline` - Accept screenshots as new baseline
- `--port <port>` - Server port (default: 47392)
- `--threshold <number>` - Comparison threshold in CIEDE2000 Delta E (default: 2.0)
- `--baseline-build <id>` - Use specific build as baseline
- `--timeout <ms>` - Server timeout (default: 30000)

**Stop Dashboard Server**
```bash
vizzly tdd stop
```

### Legacy Command

The legacy command format is still supported:

```bash
vizzly tdd "npm test"  # Equivalent to: vizzly tdd run "npm test"
```

## TDD Workflow

### Interactive TDD (Recommended)

```bash
# Start dashboard (runs in background)
npx vizzly tdd start

# Run tests in watch mode (same terminal or new one)
npm test -- --watch

# Browser: Open http://localhost:47392
# See live comparisons as you code

# Accept changes from dashboard UI when ready
# Stop when done: npx vizzly tdd stop
```

### Single-Shot Testing

```bash
# Run tests once with comparison
npx vizzly tdd run "npm test"

# Accept changes as new baseline
npx vizzly tdd run "npm test" --set-baseline

# Upload to Vizzly when satisfied
npx vizzly run "npm test" --wait
```

### Feature TDD Workflow

```bash
# Start interactive dashboard (runs in background)
npx vizzly tdd start

# Run tests in watch mode (same terminal or new one)
npm test -- --watch

# Make changes and see live feedback in browser
# Accept baselines directly from dashboard UI

# When feature complete, upload to Vizzly
npx vizzly run "npm test" --build-name "Feature: New Dashboard"
```

### Bug Fixing

```bash
# Quick verification with single-shot mode
npx vizzly tdd run "npm test"

# Tests should pass when bug is fixed
# Then upload the fix
npx vizzly run "npm test" --build-name "Fix: Header alignment issue"
```

## Hotspot Filtering

When connected to Vizzly cloud, TDD mode automatically filters out "noise" from known hotspot areas - regions that frequently change across builds (like timestamps, animations, or dynamic content).

### How It Works

1. **Download baselines** - Use the TDD dashboard's Builds page to download baselines from the cloud (hotspot data is included automatically)
2. **Automatic filtering** - During comparisons, if a diff falls within a known hotspot region, it's automatically marked as passed
3. **Visual feedback** - You'll see output like:
   ```
   ✅ PASSED Dashboard - differences in known hotspots (0.15% different, 42 pixels, 1 region, 95% in hotspots)
   ```

### Requirements

Hotspot filtering activates automatically when:
- You have an API token configured (`vizzly login` or `VIZZLY_TOKEN`)
- You've downloaded baselines from the cloud (via the TDD dashboard's Builds page)
- The cloud has enough historical build data to calculate hotspot regions

### Filtering Criteria

A diff is filtered (auto-passed) when:
- **80%+ of the diff** falls within known hotspot regions
- **High confidence** hotspot data (confidence score ≥ 70)

If the diff falls outside hotspots or confidence is low, the comparison fails normally so you can review it.

### Benefits

- **Reduced noise** - Stop seeing the same timestamp/animation diffs over and over
- **Faster reviews** - Focus on real visual changes, not known dynamic areas
- **Smart detection** - Hotspots are calculated from your actual build history, not manual configuration

## Comparison Settings

TDD Mode uses the same comparison settings as production:

- **Threshold matching** - Uses your configured threshold
- **Anti-aliasing detection** - Handles font rendering differences
- **Color tolerance** - Accounts for minor color variations

Configure in `vizzly.config.js`:

```javascript
export default {
  comparison: {
    threshold: 2.0        // CIEDE2000 Delta E (0=exact, 1=JND, 2=recommended)
  }
};
```

## Managing Baselines

### Check Current Baseline Status

```bash
npx vizzly status  # Shows latest build info
```

### Update Baselines

Download new baselines from a different build:

```bash
npx vizzly tdd run "npm test" --baseline-build build-xyz789
```

### Force Baseline Refresh

Delete local baselines to force re-download:

```bash
rm -rf .vizzly/baselines/
npx vizzly tdd run "npm test"
```

## Baseline Signature Properties

Vizzly matches screenshots to baselines using a **signature**:

```
name | viewport_width | browser
```

For example: `homepage|1920|chromium`

### Custom Properties vs Baseline Signature Properties

You can pass custom properties when capturing screenshots:

```javascript
await vizzlyScreenshot('dashboard', screenshot, {
  theme: 'dark',
  locale: 'en-US',
  mobile: true
});
```

By default, these properties help you organize and filter in the dashboard—they don't affect baseline matching. Two screenshots with the same name but different `theme` values would match the same baseline.

### Making Properties Affect Matching

To create separate baselines for different variants, configure **Baseline Signature Properties** in your project settings:

1. Go to your project on [app.vizzly.dev](https://app.vizzly.dev)
2. Navigate to **Settings** → **Baseline Signature Properties**
3. Add the property names (e.g., `theme`, `mobile`)

With `theme` configured:

```
dashboard|1920|chromium|dark    → separate baseline
dashboard|1920|chromium|light   → separate baseline
```

### TDD Mode Support

When you download cloud baselines, the CLI automatically:

1. Fetches your project's baseline signature properties
2. Downloads baselines with variant-aware filenames
3. Uses matching signatures during comparison

This keeps TDD mode behavior consistent with cloud comparisons.

### Quick Reference

| Type | Purpose | Affects Matching? |
|------|---------|-------------------|
| Custom Properties | Organize and filter in dashboard | No |
| Baseline Signature Properties | Create separate baselines per variant | Yes |

## Advanced Usage

### Conditional TDD Mode

Check if TDD mode is active in your tests:

```javascript
import { getVizzlyInfo } from '@vizzly-testing/cli/client';

const info = getVizzlyInfo();
if (info.tddMode) {
  console.log('Running in TDD mode - fast local comparison');
} else {
  console.log('Running in upload mode - results going to Vizzly');
}
```

### Custom Comparison Logic

TDD mode respects your screenshot properties:

```javascript
await vizzlyScreenshot('homepage', screenshot, {
  threshold: 0.05,  // More tolerant for this specific screenshot
  browser: 'chrome',
  viewport: '1920x1080'
});
```

## CI/CD Integration

Use TDD mode for faster PR builds:

```yaml
name: Visual Tests
on: [push, pull_request]

jobs:
  visual-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
      - run: npm ci

      # Use TDD mode for PR builds (faster, no uploads)
      - name: TDD Visual Tests (PR)
        if: github.event_name == 'pull_request'
        run: npx vizzly tdd run "npm test"
        env:
          VIZZLY_TOKEN: ${{ secrets.VIZZLY_TOKEN }}

      # Upload full build for main branch
      - name: Full Visual Tests (main)
        if: github.ref == 'refs/heads/main'
        run: npx vizzly run "npm test" --wait
        env:
          VIZZLY_TOKEN: ${{ secrets.VIZZLY_TOKEN }}
```

## Benefits

### Speed
- **No network uploads** - Everything happens locally
- **Immediate feedback** - See results in seconds
- **No API rate limits** - Test as often as needed

### TDD Experience
- **Fast iteration** - Make changes and test immediately
- **Visual debugging** - See exact pixel differences
- **Offline capable** - Works without internet (after initial baseline download)
- **Clean console output** - Reduced logging noise, only shows important information
- **Silent mode** - Vizzly client auto-disables after first warning if not initialized

### Cost Efficiency
- **Reduced API usage** - Only upload final results
- **Faster CI builds** - Use TDD mode for PR validation
- **Local development** - No cloud resources consumed

## Troubleshooting

### No Baseline Found
```
Error: No baseline found for screenshot 'homepage'
```

**Solution**: Run a successful build first to create baselines:
```bash
npx vizzly run "npm test" --wait
```

### Visual Difference Detected
```
Error: Visual difference detected in 'homepage' (threshold: 1.5%)
```

This is expected behavior! Check the comparison image:
```bash
open .vizzly/comparisons/homepage.png
```

Fix the visual issue or update your baseline if the change is intentional.

### Comparison Failed
```
Error: Failed to compare 'homepage': baseline image not found
```

**Solution**: Refresh baselines:
```bash
rm -rf .vizzly/baselines/
npx vizzly tdd run "npm test"
```

### Honeydiff Not Found
```
Error: Cannot find module '@vizzly-testing/honeydiff'
```

**Solution**: The `@vizzly-testing/honeydiff` package should be installed automatically. Try:
```bash
npm install @vizzly-testing/honeydiff
```

## Best Practices

### Use TDD Mode For
- **Local development** - Fast iteration on UI changes
- **Bug fixing** - Verify visual fixes immediately
- **PR validation** - Quick checks without uploading
- **Debugging** - Understand exactly what changed visually

### Use Upload Mode For
- **Final results** - When changes are ready for review
- **Baseline creation** - Initial setup and approved changes
- **Production releases** - Official visual regression testing
- **Team collaboration** - Sharing results with designers/stakeholders

### Directory Management
- **Add to .gitignore** - Never commit `.vizzly/` directory
- **Regular cleanup** - Delete old comparison images periodically
- **Baseline updates** - Refresh baselines when UI intentionally changes

## Next Steps

- Learn about [Test Integration](./test-integration.md) for screenshot capture
- Explore [Upload Command](./upload-command.md) for direct uploads
- Check the [API Reference](./api-reference.md) for programmatic usage