@testresult/playwright-reporter/README.md

A Playwright reporter that sends test results to TestResult.co for analysis and tracking

# @testresult/playwright-reporter

A Playwright reporter that sends test results to [TestResult.co](https://testresult.co) for analysis and tracking. Track flaky tests, monitor test performance, and get insights into your test suite reliability.

## Features

- 📊 **Test Analytics**: Comprehensive test result tracking and analysis
- 🔍 **Flaky Test Detection**: Automatically identify and categorize flaky tests
- 📈 **Performance Monitoring**: Track test execution times and trends
- 🎯 **Shard Support**: Built-in support for parallel test execution
- 🔧 **Easy Integration**: Simple configuration with environment variables
- 💾 **Debug Mode**: Local JSON output for debugging and development

## Installation

```bash
npm install @testresult/playwright-reporter --save-dev
```

## Configuration

### 1. Environment Variables

Set up the required environment variables:

```bash
# Required
export API_KEY="your-api-key"
export TEST_SUITE_UUID="your-test-suite-uuid"

# Optional
export API_URL="https://app.testresult.co/api/v1"  # Default API URL
export DEBUG_REPORT="1"                            # Enable debug output
export MUTE_CONSOLE="1"                            # Disable console output
export TEST_RUN_ID="custom-run-id"                 # Custom run identifier
```

### 2. Playwright Configuration

Add the reporter to your `playwright.config.js` or `playwright.config.ts`:

#### TypeScript Configuration

```typescript
import { defineConfig } from "@playwright/test";

export default defineConfig({
  // ... other config
  reporter: [
    ["@testresult/playwright-reporter"],
    ["html"], // You can still use other reporters
  ],
});
```

#### JavaScript Configuration

```javascript
// For JavaScript projects, use the JS version directly
const { defineConfig } = require("@playwright/test");

module.exports = defineConfig({
  // ... other config
  reporter: [
    ["./node_modules/@testresult/playwright-reporter/testresult-reporter.js"],
    ["html"],
  ],
});
```

### 3. Advanced Configuration

You can also configure the reporter with options:

```typescript
import { defineConfig } from "@playwright/test";

export default defineConfig({
  reporter: [
    [
      "@testresult/playwright-reporter",
      {
        debugReport: true, // Enable debug output
        muteConsole: false, // Enable console output
      },
    ],
  ],
});
```

## Getting Started

1. **Create an Account**: Sign up at [TestResult.co](https://testresult.co)
2. **Get Your API Key**: Generate an API key in your account settings
3. **Create a Test Suite**: Create a new test suite and note the UUID
4. **Configure Environment**: Set the `API_KEY` and `TEST_SUITE_UUID` variables
5. **Run Tests**: Execute your Playwright tests as usual

```bash
# Set environment variables
export API_KEY="your-api-key-here"
export TEST_SUITE_UUID="your-test-suite-uuid-here"

# Run your tests
npx playwright test
```

## Usage Examples

### Basic Usage

```bash
# Set required environment variables
export API_KEY="tr_1234567890abcdef"
export TEST_SUITE_UUID="550e8400-e29b-41d4-a716-446655440000"

# Run tests
npx playwright test
```

### With Debug Output

```bash
export API_KEY="tr_1234567890abcdef"
export TEST_SUITE_UUID="550e8400-e29b-41d4-a716-446655440000"
export DEBUG_REPORT="1"

npx playwright test
```

### Sharded Test Execution

```bash
# Shard 1 of 4
npx playwright test --shard=1/4

# Shard 2 of 4
npx playwright test --shard=2/4
```

The reporter automatically detects shard information and groups results appropriately.

## Debug Mode

When `DEBUG_REPORT=1` is set, the reporter will:

- Output detailed information to the console
- Save test results to local JSON files in `playwright-debug/` directory
- Show the exact payload being sent to TestResult.co

This is useful for troubleshooting and development.

## Output

The reporter provides colorized console output showing:

- Individual test results with pass/fail status
- Test run summary with duration and totals
- Shard information (if applicable)
- Processing status and completion confirmation

Example output:

```
[chromium] should login successfully - PASSED
[chromium] should handle invalid credentials - FAILED
[firefox] should login successfully - PASSED

📊 Test Run Summary:
Duration: 45.23s
Total Tests: 127
Status: FAILED
Shard: 1/4

⏳ Processing results...
✅ Processing completed in 2.3s
✅ Test results uploaded successfully
```

## Environment Variables Reference

| Variable          | Required | Default                            | Description                             |
| ----------------- | -------- | ---------------------------------- | --------------------------------------- |
| `API_KEY`         | Yes      | -                                  | Your TestResult.co API key              |
| `TEST_SUITE_UUID` | Yes      | -                                  | UUID of your test suite                 |
| `API_URL`         | No       | `https://app.testresult.co/api/v1` | TestResult.co API URL                   |
| `DEBUG_REPORT`    | No       | `0`                                | Enable debug output (`1` to enable)     |
| `MUTE_CONSOLE`    | No       | `0`                                | Disable console output (`1` to disable) |
| `TEST_RUN_ID`     | No       | Auto-generated                     | Custom test run identifier              |

## Troubleshooting

### Common Issues

**Authentication Error**: Verify your `API_KEY` is correct and has proper permissions.

**Test Suite Not Found**: Ensure `TEST_SUITE_UUID` matches an existing test suite in your account.

**Connection Issues**: Check your network connection and verify the `API_URL` if using a custom endpoint.

**TypeScript Errors**: Make sure `@playwright/test` is installed as a peer dependency.

### Debug Mode

Enable debug mode to see detailed information:

```bash
export DEBUG_REPORT="1"
npx playwright test
```

This will create JSON files in `playwright-debug/` with the exact data being sent.

## Support

- **Documentation**: [TestResult.co Documentation](https://docs.testresult.co)
- **Support**: [Contact Support](https://testresult.co/support)

## License

MIT License - see LICENSE file for details.