@clipwhisperer/common/README.md

ClipWhisperer Common - Shared library providing core utilities, database schemas, authentication, bucket management, and common functionality across all ClipWhisperer microservices

# @clipwhisperer/common

Shared utility library and schema definitions for the ClipWhisperer microservices ecosystem.

## 🎯 Purpose

The `@clipwhisperer/common` library provides shared functionality, type definitions, and validation schemas used across all ClipWhisperer microservices:

- **Hub** - Content management and analytics
- **Narrator** - Text-to-speech processing
- **Renderer** - API rendering and responses
- **Scraper** - Data extraction from external sources

## 📦 Installation

### For ClipWhisperer Services

```bash
npm install @clipwhisperer/common
```

### For Development

```bash
# Clone and setup
git clone https://github.com/ClipWhisperer/common.git
cd common
npm install

# Run tests
npm test

# Build the library
npm run build
```

## 🏗️ Architecture

### Core Modules

```typescript
import {
  // Schema validation
  validateVideoData,
  validateNarratorRequest,
  validateAnalyticsData,

  // Utility functions
  sanitizeInput,
  generateId,
  formatDate,

  // Type definitions
  VideoData,
  NarratorRequest,
  AnalyticsMetrics,

  // Constants
  VIDEO_STATUSES,
  VOICE_ENGINES,
  API_ENDPOINTS,
} from "@clipwhisperer/common";
```

## 📋 Features

### Schema Validation

- **Zod-based validation** - Type-safe runtime validation
- **Video data schemas** - Content metadata validation
- **Narrator request schemas** - Text-to-speech parameter validation
- **Analytics schemas** - Metrics and reporting data validation
- **API response schemas** - Consistent response format validation

### Utility Functions

- **Input sanitization** - XSS protection and data cleaning
- **ID generation** - UUID and custom ID generators
- **Date formatting** - Standardized date/time utilities
- **Error handling** - Consistent error types and handling
- **Validation helpers** - Common validation patterns

### Type Definitions

- **Shared interfaces** - TypeScript definitions for all services
- **Enum definitions** - Status codes, engine types, format options
- **API contracts** - Request/response type definitions
- **Database models** - Shared entity definitions

### Constants & Configuration

- **API endpoints** - Centralized endpoint definitions
- **Status codes** - HTTP and application status constants
- **Voice engines** - Supported TTS engine configurations
- **Default values** - Shared default configurations

## 🧪 Testing

The Common library maintains comprehensive test coverage:

- **Total Tests**: 63/63 passing (100% pass rate)
- **Test Categories**:
  - Schema validation tests
  - Utility function tests
  - Type safety tests
  - Integration tests

### Running Tests

```bash
# Run all tests
npm test

# Run tests with coverage
npm run test:coverage

# Run tests in watch mode
npm run test:watch

# Run specific test file
npm test -- validation.test.ts
```

## 📊 Usage Examples

### Schema Validation

```typescript
import { validateVideoData, VideoDataSchema } from "@clipwhisperer/common";

// Validate video data
const videoData = {
  title: "Sample Video",
  duration: 120,
  source: "youtube",
  url: "https://youtube.com/watch?v=example",
};

try {
  const validatedData = validateVideoData(videoData);
  console.log("Valid video data:", validatedData);
} catch (error) {
  console.error("Validation failed:", error.message);
}
```

### Utility Functions

```typescript
import { sanitizeInput, generateId, formatDate } from "@clipwhisperer/common";

// Sanitize user input
const cleanInput = sanitizeInput('<script>alert("xss")</script>Hello World');
// Result: "Hello World"

// Generate unique ID
const videoId = generateId("video");
// Result: "video_1a2b3c4d-5e6f-7g8h-9i0j-k1l2m3n4o5p6"

// Format date consistently
const formattedDate = formatDate(new Date(), "iso");
// Result: "2024-01-15T10:30:00.000Z"
```

### Type Definitions

```typescript
import {
  VideoData,
  NarratorRequest,
  AnalyticsMetrics,
} from "@clipwhisperer/common";

// Use shared types
const video: VideoData = {
  id: "video_123",
  title: "Example Video",
  duration: 180,
  status: "processed",
  metadata: {
    views: 1000,
    likes: 50,
    source: "youtube",
  },
};

const narratorJob: NarratorRequest = {
  text: "Hello, this is a sample narration.",
  voice: "Matthew",
  engine: "generative",
  format: "mp3",
  options: {
    speed: 1.0,
    pitch: 0,
  },
};
```

## 🔧 Development

### Project Structure

```
common/
├── src/
│   ├── schemas/           # Zod validation schemas
│   │   ├── video.ts
│   │   ├── narrator.ts
│   │   └── analytics.ts
│   ├── utils/             # Utility functions
│   │   ├── sanitize.ts
│   │   ├── validation.ts
│   │   └── formatting.ts
│   ├── types/             # TypeScript type definitions
│   │   ├── video.ts
│   │   ├── narrator.ts
│   │   └── analytics.ts
│   ├── constants/         # Shared constants
│   │   ├── statuses.ts
│   │   ├── endpoints.ts
│   │   └── defaults.ts
│   └── index.ts           # Main export file
├── dist/                  # Compiled JavaScript
├── coverage/              # Test coverage reports
├── __tests__/             # Test files
│   ├── schemas/
│   ├── utils/
│   └── integration/
├── package.json
├── tsconfig.json
├── jest.config.js
└── README.md
```

### Build Process

```bash
# Development build
npm run build

# Production build
npm run build:prod

# Clean build artifacts
npm run clean

# Lint code
npm run lint

# Format code
npm run format
```

### Publishing

```bash
# Version bump and publish
npm version patch
npm publish

# Or use the ClipWhisperer management script
cd ../Utils
npm start
> 2 (update Common library across all projects)
```

## 🔄 Integration with ClipWhisperer Services

### Hub Service

- Video data validation
- Analytics schema validation
- Database model types
- API response formatting

### Narrator Service

- TTS request validation
- Voice/engine compatibility checking
- Audio format validation
- Job queue type definitions

### Renderer Service

- API route schema validation
- Response formatting utilities
- Error handling types
- Status code constants

### Scraper Service

- Content validation schemas
- Data sanitization utilities
- Source format definitions
- Extraction result types

## 📈 Version Management

The Common library follows semantic versioning and is automatically managed across all ClipWhisperer services:

```bash
# Check current versions across services
cd ../Utils && npm start
# Press Enter to see current versions

# Update Common library in all services
cd ../Utils && npm start
> 2 (update @clipwhisperer/common library)

# Check for version inconsistencies
cd ../Utils && npm start
```

## 🤝 Contributing

### Development Workflow

1. **Make changes** to the Common library
2. **Run tests** to ensure nothing breaks: `npm test`
3. **Build the project**: `npm run build`
4. **Update version**: `npm version patch`
5. **Publish to npm**: `npm publish`
6. **Update across services**:
   ```bash
   cd ../Utils
   npm start
   > 2 (update Common library)
   ```

### Guidelines

- **Maintain backward compatibility** when possible
- **Add tests** for new functionality
- **Update TypeScript types** for new schemas
- **Document breaking changes** in version notes
- **Test integration** with all services before publishing

## 📚 API Reference

### Validation Functions

- `validateVideoData(data)` - Validates video metadata
- `validateNarratorRequest(request)` - Validates TTS requests
- `validateAnalyticsData(data)` - Validates analytics metrics
- `validateApiResponse(response)` - Validates API responses

### Utility Functions

- `sanitizeInput(input)` - Removes XSS and dangerous content
- `generateId(prefix?)` - Generates unique identifiers
- `formatDate(date, format)` - Formats dates consistently
- `parseVideoUrl(url)` - Extracts video info from URLs
- `calculateEngagementRate(metrics)` - Computes engagement metrics

### Constants

- `VIDEO_STATUSES` - Valid video status values
- `VOICE_ENGINES` - Supported TTS engines
- `API_ENDPOINTS` - Service endpoint definitions
- `ERROR_CODES` - Standardized error codes
- `DEFAULT_CONFIG` - Default configuration values

## 🔒 Security

- **Input validation** - All user inputs validated before processing
- **XSS protection** - Built-in sanitization utilities
- **Type safety** - Runtime validation with compile-time types
- **No sensitive data** - Contains no secrets or credentials

## 📄 License

MIT License - Part of the ClipWhisperer ecosystem.

## 🐛 Troubleshooting

For issues related to the Common library:

1. Check the test results: `npm test`
2. Review integration with services using the management script
3. Verify version consistency across projects
4. Check the main project status report for dependency issues

---

**Part of the ClipWhisperer Microservices Ecosystem**  
Managed via [ClipWhisperer/Utils]() - Centralized project management tools