@bramato/openrouter-mock-generator/README.md

AI-powered mock data generator using OpenRouter API with JSON mode support

# OpenRouter AI Mock Generator

[![npm version](https://badge.fury.io/js/%40bramato%2Fopenrouter-mock-generator.svg)](https://www.npmjs.com/package/@bramato/openrouter-mock-generator)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.3.0-blue.svg)](https://www.typescriptlang.org/)
[![OpenRouter Agents](https://img.shields.io/badge/OpenRouter--Agents-v1.2.1-green.svg)](https://github.com/bramato/openrouter-agents)

**Generate realistic mock data with AI-powered intelligence and professional image processing.**

Transform your development workflow with an advanced mock data generator that combines state-of-the-art language models with intelligent image processing. Perfect for frontend development, testing, prototyping, and creating compelling demos with realistic content.

> 🚨 **BREAKING CHANGES in v2.0.0**: This version introduces a new architecture powered by [OpenRouter Agents](https://github.com/bramato/openrouter-agents) for enhanced AI capabilities and specialized agents. See [Migration Guide](#-migration-from-v1x-to-v200) below.

## Why Choose This Generator?

### 🎯 **Intelligent Data Generation**
- Analyzes your JSON structure and generates contextually appropriate data
- Supports complex nested objects and arrays
- Maintains data relationships and patterns from your examples

### 🖼️ **Professional Image Processing**
- Replaces placeholder images with AI-generated, contextually relevant images
- Automatic cloud storage integration (AWS S3, DigitalOcean Spaces)
- Smart optimization reduces image generation costs by up to 30%

### 🚀 **Developer-First Experience**
- Simple CLI commands get you started in minutes
- Comprehensive TypeScript API for programmatic usage
- Interactive configuration wizard handles complex setup

### 🤖 **NEW v2.0.0: Powered by OpenRouter Agents**
- **Specialized AI Agents**: Uses dedicated MockDataAgent for intelligent data generation
- **Enhanced Architecture**: Modular design with better separation of concerns
- **Future-Ready**: Access to the entire OpenRouter Agents ecosystem
- **Backward Compatible**: Same CLI and API interface you know and love

---

## Quick Installation

```bash
npm install @bramato/openrouter-mock-generator
```

## Getting Started

### Step 1: Initial Setup

Run the interactive configuration wizard to set up your API keys and preferences:

```bash
npx ai-init
```

The wizard will configure:
- **OpenRouter API Key** (Required) - For AI text generation
- **Model Selection** - Choose optimal models for your use case
- **Image Processing** (Optional) - Hugging Face API + Cloud storage setup

### Step 2: Generate Your First Mock Data

```bash
# Try the included example file (users, products, orders)
npx ai-generate-mock example.json --count 20

# Basic generation with placeholder images
npx ai-generate-mock your-sample.json --count 50

# Professional generation with AI images (requires setup)
npx ai-generate-mock products.json --images --count 25
```

### Step 3: Customize Your Output

```bash
# Add custom preferences for more targeted data
npx ai-generate-mock products.json \
  --preferences "Luxury Italian fashion brands with premium pricing" \
  --count 30 \
  --images
```

**That's it!** Your mock data is generated and ready to use in your application.

## CLI Reference

### Core Commands

| Command | Description | Example |
|---------|-------------|---------|
| `ai-init` | Interactive setup wizard | `npx ai-init` |
| `ai-generate-mock` | Generate mock data from JSON sample | `npx ai-generate-mock data.json --count 50` |

### Command Options

| Option | Short | Description | Example |
|--------|-------|-------------|---------|
| `--count` | `-c` | Number of items to generate | `--count 100` |
| `--output` | `-o` | Output file path | `--output results.json` |
| `--preferences` | `--pref` | Custom generation instructions | `--pref "Italian luxury brands"` |
| `--array-path` | `-p` | Target specific JSON array | `--array-path "data.products"` |
| `--images` | | Force enable AI image processing | `--images` |
| `--no-images` | | Force disable AI image processing | `--no-images` |
| `--analyze` | `-a` | Analyze structure without generating | `--analyze` |

### Common Usage Patterns

```bash
# Standard data generation
npx ai-generate-mock products.json --count 50

# Professional images with custom preferences
npx ai-generate-mock catalog.json \
  --images \
  --preferences "High-end electronics with professional photography" \
  --count 25

# Target specific array in complex JSON
npx ai-generate-mock complex-data.json \
  --array-path "store.inventory.products" \
  --count 100 \
  --output inventory.json

# Analyze before generating
npx ai-generate-mock data.json --analyze
```

## TypeScript/JavaScript API

### Basic Implementation (v2.0.0+)

```typescript
import { MockGeneratorService } from '@bramato/openrouter-mock-generator';

// MockGeneratorService now uses OpenRouter Agents internally
const generator = new MockGeneratorService();

const result = await generator.generateMockData({
  inputFile: 'products.json',
  outputFile: 'generated_products.json',
  count: 50,
  preferences: 'Italian fashion brands with euro pricing'
});

if (result.success) {
  console.log(`✅ Generated ${result.generatedCount} items`);
  console.log(`🤖 Powered by MockDataAgent from openrouter-agents`);
} else {
  console.error(`❌ Generation failed: ${result.error}`);
}
```

### Advanced Configuration with OpenRouter Agents

```typescript
import { MockGeneratorService } from '@bramato/openrouter-mock-generator';
import { AgentManager } from 'openrouter-agents';

// Direct access to the underlying Agent ecosystem (v2.0.0+)
const agentManager = new AgentManager({
  openRouter: {
    apiKey: 'your-api-key',
    baseURL: 'https://openrouter.ai/api/v1'
  }
});

// Get the specialized MockDataAgent
const mockAgent = agentManager.getAgent('mock-data');

// Traditional API still works (uses agents internally)
const generator = new MockGeneratorService(
  undefined, // Use default OpenRouter config
  true       // Enable image processing
);

const result = await generator.generateMockData({
  inputFile: 'catalog.json',
  outputFile: 'enhanced_catalog.json',
  count: 25,
  preferences: 'Luxury electronics with professional product photography',
  enableImageProcessing: true
});
```

### Using MockDataAgent Directly (New v2.0.0 Feature)

```typescript
import { MockDataAgent } from 'openrouter-agents';

// Create specialized mock data agent
const agent = new MockDataAgent({
  name: 'product-generator',
  openRouter: {
    apiKey: process.env.OPENROUTER_API_KEY!,
    model: 'anthropic/claude-3.5-sonnet'
  },
  temperature: 0.7
});

// Generate structured data
const response = await agent.process({
  input: {
    template: { name: 'string', price: 'number', category: 'string' },
    count: 10,
    preferences: 'Italian luxury fashion'
  },
  options: { forceJson: true }
});

console.log(response.data); // Structured JSON array
```

### Standalone Image Processing

Transform existing mock data by replacing placeholder images with AI-generated ones:

```typescript
import { PostProcessingOrchestrator } from '@bramato/openrouter-mock-generator';
import fs from 'fs';

const processor = new PostProcessingOrchestrator('your-hf-api-key', {
  verbose: true,
  enableOptimization: true,
  uploadToCloud: true
});

// Load existing mock data
const mockData = JSON.parse(fs.readFileSync('existing-data.json', 'utf8'));

// Process images
const result = await processor.processData(mockData);

if (result.success) {
  // Save enhanced data with real images
  fs.writeFileSync('enhanced-data.json', 
    JSON.stringify(result.processedData, null, 2)
  );
  console.log('✅ Image processing complete');
}
```

## Configuration

### Required Environment Variables

```bash
# OpenRouter API (Required)
OPENROUTER_API_KEY=your-openrouter-api-key
```

### Optional Configuration

#### AI Model Settings
```bash
OPENROUTER_DEFAULT_MODEL=anthropic/claude-3.5-sonnet
OPENROUTER_MOCK_GENERATOR_MODEL=openai/gpt-4o-mini
```

#### AI Image Processing
```bash
HUGGINGFACE_API_KEY=your-huggingface-token
STORAGE_PROVIDER=aws  # or 'digitalocean'
```

#### AWS S3 Storage
```bash
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1
AWS_S3_BUCKET_NAME=your-bucket-name
AWS_S3_ENDPOINT=custom-endpoint  # optional
```

#### DigitalOcean Spaces Storage
```bash
DO_SPACES_ACCESS_KEY=your-access-key
DO_SPACES_SECRET_KEY=your-secret-key
DO_SPACES_REGION=fra1
DO_SPACES_NAME=your-space-name
DO_API_TOKEN=your-api-token  # optional
```

> **💡 Tip:** Use the `npx ai-init` wizard to set up these variables interactively.

## Supported AI Models

**Automatic JSON-mode compatibility** - Only models that support structured JSON output are used:

| Provider | Models | Best For |
|----------|--------|----------|
| **OpenAI** | GPT-4o, GPT-4o Mini, GPT-4 Turbo, GPT-3.5 Turbo | General purpose, consistent output |
| **Anthropic** | Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Haiku | Complex reasoning, detailed content |
| **Google** | Gemini Pro, Gemini Flash | Fast generation, multilingual |
| **Meta** | Llama models via OpenRouter | Cost-effective, good performance |

> The system automatically selects the best model based on your configuration and task complexity.

---

## Real-World Examples

### 🚀 Complete Example: Multi-Entity System

Try the included example file to see the full power of the generator:

```bash
# Generate realistic data for users, products, and orders
npx ai-generate-mock example.json --count 20 --images

# Or target a specific entity type
npx ai-generate-mock example.json --count 50 --array-path users
npx ai-generate-mock example.json --count 30 --array-path products --images
```

**Using the included `example.json`** which contains:
- **👥 User profiles** with addresses, preferences, and social data
- **📱 Products** with specifications, ratings, and image galleries  
- **📦 Orders** with shipping, payment, and tracking information

**Generated results include:**
- Professional user avatars and product photography
- Contextual data relationships (orders link to existing users/products)
- Italian locale with realistic names and addresses
- Complex nested objects and arrays
- Consistent image seeds for repeatable results

**Sample generated user:**
```json
{
  "id": 2,
  "name": "Giulia Bianchi",
  "email": "giulia.bianchi@example.com",
  "age": 28,
  "avatar": "https://picsum.photos/200/200?random=giulia",
  "address": {
    "street": "Corso Buenos Aires 45",
    "city": "Milano", 
    "zipCode": "20124",
    "country": "Italia"
  },
  "preferences": {
    "newsletter": true,
    "theme": "light",
    "language": "it"
  },
  "profile": {
    "bio": "UX Designer specializzata in interfacce mobile innovative",
    "website": "https://giuliabianchi.design",
    "social": {
      "linkedin": "https://linkedin.com/in/giuliabianchi",
      "github": "https://github.com/giuliabianchi"
    }
  }
}
```

### 🛍️ E-commerce Product Catalog

```bash
npx ai-generate-mock products.json \
  --images \
  --preferences "Luxury Italian fashion from Milano boutiques" \
  --count 30
```

**Generated content includes:**
- Professional product photography
- Brand-appropriate thumbnails and galleries
- Contextual banner images for promotions

### 🍕 Restaurant Menu System

```bash
npx ai-generate-mock menu.json \
  --images \
  --preferences "Traditional Italian regional cuisine with wine pairings" \
  --count 20
```

**Produces:**
- High-quality food photography
- Restaurant ambiance images
- Chef and staff portraits

### 👥 User Management System

```bash
npx ai-generate-mock users.json \
  --images \
  --preferences "Professional Italian business people from major cities" \
  --count 100
```

**Creates:**
- Professional headshot portraits
- Company logos and branding
- Office environment backgrounds

---

## Use Cases

### 🔧 **Development & Testing**
- **Frontend Development**: Realistic data for UI components and layouts
- **API Testing**: Comprehensive datasets for endpoint validation  
- **Database Seeding**: Meaningful sample data for development environments
- **Demo Applications**: Professional-grade content for presentations

### 🎨 **Design & Prototyping**
- **UI/UX Design**: Contextual content for design systems and mockups
- **Client Presentations**: Realistic prototypes with professional imagery
- **A/B Testing**: Varied content sets for testing different approaches
- **Design Validation**: Real-world data scenarios for user testing

### 📈 **Marketing & Content**
- **Campaign Development**: Sample content for marketing materials
- **Portfolio Showcase**: Professional examples for service demonstrations
- **Content Strategy**: Diverse datasets for testing content approaches
- **Brand Guidelines**: Consistent imagery and messaging examples

## Custom Preferences Guide

Enhance your generated data with specific instructions to guide the AI:

### Industry-Specific Examples

```bash
# E-commerce & Retail
--preferences "Luxury Italian fashion brands with Milano designers and euro pricing"

# Food & Hospitality  
--preferences "Traditional Italian regional cuisine with seasonal ingredients and wine pairings"

# Professional Services
--preferences "Italian business professionals from Rome, Milan, and Naples with LinkedIn-style profiles"

# Real Estate
--preferences "Historic Italian properties in city centers with Renaissance and Baroque architecture"

# Technology
--preferences "Italian tech companies and startups with modern, innovation-focused branding"

# Travel & Tourism
--preferences "Italian cultural events, festivals, and tourist attractions with regional authenticity"
```

### Best Practices
- **Be Specific**: Include geographical, cultural, or industry context
- **Set Tone**: Specify whether content should be formal, casual, luxury, budget-friendly, etc.
- **Add Details**: Mention specific attributes like pricing ranges, demographics, or styles
- **Use Examples**: Reference well-known brands or concepts for clearer direction

---

## Advanced AI Image Processing

Transform placeholder images into professional, contextually-appropriate visuals automatically.

### 🔄 **Processing Pipeline**

1. **Smart Detection** - Identifies placeholder image URLs (`https://picsum.photos/*`)
2. **Context Analysis** - Analyzes surrounding JSON data for image context
3. **Intelligent Optimization** - Groups similar images to minimize generation costs
4. **AI Generation** - Creates professional images using FLUX.1-dev and Qwen-Image models
5. **Cloud Integration** - Uploads to AWS S3 or DigitalOcean Spaces with CDN optimization
6. **Seamless Replacement** - Updates JSON with new cloud-hosted URLs

### 📸 **Image Categories**

| Type | Description | Use Cases |
|------|-------------|-----------|
| **Product Photos** | Professional product photography with proper lighting | E-commerce catalogs, inventory |
| **Portrait & Avatars** | High-quality headshots and professional portraits | User profiles, team pages |
| **Banners & Headers** | Wide-format promotional and branding images | Marketing materials, headers |
| **Thumbnails** | Optimized small-format preview images | Card layouts, galleries |
| **Logos & Branding** | Professional brand identity elements | Company profiles, partnerships |
| **Backgrounds** | Contextual environment and texture images | UI backgrounds, hero sections |

### ⚡ **Smart Optimizations**

- **Cost Reduction**: Up to 30% savings through intelligent image reuse
- **Context Awareness**: Images generated based on surrounding data context
- **Batch Efficiency**: Concurrent processing with intelligent rate limiting
- **Graceful Fallbacks**: Maintains placeholder images if generation fails
- **CDN Integration**: Automatic content delivery network optimization


## Troubleshooting

### Common Issues & Solutions

#### 🚫 **Image Processing Not Working**

**Symptoms**: Images remain as placeholder URLs
```bash
# 1. Verify configuration
npx ai-init

# 2. Test with minimal example
npx ai-generate-mock sample.json --images --count 1

# 3. Check environment variables
echo $HUGGINGFACE_API_KEY
echo $STORAGE_PROVIDER
```

#### ☁️ **Cloud Storage Upload Failures**

**Symptoms**: Generated images not accessible via URLs
```bash
# Verify cloud storage configuration
cat .env | grep -E "(STORAGE_PROVIDER|AWS_|DO_)"

# Test permissions
# AWS: Check IAM permissions for S3 bucket access
# DigitalOcean: Verify Spaces access keys and region
```

#### 🔑 **API Authentication Errors**

**OpenRouter Issues**:
- Verify API key at [OpenRouter Dashboard](https://openrouter.ai/keys)
- Check account balance and usage limits

**Hugging Face Issues**:
- Confirm token permissions at [HF Settings](https://huggingface.co/settings/tokens)
- Ensure "Make calls to the serverless Inference API" is enabled

#### 📄 **JSON Processing Errors**

**Symptoms**: "Cannot parse input file" or similar errors
```bash
# Analyze your JSON structure
npx ai-generate-mock your-file.json --analyze

# Common fixes:
# - Ensure valid JSON syntax
# - Check for proper array structures  
# - Verify file encoding (UTF-8)
```

#### ⚡ **Rate Limiting & Performance**

**Symptoms**: Slow generation or timeout errors
- **Reduce count**: Use smaller `--count` values for testing
- **Check quotas**: Review API usage limits on provider dashboards
- **Optimize batching**: System auto-adjusts, but complex data may need smaller batches

#### 🔧 **Environment Configuration**

**Problem**: Environment variables not loading
```bash
# ✅ Correct format
OPENROUTER_API_KEY=your-actual-key

# ❌ Incorrect (spaces around =)
OPENROUTER_API_KEY = your-actual-key
```

**Debug steps**:
1. Check `.env` file exists in project root
2. Verify no extra spaces or quotes around values
3. Restart terminal/application after changes

---

## Frequently Asked Questions

### **API Requirements**

**Q: Do I need both OpenRouter and Hugging Face API keys?**  
A: OpenRouter is required for text generation. Hugging Face is optional - only needed for AI image processing.

**Q: Are there free tiers available?**  
A: Yes! OpenRouter offers free credits for new accounts, and Hugging Face has a generous free tier with rate limits.

### **Storage & Configuration**

**Q: Can I use both AWS S3 and DigitalOcean Spaces?**  
A: No, you must choose one storage provider. Configure this during `npx ai-init` setup.

**Q: What happens if cloud storage fails?**  
A: The system gracefully falls back to placeholder images, ensuring your mock data generation always succeeds.

### **Image Processing**

**Q: Can I process existing mock data with AI images?**  
A: Absolutely! Use the `PostProcessingOrchestrator` class programmatically, or re-run generation with `--images` flag.

**Q: What image formats and sizes are supported?**  
A: Generates PNG images with contextually appropriate dimensions. Supports product photos, avatars, banners, thumbnails, and more.

**Q: How does cost optimization work?**  
A: The system intelligently detects similar image contexts and reuses generated images, reducing generation costs by up to 30%.

### **Technical Details**

**Q: Which AI models work best?**  
A: The system auto-selects optimal models. GPT-4o Mini offers great balance of speed/quality, while Claude 3.5 Sonnet excels at complex data structures.

**Q: Can I use this in production applications?**  
A: Yes! The tool is designed for professional use with proper error handling, rate limiting, and cloud integration.

---

## Architecture Overview

### **Microservices Design**
- **ImageUrlExtractor**: Identifies and categorizes image URLs in JSON data
- **DescriptionGenerator**: Creates contextual AI prompts based on surrounding data
- **ImageProcessingAnalyzer**: Optimizes generation strategy and detects duplicates
- **UrlReplacer**: Seamlessly updates JSON with new cloud-hosted image URLs
- **PostProcessingOrchestrator**: Coordinates all services with intelligent error handling

### **Performance Features**
- **Smart Batching**: Automatic optimization based on data complexity
- **Concurrent Processing**: Parallel image generation with rate limiting
- **CDN Integration**: Automatic content delivery network optimization
- **Intelligent Caching**: Reuses similar images for cost efficiency

## 🔄 Migration from v1.x to v2.0.0

### What Changed

**v2.0.0 introduces a new architecture** powered by [OpenRouter Agents](https://github.com/bramato/openrouter-agents), providing:

- **Specialized AI Agents**: MockDataAgent optimized for data generation
- **Better Modularity**: Cleaner separation between text generation and image processing
- **Future Extensibility**: Access to the growing OpenRouter Agents ecosystem
- **Enhanced Performance**: Improved error handling and response processing

### Breaking Changes

❌ **Removed Dependencies**:
- Internal `openrouter.ts` and `openrouter-api.ts` services (moved to openrouter-agents package)

✅ **Added Dependencies**:
- `openrouter-agents: ^1.2.1` - Core agent functionality

### Migration Steps

1. **Update to v2.0.0**:
   ```bash
   npm install @bramato/openrouter-mock-generator@^2.0.0
   ```

2. **No Code Changes Required**: 
   - All existing CLI commands work unchanged
   - All existing TypeScript/JavaScript APIs remain compatible
   - Configuration and environment variables are identical

3. **Optional: Explore New Features**:
   ```typescript
   // Access the underlying agent system (new in v2.0.0)
   import { MockDataAgent } from 'openrouter-agents';
   
   const agent = new MockDataAgent({
     name: 'my-generator',
     openRouter: { apiKey: 'your-key' }
   });
   ```

### Compatibility Matrix

| Feature | v1.x | v2.0.0 | Notes |
|---------|------|--------|---------|
| CLI Commands | ✅ | ✅ | Identical interface |
| MockGeneratorService API | ✅ | ✅ | Same methods and options |
| Image Processing | ✅ | ✅ | Enhanced performance |
| Environment Variables | ✅ | ✅ | No changes required |
| TypeScript Types | ✅ | ✅ | Improved with agent types |
| **MockDataAgent Access** | ❌ | ✅ | **New:** Direct agent usage |
| **Agent Ecosystem** | ❌ | ✅ | **New:** Access to all OpenRouter Agents |

### Why Upgrade?

- 🎯 **Specialized Agents**: Purpose-built AI agents for better performance
- 🔧 **Better Architecture**: More maintainable and extensible codebase
- 🚀 **Future Features**: Access to new agents as they're released
- 📈 **Enhanced Reliability**: Improved error handling and response processing
- 🔗 **Ecosystem Access**: Use code generators, translators, and more from the same package

### Need Help?

If you encounter issues during migration:
1. Check the [OpenRouter Agents documentation](https://github.com/bramato/openrouter-agents#readme)
2. File an issue on [GitHub Issues](https://github.com/bramato/openrouter-mock-generator/issues)
3. Review the [v2.0.0 changelog](#changelog) below

---

## 📋 Changelog

### v2.0.0 (Latest) - Major Architecture Update
**Release Date**: Current

**🚨 BREAKING CHANGES:**
- Migrated to OpenRouter Agents architecture
- Removed internal OpenRouter service files
- Added `openrouter-agents ^1.2.1` dependency

**✨ NEW FEATURES:**
- **MockDataAgent Integration**: Specialized AI agent for data generation
- **Agent Ecosystem Access**: Direct access to OpenRouter Agents platform
- **Enhanced Architecture**: Improved modularity and extensibility
- **Better Error Handling**: More robust processing with agent-based approach

**🔧 IMPROVEMENTS:**
- Same CLI interface with improved backend performance
- Enhanced TypeScript support with agent types
- Better separation of concerns between text and image generation
- Backward-compatible API maintains all existing functionality

**📦 DEPENDENCIES:**
- Added: `openrouter-agents ^1.2.1`
- Updated: Core dependencies for better compatibility

### v1.3.2 (Previous) - Stability Improvements
- Enhanced image processing pipeline
- Improved error handling for cloud storage
- Better configuration management
- Performance optimizations

---

## License & Support

**License**: MIT - Free for commercial and personal use

**Author**: [bramato](https://github.com/bramato)

### **Contributing**
We welcome contributions! Please feel free to submit issues, feature requests, or pull requests on GitHub.

### **Support**
- 📖 **Documentation**: This README covers all features and use cases
- 🤖 **OpenRouter Agents**: Check the [OpenRouter Agents documentation](https://github.com/bramato/openrouter-agents#readme)
- 🐛 **Bug Reports**: Please use GitHub Issues with detailed reproduction steps
- 💡 **Feature Requests**: Share your ideas through GitHub Discussions
- 🔧 **Configuration Help**: Use the interactive `npx ai-init` wizard

### **Related Projects**
- 🤖 **[OpenRouter Agents](https://github.com/bramato/openrouter-agents)**: The core AI agent platform powering this generator
- 📦 **[NPM Package](https://www.npmjs.com/package/openrouter-agents)**: OpenRouter Agents on NPM
- 🌐 **[OpenRouter Platform](https://openrouter.ai)**: Access to 100+ AI models

---

⭐ **If this tool accelerates your development workflow, please star both repositories:**
- **[Mock Generator](https://github.com/bramato/openrouter-mock-generator)** - This project
- **[OpenRouter Agents](https://github.com/bramato/openrouter-agents)** - The underlying AI platform