@yogesh0333/yogiway-prompt/README.md

Free & Open Source Prompt Optimization Library - Save 30-50% on AI API costs. Multi-language, multi-platform support.

# @yogesh0333/yogiway-prompt

**Free & Open Source Prompt Optimization Library** - Save 30-50% on AI API costs!

## 🎯 What is YOGIWAY-PROMPT?

YOGIWAY-PROMPT optimizes **text prompts** (instructions, questions, descriptions) by removing redundancy, compressing whitespace, and improving clarity - while preserving meaning.

**Key Difference from YOGIWAY:**
- **YOGIWAY** = Compresses **structured data** (JSON, objects) → Custom format
- **YOGIWAY-PROMPT** = Optimizes **text prompts** (instructions) → Optimized text

**Use both together for maximum savings (50-70%)!** 🚀

### ⚠️ Important Notes

- **Token counting is approximate** (±5-10% accuracy) - verify with actual API responses
- **Optimization optimized for English** - other languages see varying effectiveness
- **Test optimized prompts** - especially when using aggressive mode
- **Pricing is approximate** - verify current pricing with providers
- See [Important Notes & Limitations](#-important-notes--limitations) section for details

---

## 🚀 Features

- ✅ **Multi-Language Support** - TypeScript, JavaScript, Python
- ✅ **Multi-Platform** - Node.js, Browser, Deno, Bun
- ✅ **Token Optimization** - Save 30-50% on API costs
- ✅ **Context Window Management** - Smart chunking for large prompts
- ✅ **Cost Calculator** - Estimate costs across providers
- ✅ **Provider Support** - OpenAI, Anthropic, Google, Cohere, Mistral
- ✅ **100% Free** - All premium features for free
- ✅ **Type-Safe** - Full TypeScript support
- ✅ **Zero Dependencies** - Lightweight and fast
- ✅ **Preserves Code Blocks** - Keeps code intact
- ✅ **Preserves Markdown** - Maintains formatting

---

## 📦 Installation

```bash
npm install @yogesh0333/yogiway-prompt
```

### Python (Coming Soon)

```bash
pip install yogiway-prompt
```

---

## 🎯 Quick Start

### Basic Optimization

```typescript
import { optimize } from '@yogesh0333/yogiway-prompt';

const prompt = `
Please note that it is very important to understand that we need to 
process this data carefully. It is absolutely essential that we handle 
this with great care and attention to detail.
`;

const result = optimize(prompt);

console.log(result.optimized);
// "Process this data carefully. Handle with care and attention to detail."

console.log(`Saved ${result.reduction.percentage}% tokens`);
// "Saved 45.2% tokens"

console.log(`Cost savings: $${result.savings.estimated}`);
// "Cost savings: $0.000012"
```

---

## 💰 Real-World Cost Savings

### Example 1: AI Code Assistant (10,000 requests/day)

**Before Optimization:**
```
Average prompt: 2,000 tokens
Cost per request: $0.06 (GPT-4)
Daily cost: $600
Monthly cost: $18,000
Yearly cost: $216,000
```

**After Optimization (40% reduction):**
```
Average prompt: 1,200 tokens
Cost per request: $0.036
Daily cost: $360
Monthly cost: $10,800
Yearly cost: $129,600
```

**💰 Savings: $7,200/month = $86,400/year!**

---

### Example 2: Content Generation (50,000 prompts/day)

**Before:**
- 1,500 tokens/prompt
- $0.045 per request
- **$2,250/day = $67,500/month**

**After (35% reduction):**
- 975 tokens/prompt
- $0.029 per request
- **$1,462/day = $43,875/month**

**💰 Savings: $23,625/month = $283,500/year!**

---

### Example 3: Customer Support Bot (100,000 conversations/day)

**Before:**
- 800 tokens/prompt
- $0.024 per request
- **$2,400/day = $72,000/month**

**After (30% reduction):**
- 560 tokens/prompt
- $0.017 per request
- **$1,680/day = $50,400/month**

**💰 Savings: $21,600/month = $259,200/year!**

---

## 📚 Complete Usage Examples

### 1. Basic Optimization

```typescript
import { optimize } from '@yogesh0333/yogiway-prompt';

const verbosePrompt = `
Please note that it is very important to understand that we need to 
carefully process this request. It is absolutely essential that we 
handle this with great care and attention to detail. As you can see, 
this is a very verbose way of saying something simple.
`;

const result = optimize(verbosePrompt);

console.log('Original:', result.original);
console.log('Optimized:', result.optimized);
console.log(`Reduction: ${result.reduction.percentage}%`);
console.log(`Tokens saved: ${result.reduction.tokens}`);
console.log(`Cost savings: $${result.savings.estimated}`);
```

**Output:**
```
Original: Please note that it is very important...
Optimized: Process this request carefully. Handle with care and attention to detail. This is a simple statement.
Reduction: 45.2%
Tokens saved: 23
Cost savings: $0.000012
```

---

### 2. Token Counting

```typescript
import { count, getTokenCount } from '@yogesh0333/yogiway-prompt';

// Simple count
const tokens = count("Hello, how are you today?", "openai");
console.log(`Tokens: ${tokens}`); // 6

// Detailed count with cost
const details = getTokenCount("Your prompt here", "openai");
console.log(details);
// {
//   tokens: 150,
//   characters: 600,
//   words: 100,
//   estimatedCost: 0.000075,
//   breakdown: [
//     { provider: 'openai', tokens: 150, cost: 0.000075 },
//     { provider: 'anthropic', tokens: 171, cost: 0.000513 },
//     ...
//   ]
// }
```

---

### 3. Cost Calculation

```typescript
import { getCost, compareCosts } from '@yogesh0333/yogiway-prompt';

// Calculate cost for single model
const cost = getCost(
  "Write a 1000-word article about AI",
  "openai",
  "gpt-4"
);

console.log(`Input tokens: ${cost.inputTokens}`);
console.log(`Estimated output tokens: ${cost.estimatedOutputTokens}`);
console.log(`Input cost: $${cost.inputCost}`);
console.log(`Output cost: $${cost.outputCost}`);
console.log(`Total cost: $${cost.totalCost}`);
```

**Output:**
```
Input tokens: 12
Estimated output tokens: 250
Input cost: $0.00036
Output cost: $0.015
Total cost: $0.01536
```

---

### 4. Compare Costs Across Providers

```typescript
import { compareCosts } from '@yogesh0333/yogiway-prompt';

const prompt = "Explain quantum computing in simple terms";

const comparison = compareCosts(prompt, [
  { provider: "openai", model: "gpt-4" },
  { provider: "openai", model: "gpt-3.5-turbo" },
  { provider: "anthropic", model: "claude-3-opus" },
  { provider: "anthropic", model: "claude-3-haiku" },
  { provider: "google", model: "gemini-pro" },
]);

comparison.forEach(c => {
  console.log(`${c.provider} (${c.model}): $${c.totalCost}`);
});
```

**Output:**
```
openai (gpt-3.5-turbo): $0.00015
google (gemini-pro): $0.00025
anthropic (claude-3-haiku): $0.0003
openai (gpt-4): $0.00036
anthropic (claude-3-opus): $0.0015
```

---

### 5. Calculate Savings

```typescript
import { optimize, getSavings } from '@yogesh0333/yogiway-prompt';

const original = `
Please note that it is very important to understand that we need to 
carefully review this code and provide detailed feedback. It is 
absolutely essential that we handle this with great care.
`;

const optimized = optimize(original).optimized;

const savings = getSavings(
  original,
  optimized,
  "openai",
  "gpt-4",
  1000 // requests per day
);

console.log(`Per request: $${savings.perRequest}`);
console.log(`Per day (1000 requests): $${savings.perDay}`);
console.log(`Per month: $${savings.perMonth}`);
console.log(`Per year: $${savings.perYear}`);
console.log(`Percentage saved: ${savings.percentage}%`);
```

**Output:**
```
Per request: $0.000012
Per day (1000 requests): $0.012
Per month: $0.36
Per year: $4.38
Percentage saved: 40.5%
```

---

### 6. Context Window Management

```typescript
import { manageContext } from '@yogesh0333/yogiway-prompt';

const largePrompt = "..."; // Very long prompt (20,000 tokens)

const result = manageContext(largePrompt, {
  maxTokens: 8000,
  reserveTokens: 1000, // Reserve for response
  overlap: 200, // Token overlap between chunks
});

console.log(`Total chunks: ${result.chunks.length}`);
result.chunks.forEach((chunk, i) => {
  console.log(`Chunk ${i + 1}: ${chunk.length} chars, ${result.chunkTokens[i]} tokens`);
});
```

---

### 7. Smart Chunking

```typescript
import { chunk } from '@yogesh0333/yogiway-prompt';

const largeText = "..."; // Large text

const chunks = chunk(largeText, {
  maxTokens: 4000,
  overlap: 200,
  strategy: 'smart', // 'paragraph' | 'sentence' | 'word' | 'smart'
  preserveStructure: true,
});

chunks.forEach((chunk, i) => {
  console.log(`Chunk ${i + 1}:`);
  console.log(`  Text: ${chunk.text.substring(0, 100)}...`);
  console.log(`  Tokens: ${chunk.tokens}`);
  console.log(`  Position: ${chunk.start}-${chunk.end}`);
});
```

---

### 8. Format for Providers

```typescript
import { format } from '@yogesh0333/yogiway-prompt';

// Format for OpenAI
const openaiFormat = format({
  provider: "openai",
  systemPrompt: "You are a helpful assistant",
  userPrompt: "Explain quantum computing",
});

console.log(openaiFormat);
// {
//   messages: [
//     { role: 'system', content: 'You are a helpful assistant' },
//     { role: 'user', content: 'Explain quantum computing' }
//   ],
//   model: 'gpt-4'
// }

// Format for Anthropic
const claudeFormat = format({
  provider: "anthropic",
  systemPrompt: "You are a helpful assistant",
  userPrompt: "Explain quantum computing",
});

console.log(claudeFormat);
// {
//   system: 'You are a helpful assistant',
//   messages: [
//     { role: 'user', content: 'Explain quantum computing' }
//   ],
//   model: 'claude-3-opus-20240229',
//   max_tokens: 4096
// }
```

---

### 9. Advanced Optimization Options

```typescript
import { optimize } from '@yogesh0333/yogiway-prompt';

const prompt = "...";

// Standard optimization
const standard = optimize(prompt);

// Aggressive optimization (more reduction)
const aggressive = optimize(prompt, {
  aggressive: true,
  removePunctuation: true,
  targetReduction: 50,
});

// Conservative optimization (preserve more)
const conservative = optimize(prompt, {
  removeRedundancy: true,
  compressWhitespace: true,
  preserveCodeBlocks: true,
  preserveMarkdown: true,
  aggressive: false,
});

console.log(`Standard: ${standard.reduction.percentage}%`);
console.log(`Aggressive: ${aggressive.reduction.percentage}%`);
console.log(`Conservative: ${conservative.reduction.percentage}%`);
```

---

### 10. Using with YOGIWAY Together (Maximum Savings)

```typescript
import { optimize } from '@yogesh0333/yogiway-prompt';
import { encode } from '@yogesh0333/yogiway';

// Step 1: Optimize your prompt text
const prompt = "Please review this code and provide detailed feedback...";
const optimizedPrompt = optimize(prompt).optimized;

// Step 2: Compress your code context
const codeContext = {
  files: [
    { name: "app.js", code: "..." },
    { name: "utils.js", code: "..." },
  ],
};
const compressedCode = encode(codeContext);

// Step 3: Combine for maximum efficiency
const finalPrompt = `${optimizedPrompt}\n\nCode Context:\n${compressedCode}`;

// Result: 50-70% total token savings! 🚀
```

---

## 🎯 Use Cases

### 1. AI Chatbots
```typescript
// Optimize user questions before sending to AI
const userQuestion = "Can you please help me understand how this works?";
const optimized = optimize(userQuestion).optimized;
// "Can you help me understand how this works?"
```

### 2. Code Generation
```typescript
// Optimize instructions for AI coding assistants
const instruction = `
Please note that it is very important to create a function that 
carefully processes the data and handles errors properly.
`;
const optimized = optimize(instruction).optimized;
// "Create a function that processes data and handles errors."
```

### 3. Content Generation
```typescript
// Optimize content prompts
const contentPrompt = `
Write a detailed article about artificial intelligence, making sure 
to cover all the important aspects and provide comprehensive information.
`;
const optimized = optimize(contentPrompt).optimized;
// "Write an article about artificial intelligence covering important aspects."
```

### 4. Data Processing
```typescript
// Optimize data analysis prompts
const analysisPrompt = `
Please analyze this data carefully and provide detailed insights 
about the trends and patterns that you observe.
`;
const optimized = optimize(analysisPrompt).optimized;
// "Analyze this data and provide insights about trends and patterns."
```

---

## 📊 Optimization Strategies

### Standard Mode (Default)
- Removes redundant phrases
- Compresses whitespace
- Preserves code blocks
- Preserves markdown
- **Savings: 30-40%**

### Aggressive Mode
- Removes all redundant words
- Removes unnecessary punctuation
- More aggressive compression
- **Savings: 40-50%**

### Conservative Mode
- Minimal changes
- Preserves more original wording
- **Savings: 20-30%**

---

## 🔧 API Reference

### `optimize(prompt, options?)`

Optimize a prompt to reduce tokens.

**Options:**
- `targetReduction?: number` - Target reduction % (default: 30)
- `removeRedundancy?: boolean` - Remove redundant words (default: true)
- `compressWhitespace?: boolean` - Compress whitespace (default: true)
- `removePunctuation?: boolean` - Remove punctuation (default: false)
- `preserveCodeBlocks?: boolean` - Preserve code blocks (default: true)
- `preserveMarkdown?: boolean` - Preserve markdown (default: true)
- `aggressive?: boolean` - Aggressive optimization (default: false)

**Returns:**
```typescript
{
  original: string;
  optimized: string;
  reduction: {
    tokens: number;
    percentage: number;
    characters: number;
    words: number;
  };
  savings: {
    estimated: number;
    currency: string;
  };
  stats: {
    originalTokens: number;
    optimizedTokens: number;
    originalChars: number;
    optimizedChars: number;
  };
}
```

---

### `count(text, provider?)`

Count tokens in text.

**Parameters:**
- `text: string` - Text to count
- `provider?: LLMProvider` - Provider (default: 'openai')

**Returns:** `number` - Token count

---

### `getTokenCount(text, provider?)`

Get detailed token count with cost estimation.

**Returns:**
```typescript
{
  tokens: number;
  characters: number;
  words: number;
  estimatedCost: number;
  breakdown: Array<{
    provider: LLMProvider;
    tokens: number;
    cost: number;
  }>;
}
```

---

### `getCost(prompt, provider?, model?, estimatedOutputTokens?)`

Calculate cost estimate.

**Returns:**
```typescript
{
  provider: LLMProvider;
  model: string;
  inputTokens: number;
  estimatedOutputTokens: number;
  inputCost: number;
  outputCost: number;
  totalCost: number;
  currency: string;
}
```

---

### `getSavings(original, optimized, provider?, model?, requestsPerDay?)`

Calculate savings from optimization.

**Returns:**
```typescript
{
  perRequest: number;
  perDay: number;
  perMonth: number;
  perYear: number;
  percentage: number;
}
```

---

### `chunk(text, options)`

Chunk large text intelligently.

**Options:**
- `maxTokens: number` - Maximum tokens per chunk
- `overlap?: number` - Token overlap between chunks
- `strategy?: 'paragraph' | 'sentence' | 'word' | 'smart'` - Chunking strategy
- `preserveStructure?: boolean` - Preserve structure (default: true)

---

### `format(options)`

Format prompt for specific provider.

**Options:**
- `provider: LLMProvider` - Provider
- `systemPrompt?: string` - System prompt
- `userPrompt: string` - User prompt
- `assistantPrompt?: string` - Assistant prompt
- `temperature?: number` - Temperature
- `maxTokens?: number` - Max tokens

---

## 🌍 Multi-Platform Support

### Node.js
```bash
npm install @yogesh0333/yogiway-prompt
```

### Browser
```html
<script type="module">
  import { optimize } from 'https://cdn.jsdelivr.net/npm/@yogesh0333/yogiway-prompt@latest/dist/index.js';
</script>
```

### Deno
```typescript
import { optimize } from "https://esm.sh/@yogesh0333/yogiway-prompt";
```

### Bun
```bash
bun add @yogesh0333/yogiway-prompt
```

### Python (Coming Soon)
```bash
pip install yogiway-prompt
```

```python
from yogiway_prompt import optimize, count, get_cost

result = optimize("Your prompt here")
tokens = count("Hello, world!")
cost = get_cost("Your prompt", "openai", "gpt-4")
```

---

## 💡 Best Practices

### 1. Optimize Before Sending
```typescript
// Always optimize prompts before sending to AI
const optimized = optimize(userPrompt).optimized;
await aiAPI.send(optimized);
```

### 2. Use with YOGIWAY for Maximum Savings
```typescript
// Optimize prompt + compress data = 50-70% savings
const optimizedPrompt = optimize(prompt).optimized;
const compressedData = encode(data);
```

### 3. Compare Providers
```typescript
// Always compare costs before choosing provider
const comparison = compareCosts(prompt, [
  { provider: "openai", model: "gpt-4" },
  { provider: "anthropic", model: "claude-3-haiku" },
]);
```

### 4. Monitor Savings
```typescript
// Track your savings over time
const savings = getSavings(original, optimized, "openai", "gpt-4", requestsPerDay);
console.log(`Yearly savings: $${savings.perYear}`);
```

---

## ⚠️ Important Notes & Limitations

### Current Limitations

#### 1. **Token Counting Accuracy**
- **Challenge**: Token counting is approximate (word + char-based estimation)
- **Impact**: Actual token counts may vary by ±5-10% from real API counts
- **Why**: Different models use different tokenizers (tiktoken, etc.)
- **Workaround**: Use actual API responses to calibrate for your use case
- **Future**: Integration with official tokenizers (tiktoken, etc.)

#### 2. **Language Support**
- **Note**: Optimization optimized for English text
- **Impact**: Other languages may see 20-30% reduction (vs 30-50% for English)
- **Why**: Redundancy patterns are language-specific
- **Status**: Works for most languages, with varying effectiveness

#### 3. **Context Preservation**
- **Note**: Aggressive optimization may alter subtle nuances
- **Impact**: Very aggressive mode might change emphasis or tone
- **Recommendation**: Use standard mode for critical prompts, test before production
- **Status**: Standard mode preserves meaning well

#### 4. **Code Block Handling**
- **Note**: Code blocks are fully preserved (not optimized)
- **Impact**: Code comments within blocks remain unchanged
- **Why**: Preserving code integrity is priority
- **Status**: By design - code blocks are always preserved

#### 5. **Pricing Accuracy**
- **Note**: Pricing data is approximate and may need updates
- **Impact**: Cost estimates should be verified with provider documentation
- **Why**: Provider pricing changes frequently
- **Recommendation**: Verify current pricing with provider before making decisions
- **Status**: Pricing included for estimation purposes only

#### 6. **Large Prompt Performance**
- **Note**: Very large prompts (>100K tokens) may process slower
- **Impact**: Optimization time increases with prompt size
- **Recommendation**: Chunk large prompts first, then optimize each chunk
- **Status**: Handles most use cases efficiently

#### 7. **Provider-Specific Features**
- **Note**: Focus on universal optimization across providers
- **Impact**: May not leverage all provider-specific optimizations
- **Status**: Works well for all major providers

#### 8. **Optimization Method**
- **Note**: Uses rule-based optimization (not semantic AI)
- **Impact**: Works well for most common patterns
- **Status**: Effective for typical use cases

---

## 💡 Best Practices

### 1. Test Before Production
Always test optimized prompts to ensure meaning is preserved, especially in aggressive mode.

### 2. Calibrate Token Counts
Use actual API responses to calibrate token counting for your specific use case.

### 3. Verify Pricing
Check current provider pricing before making cost-based decisions.

### 4. Chunk Large Prompts
For very large prompts (>100K tokens), chunk first, then optimize each chunk.

### 5. Use Standard Mode for Critical Prompts
Use standard optimization mode for important prompts to preserve meaning.

---

## 📚 Additional Resources

- **GitHub Repository**: https://github.com/yogesh0333/yogiway
- **Issues & Feature Requests**: https://github.com/yogesh0333/yogiway/issues
- **Related Library**: [@yogesh0333/yogiway](./DIFFERENCE.md) - Compress structured data

---

## 📄 License

MIT License - Free to use in any project

---

## 🤝 Contributing

Contributions welcome! This is a free and open source project.

---

## 📞 Support

- GitHub: https://github.com/yogesh0333/yogiway
- Issues: https://github.com/yogesh0333/yogiway/issues

---

## 🔗 Related Libraries

- **@yogesh0333/yogiway** - Compress structured data (JSON, objects)
- **@yogesh0333/yogiway-prompt** - Optimize text prompts (this library)

**Use both together for maximum savings (50-70%)!** 🚀

---

**Made with ❤️ for developers - 100% Free Forever**