@endlessriver/optimaiz/README.md

Client SDK for interacting with the Optimaiz logging & trace system.

# @endlessriver/optimaiz

> Unified tracing, feedback, and cost analytics for LLM-based apps.  
> Drop-in SDK to track prompts, responses, cost, errors, and feedback across OpenAI, LangChain, Sarvam, Gemini, and more.
> Visit https://optimaiz.io to observe, analyze, optimize and comply

---

## 📦 Installation

```bash
npm install @endlessriver/optimaiz
```

---

## ✨ Key Features

<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
    <div className="flex items-start gap-3">
        <Brain className="h-5 w-5 text-blue-500 mt-1" />
        <div>
            <h5 className="font-medium">Smart Model Selection</h5>
            <p className="text-sm text-muted-foreground">Automatically selects the best model based on your needs</p>
        </div>
    </div>
    <div className="flex items-start gap-3">
        <Database className="h-5 w-5 text-purple-500 mt-1" />
        <div>
            <h5 className="font-medium">Intelligent Caching</h5>
            <p className="text-sm text-muted-foreground">Optimizes response times with smart caching</p>
        </div>
    </div>
    <div className="flex items-start gap-3">
        <Zap className="h-5 w-5 text-amber-500 mt-1" />
        <div>
            <h5 className="font-medium">Performance Optimization</h5>
            <p className="text-sm text-muted-foreground">Continuously optimizes for best results</p>
        </div>
    </div>
</div>

---

## 🛠️ Initialization

```ts
import { OptimaizClient } from "@endlessriver/optimaiz";

const optimaiz = new OptimaizClient({
  token: process.env.OPTIMAIZ_API_KEY!,
});
```

---

## 🚀 Basic Usage: `call`

The `call` function provides a unified way to interact with the Optimaiz API, handling all the complexity of model selection, caching, and optimization behind the scenes.

```ts
const { response, status } = await optimaiz.call({
  promptTemplate: [
    {
      type: "text",
      role: "user",
      value: "Summarize this: {text}"
    }
  ],
  promptVariables: {
    text: "Your text to summarize"
  },
  modelParams: {
    temperature: 0.7
  },
  threadId: "summary_thread",
  userId: "user_123",
  agentId: "Tool:LLM"
});
```

The function returns:
- `data`: The model's response with traceId
- `errot`: Relevant error, null if success

Key benefits of using `call`:
- ✅ Automatic model selection based on your needs
- ✅ Built-in error handling and logging
- ✅ Intelligent caching for faster responses
- ✅ Automatic trace generation and management
- ✅ Seamless integration with the Optimaiz platform

---

## 🚀 Basic Usage: `wrapLLMCall`

```ts
const { response, traceId } = await optimaiz.wrapLLMCall({
  provider: "openai",
  model: "gpt-4o",
  promptTemplate: [{ role: "user", type: "text", value: "Summarize this" }],
  promptVariables: {},
  call: () => openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Summarize this" }],
  }),
});
```

This handles:
- ✅ Start trace
- ✅ Append raw response
- ✅ Finalize trace with latency
- ✅ Log any errors

---

## ⚙️ Advanced Usage with IDs

```ts
const { response } = await optimaiz.wrapLLMCall({
  traceId: "trace_123",
  agentId: "agent:translator",
  userId: "user_456",
  flowId: "translate_email",
  threadId: "email_translation",
  sessionId: "session_2025_06_01_user_456",
  provider: "openai",
  model: "gpt-4o",
  promptTemplate: [{ role: "user", type: "text", value: "Translate to French: {text}" }],
  promptVariables: { text: "Hello, how are you?" },
  call: () => openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Translate to French: Hello, how are you?" }],
  }),
});
```

---

## 🧩 Manual Usage (Start, Append, Finalize)

Sometimes you need lower-level control (e.g., multiple responses, partial logs).

### 🔹 Start a trace manually

```ts
await optimaiz.startTrace({
  traceId: "trace_xyz",
  agentId: "imageAnalyzer",
  userId: "user_999",
  flowId: "caption_image",
  promptTemplate: [
    { role: "user", type: "image", value: "https://cdn.site/image.png" },
    { role: "user", type: "text", value: "What's in this image?" }
  ],
  promptVariables: {},
  provider: "openai",
  model: "gpt-4o"
});
```

### 🔹 Append a model response

```ts
await optimaiz.appendResponse({
  traceId: "trace_xyz",
  rawResponse: response,
  provider: "openai",
  model: "gpt-4o"
});
```

### 🔹 Finalize the trace

```ts
await optimaiz.finalizeTrace("trace_xyz");
```

### ❌ Log an Error to a Trace

```ts
await optimaiz.logError("trace_abc123", {
  message: "Timeout waiting for OpenAI response",
  code: "TIMEOUT_ERROR",
  details: {
    timeout: "30s",
    model: "gpt-4o",
    retryAttempt: 1,
  },
});
```


### 🔧 Example Usage

```ts
try {
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Summarize this" }],
  });

  await optimaiz.appendResponse({
    traceId,
    rawResponse: response,
    provider: "openai",
    model: "gpt-4o",
  });

  await optimaiz.finalizeTrace(traceId);
} catch (err: any) {
  await optimaiz.logError(traceId, {
    message: err.message,
    code: err.code || "UNCAUGHT_EXCEPTION",
    details: err.stack,
  });
  throw err;
}
```

---

## 🧪 Tool Prompt Helper

```ts
const { promptTemplate, promptVariables } = optimaiz.generatePromptFromTools({
  toolInfo: [weatherTool],
  toolInput: { name: "get_weather", arguments: { location: "Delhi" } },
});
```

---

## 🔄 Compose Prompts from Template

```ts
const { prompts, promptTemplate, promptVariables } = optimaiz.composePrompts(
  [
    { role: "system", content: "You are a poet." },
    { role: "user", content: "Write a haiku about {topic}" },
  ],
  { topic: "the ocean" }
);
```

---

## 📂 Integration Examples

### ✅ OpenAI SDK

```ts
const userPrompt = "Summarize this blog about AI agents";

await optimaiz.wrapLLMCall({
  provider: "openai",
  model: "gpt-4o",
  agentId: "summarizer",
  userId: "user_123",
  promptTemplate: [{ role: "user", type: "text", value: userPrompt }],
  promptVariables: {},
  call: () => openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: userPrompt }],
  }),
});
```

### ✅ LangChain

```ts
const prompt = PromptTemplate.fromTemplate("Tell me a joke about {topic}");
const formatted = await prompt.format({ topic: "elephants" });

await optimaiz.wrapLLMCall({
  provider: "openai",
  model: "gpt-4o",
  agentId: "joke-bot",
  userId: "user_321",
  flowId: "joke-generation",
  promptTemplate: [{ role: "user", type: "text", value: "Tell me a joke about {topic}" }],
  promptVariables: { topic: "elephants" },
  call: () => langchainModel.invoke(formatted),
});
```

### ✅ Sarvam AI (Audio)

```ts
await optimaiz.wrapLLMCall({
  provider: "sarvam",
  model: "shivang",
  agentId: "transcriber",
  userId: "user_999",
  flowId: "transcribe",
  promptTemplate: [{ role: "user", type: "audio", value: "https://cdn.site/audio.wav" }],
  promptVariables: {},
  call: () => sarvam.speechToText({ url: "https://cdn.site/audio.wav" }),
});
```

### ✅ Gemini (Google Vertex AI)

```ts
await optimaiz.wrapLLMCall({
  provider: "google",
  model: "gemini-pro",
  promptTemplate: [{ role: "user", type: "text", value: "Write a haiku about the ocean." }],
  promptVariables: {},
  call: () => gemini.generateContent({
    contents: [{ role: "user", parts: [{ text: "Write a haiku about the ocean." }] }],
  }),
});
```

---

## 📊 Field Scope & Best Practices

| Field        | Scope         | Used for...                                     | Example Value               |
|--------------|---------------|--------------------------------------------------|-----------------------------|
| `traceId`    | Per action    | Track 1 LLM/tool call                            | `trace_a9f3`                |
| `flowId`     | Per task      | Multi-step task grouping                         | `flow_generate_poem`        |
| `agentId`    | Per trace     | Identify AI agent handling task                  | `calendarAgent`             |
| `threadId`   | Per topic     | Group related flows by theme/intent              | `thread_booking`            |
| `sessionId`  | Per session   | Temporal or login-bound grouping                 | `session_2025_06_01_user1`  |
| `userId`     | Global        | Usage, feedback, and cost attribution            | `user_321`                  |

### ✅ Use These for Full Insight:

- `agentId`: Enables **per-agent cost & prompt optimization**
- `userId`: Enables **user behavior analytics & pricing insights**
- `flowId`: Helps trace **multi-step user tasks**
- `traceId`: Use like a **span** for 1 prompt/response
- `threadId`, `sessionId`: Group related interactions over time or topics

---

## ✨ Optimaiz Features

- ✅ Works with OpenAI, Gemini, Sarvam, Mistral, LangChain, Anthropic
- 🧠 RAG and function/tool-call support
- 🔍 Token usage + latency tracking
- 📉 Cost and model metadata logging
- 🧪 Error + feedback logging
- 🔄 Templated prompt builder + tool integration support
- 🧩 Full control via start/append/finalize or simple `wrapLLMCall`

---

## 🔗 Get Started

1. Install: `npm install @endlessriver/optimaiz`  
2. Add your API key: `process.env.OPTIMAIZ_API_KEY`  
3. Use `wrapLLMCall()` for LLM/tool calls  
4. Pass `userId`, `agentId`, and `flowId` for best observability  
5. Analyze and improve prompt cost, user flow, and LLM performance

---

Need hosted dashboards, insights, or tuning support?  
Visit 👉 [https://optimaiz.io](https://optimaiz.io)