@osohq/langchain/README.md

Oso observability integration for LangChain agents

# @osohq/langchain

Oso observability integration for LangChain agents.

Callback handler that automatically captures and sends all LangChain agent events to Oso's observability platform for monitoring, debugging, and security analysis.

## Installation

```bash
npm install @osohq/langchain
```

Or with yarn:

```bash
yarn add @osohq/langchain
```

## Quick Start

```typescript
import { OsoObservabilityCallback } from "@osohq/langchain";
import { createOpenAIToolsAgent, AgentExecutor } from "langchain/agents";
import { ChatOpenAI } from "@langchain/openai";

// Create the callback (reads OSO_AUTH_TOKEN from environment)
const callback = new OsoObservabilityCallback({
  agentId: "my-support-agent",
});

// Add to your agent
const llm = new ChatOpenAI({ model: "gpt-4" });
const agent = await createOpenAIToolsAgent({ llm, tools, prompt });
const agentExecutor = new AgentExecutor({
  agent,
  tools,
  callbacks: [callback],
});

// Use your agent - all events are automatically captured
const result = await agentExecutor.invoke({
  input: "Hello, how can I help?",
});

// Clean up
await callback.close();
```

## Configuration

### Environment Variables

Set these in your environment or `.env` file:

```bash
# Required: Your Oso authentication token
OSO_AUTH_TOKEN=your-token-here

# Optional: Custom Oso endpoint (defaults to https://cloud.osohq.com/api/events)
OSO_ENDPOINT=https://cloud.osohq.com/api/events

# Optional: Enable/disable observability (defaults to true)
OSO_OBSERVABILITY_ENABLED=true
```

### Constructor Parameters

```typescript
new OsoObservabilityCallback({
  endpoint: "https://cloud.osohq.com/api/events", // Oso endpoint URL
  authToken: "your-token", // Oso auth token
  enabled: true, // Enable/disable sending events
  sessionId: "unique-session-id", // Group related conversations
  metadata: { userId: "123", env: "prod" }, // Custom metadata for all events
  agentId: "my-agent", // Agent identifier
});
```

All parameters are optional and fall back to environment variables or defaults.

## What Gets Captured

The callback automatically captures all LangChain events:

### LLM Events

- Model name and configuration
- Prompts sent to the LLM
- Generated responses
- Token usage (prompt, completion, total)
- Errors and failures

### Tool Events

- Tool name and description
- Input parameters
- Output/results
- Execution duration (milliseconds)
- Errors and stack traces

### Agent Events

- Agent reasoning and thought process
- Tool selection decisions
- Tool input parameters
- Final outputs
- Complete execution flow

### Chain Events

- Chain type and name
- Input parameters
- Output values
- Nested chain execution

### Execution Summary

At the end of each agent execution, a summary event is sent with:

- Total execution duration
- Number of LLM calls, tool calls, and agent steps
- Total token usage
- Error count
- Complete execution trace

## Event Structure

Every event sent to Oso has this structure:

```json
{
  "event_type": "tool.completed",
  "execution_id": "unique-execution-id",
  "session_id": "conversation-session-id",
  "timestamp": "2024-02-15T10:30:45.123Z",
  "agent_id": "my-agent",
  "data": {
    /* event-specific data */
  },
  "metadata": {
    /* your custom metadata */
  }
}
```

### Event Types

- `llm.started` / `llm.completed` / `llm.error`
- `tool.started` / `tool.completed` / `tool.error`
- `agent.action` / `agent.finished`
- `chain.started` / `chain.completed` / `chain.error`
- `execution.summary` - Final summary with all accumulated data

## Error Handling

The callback is designed to fail gracefully:

- Network errors or timeouts won't crash your agent
- Failed event sends are logged but don't interrupt execution
- Comprehensive error logging for debugging

## Logging

The callback uses Node.js console logging. You can configure it in your application or use a custom logger:

```typescript
// Set NODE_DEBUG environment variable to see debug logs
process.env.NODE_DEBUG = "langchain-oso";
```

## Examples

### Basic Agent with Tools

```typescript
import { OsoObservabilityCallback } from "@osohq/langchain";
import { createOpenAIToolsAgent, AgentExecutor } from "langchain/agents";
import { ChatOpenAI } from "@langchain/openai";
import { tool } from "@langchain/core/tools";

const searchOrders = tool(
  async ({ customerId }: { customerId: string }) => {
    return `Orders for ${customerId}: ORD001, ORD002`;
  },
  {
    name: "search_orders",
    description: "Search for customer orders.",
    schema: z.object({
      customerId: z.string().describe("The customer ID"),
    }),
  }
);

async function main() {
  const callback = new OsoObservabilityCallback({ agentId: "support-agent" });

  const llm = new ChatOpenAI({ model: "gpt-4o-mini" });
  const tools = [searchOrders];

  const agent = await createOpenAIToolsAgent({ llm, tools, prompt });
  const agentExecutor = new AgentExecutor({
    agent,
    tools,
    callbacks: [callback],
  });

  const result = await agentExecutor.invoke({
    input: "Find orders for customer CUST001",
  });

  console.log(result.output);

  await callback.close();
}
```

### With Custom Metadata

```typescript
const callback = new OsoObservabilityCallback({
  agentId: "support-agent",
  sessionId: "user-session-123",
  metadata: {
    userId: "user-456",
    environment: "production",
    version: "1.2.3",
  },
});
```

### Multiple Agents in Same Session

```typescript
import { randomUUID } from "crypto";

const sessionId = randomUUID();

// Agent 1
const callback1 = new OsoObservabilityCallback({
  agentId: "agent-1",
  sessionId,
});
const result1 = await agent1Executor.invoke({ input: "..." });
await callback1.close();

// Agent 2 - same session
const callback2 = new OsoObservabilityCallback({
  agentId: "agent-2",
  sessionId,
});
const result2 = await agent2Executor.invoke({ input: "..." });
await callback2.close();
```

## TypeScript Support

This package is written in TypeScript and includes full type definitions. All types are exported for your convenience.

## Requirements

- Node.js 16.0.0 or higher
- @langchain/core >= 0.1.0

## License

Apache License 2.0

## Support

- Documentation: https://www.osohq.com/docs
- Website: https://www.osohq.com