mars-llm/README.md

Node.js client for Azure OpenAI using credentials from Azure Key Vault with streaming support

# Mars LLM - Azure OpenAI Client with Key Vault Integration

A Node.js client that retrieves Azure OpenAI credentials from Azure Key Vault and provides both streaming and non-streaming chat completion capabilities.

## Features

- 🔐 Secure credential retrieval from Azure Key Vault
- 🚀 Support for both streaming and non-streaming chat completions
- 🔧 Multiple Azure authentication methods
- 📝 TypeScript-ready with proper error handling
- 🎯 Simple and intuitive API

## Prerequisites

- Node.js 16+ 
- Azure subscription with:
  - Azure Key Vault instance
  - Azure OpenAI service
  - Proper authentication configured

## Installation

### Option 1: Install from npm (Recommended)

```bash
npm install mars-llm
```

### Option 2: Clone and build locally

1. Clone or download this project
2. Install dependencies:

```bash
npm install
```

3. Set the required environment variable:
```bash
export AZURE_KEY_VAULT_URL=https://your-keyvault.vault.azure.net/
```

4. Configure your Key Vault secrets (see Configuration section below)
5. For local development, run `az login` to authenticate

## Configuration

### Azure Key Vault Secrets

Your Azure Key Vault must contain the following secrets:

| Secret Name | Description | Example Value |
|-------------|-------------|---------------|
| `MARS-API-KEY` | Your Azure OpenAI API key | `abc123...` |
| `MARS-DEPLOYMENT` | Your Azure OpenAI deployment name | `gpt-4o` |
| `MARS-ENDPOINT` | Your Azure OpenAI endpoint URL | `https://your-resource.openai.azure.com/` |
| `MARS-API-VERSION` | API version | `2024-02-15-preview` |

### Environment Variables (Required)

You must set the Key Vault URL via environment variable:

```bash
export AZURE_KEY_VAULT_URL=https://your-keyvault.vault.azure.net/
```

Or on Windows:
```cmd
set AZURE_KEY_VAULT_URL=https://your-keyvault.vault.azure.net/
```

This environment variable is required for the client to work.

## Authentication

This client uses `DefaultAzureCredential` which automatically tries authentication methods in order:

### 1. Azure CLI (Recommended for local development)
Run `az login` before using the client - no additional configuration needed.

### 2. Managed Identity (For Azure-hosted applications)
Automatically works when running on Azure services - no configuration needed.

### 3. Environment Variables (For service principal if needed)
Set `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID` if required.

### 4. Azure PowerShell
Uses Azure PowerShell context if available.

### 5. Interactive Browser
Falls back to interactive authentication if other methods fail.

## Usage

### Basic Usage

```javascript
// If installed from npm
const AzureOpenAIClient = require('mars-llm');

// If using locally
// const AzureOpenAIClient = require('./azure-openai-client');

async function example() {
    const client = new AzureOpenAIClient();
    
    // Simple chat
    const response = await client.chat('Hello, how are you?');
    console.log(response.content);
}
```

### Advanced Usage

```javascript
const AzureOpenAIClient = require('./azure-openai-client');

async function advancedExample() {
    const client = new AzureOpenAIClient();
    
    // Chat with custom options
    const response = await client.chat('Explain quantum computing', {
        maxTokens: 200,
        temperature: 0.7,
        model: 'gpt-4o'
    });
    
    console.log('Response:', response.content);
    console.log('Usage:', response.usage);
}
```

### Streaming Chat

```javascript
const AzureOpenAIClient = require('./azure-openai-client');

async function streamingExample() {
    const client = new AzureOpenAIClient();
    
    // Streaming chat with callback
    await client.chatStream(
        'Tell me a story about AI',
        (chunk, fullContent) => {
            process.stdout.write(chunk); // Print each chunk as it arrives
        },
        {
            maxTokens: 500,
            temperature: 0.8
        }
    );
}
```

### Using Prompt and Text (Python-style)

```javascript
const AzureOpenAIClient = require('./azure-openai-client');

async function promptTextExample() {
    const client = new AzureOpenAIClient();
    
    const prompt = "Summarize the following text";
    const text = "Your long text content here...";
    
    // Non-streaming
    const response = await client.chatCompletion(prompt, text, {
        maxTokens: 150,
        temperature: 0.5
    });
    
    // Streaming
    await client.chatCompletionStream(
        prompt, 
        text, 
        (chunk) => process.stdout.write(chunk),
        { maxTokens: 150 }
    );
}
```

## API Reference

### Constructor

```javascript
const client = new AzureOpenAIClient(keyVaultUrl);
```

- `keyVaultUrl` (optional): Key Vault URL. If not provided, uses `AZURE_KEY_VAULT_URL` environment variable. One of these must be set.

### Methods

#### `chat(message, options)`
Simple chat completion without streaming.

**Parameters:**
- `message` (string): The message to send
- `options` (object): Optional configuration
  - `model` (string): Model name (default: 'gpt-4o')
  - `maxTokens` (number): Maximum tokens (default: 50)
  - `temperature` (number): Temperature (default: 0.7)

**Returns:** Promise with response object containing `content`, `usage`, `model`, `finishReason`

#### `chatStream(message, onChunk, options)`
Streaming chat completion.

**Parameters:**
- `message` (string): The message to send
- `onChunk` (function): Callback for each chunk `(chunk, fullContent) => {}`
- `options` (object): Optional configuration (same as chat)

**Returns:** Promise with response object containing `content`, `model`

#### `chatCompletion(prompt, text, options)`
Chat completion with separate prompt and text (Python-style).

#### `chatCompletionStream(prompt, text, onChunk, options)`
Streaming chat completion with separate prompt and text.

## Running the Examples

### Basic example:
```bash
npm start
```

### Comprehensive demo:
```bash
node example.js
```

## Troubleshooting

### Common Issues

1. **Authentication Error**: Run `az login` for local development
2. **Key Vault Access Denied**: Verify your Azure account has Key Vault access permissions
3. **Secret Not Found**: Check that secrets exist in Key Vault with correct names
4. **Invalid Endpoint**: Ensure your Azure OpenAI endpoint URL is correct

### Debug Tips

1. Test Azure CLI authentication: `az login && az account show`
2. Test Key Vault access: `az keyvault secret show --vault-name your-vault --name MARS-API-KEY`
3. Check Key Vault URL: Verify the correct Key Vault URL is used

## Security Best Practices

- Use managed identities for production deployments
- Use Azure CLI for local development
- Rotate API keys regularly
- Limit Key Vault access permissions
- Monitor Key Vault access logs

## License

MIT License