astra-assistants/README.md

Astra Assistants API - drop in replacement for OpenAI Assistants, powered by AstraDB

# astra-assistants

Astra Assistants is a **drop-in replacement** for OpenAI's assistant API. It supports the full Assistants v2 API, including **streaming** and **vector stores**. This JavaScript library extends the OpenAI SDK with additional capabilities and provides an easy way to integrate credentials for third-party LLMs.

# How to use    

Install astra-assistants:

```
npm install astra-assistants
```


import and patch your client:

```
import OpenAI from 'openai';
import { patch } from 'astra-assistants';

// Patch the client and assign it to a variable
const client = patch(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));

// Use the patched client
const response = await client.chat.completions.create({
  model: 'gpt-3.5-turbo', // or 'claude-3-5-sonnet', etc. etc.
  messages: [{ role: 'user', content: 'Hello!' }],
});

console.log(response);

```

# Server

The astra-assistants server code is now open source (Apache2)! 

Check it out here https://github.com/datastax/astra-assistants-api

# Authentication

Provide api keys for third party LLMs via environment variables. We support LLM completions through [litellm](https://github.com/BerriAI/litellm) and support litellm environmental variables and models.

Rename the [.env.bkp](./.env.bkp) to `.env` and fill in the appropriate values for the LLMs you want to use.

```
#!/bin/bash

# AstraDB -> https://astra.datastax.com/ --> tokens --> administrator user --> generate
export ASTRA_DB_APPLICATION_TOKEN=""

# OpenAI Models - https://platform.openai.com/api-keys --> create new secret key
export OPENAI_API_KEY=""

# Groq Models - https://console.groq.com/keys
export GROQ_API_KEY=""

# Anthropic claude models - https://console.anthropic.com/settings/keys
export ANTHROPIC_API_KEY=""

# Gemini models -> https://makersuite.google.com/app/apikey
export GEMINI_API_KEY=""

# Perplexity models -> https://www.perplexity.ai/settings/api  --> generate
export PERPLEXITYAI_API_KEY=""

# Cohere models -> https://dashboard.cohere.com/api-keys
export COHERE_API_KEY=""

# Bedrock models -> https://docs.aws.amazon.com/bedrock/latest/userguide/setting-up.html
export AWS_REGION_NAME=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""

# vertexai models https://console.cloud.google.com/vertex-ai
export GOOGLE_JSON_PATH=""
export GOOGLE_PROJECT_ID=""

# AI21 models
export AI21_API_KEY=""

# Aleph Alpha models
export ALEPHALPHA_API_KEY=""

# Anyscale models
export ANYSCALE_API_KEY=""

# Azure models
export AZURE_API_KEY=""
export AZURE_API_BASE=""
export AZURE_API_VERSION=""
export AZURE_AD_TOKEN=""
export AZURE_API_TYPE=""

# Baseten models
export BASETEN_API_KEY=""

# Cloudflare Workers models
export CLOUDFLARE_API_KEY=""
export CLOUDFLARE_ACCOUNT_ID=""

# DeepInfra models
export DEEPINFRA_API_KEY=""

# DeepSeek models
export DEEPSEEK_API_KEY=""

# Fireworks AI models
export FIREWORKS_AI_API_KEY=""

# Hugging Face models
export HUGGINGFACE_API_KEY=""
export HUGGINGFACE_API_BASE=""

# Mistral models
export MISTRAL_API_KEY=""

# NLP Cloud models
export NLP_CLOUD_API_KEY=""

# OpenAI models
export OPENAI_API_KEY=""
export OPENAI_ORGANIZATION=""
export OPENAI_API_BASE=""

# OpenRouter models
export OPENROUTER_API_KEY=""
export OR_SITE_URL=""
export OR_APP_NAME=""

# PaLM models
export PALM_API_KEY=""

# Replicate models
export REPLICATE_API_KEY=""

# TogetherAI models
export TOGETHERAI_API_KEY=""

# Vertex AI models
export VERTEXAI_PROJECT=""
export VERTEXAI_LOCATION=""
export GOOGLE_APPLICATION_CREDENTIALS=""

# Voyage models
export VOYAGE_API_KEY=""

# WatsonX models
export WATSONX_URL=""
export WATSONX_APIKEY=""
export WATSONX_TOKEN=""
export WATSONX_PROJECT_ID=""
export WATSONX_DEPLOYMENT_SPACE_ID=""

# XInference models
export XINFERENCE_API_BASE=""
export XINFERENCE_API_KEY=""
```