@kluai/gateway/README.md

OpenAI compatible gateway.

# Klu Gateway

Klu gateway is an OpenAI compatible gateway package. With it you can self host your own OpenAI API that supports the following models:
* OpenAI
* Azure OpenAI
* Perplexity
* Anthropic
* Replicate
* Google (Gemini models)

# Installation

To install the gateway simply run the following command:

```bash
npm i @kluai/gateway
```

We support node versions 14+. If you are using an older version of node, please upgrade.


# Running on edge

In order to run on edge (Cloudflare for example) you need to copy the `cloudflare` directory in this repository. 

Once you've copied over the code into your own project, create a cloud worker and run the following command:

```bash
npm i
wrangler dev
```

Now you will be able to run this locally. Feel free to tweak the implementation as much as you need and once you are ready to deploy run:
    
```bash
wrangler deploy
```

# Running on cloud

If you'd like to run this into a cloud you can import the OpenAI gateway function like so. 

```typescript
import { OpenAI } from "@kluai/gateway";

export default {
  async fetch(request: Request, context: ExecutionContext) {
    return OpenAI()(request, context);
  },
};
```

You could run this in a lambda or any other server and you would be ready to handle any number of requests. 

For feedback, suggestions or problems feel free to email stefan@klu.ai