@friendliai/ai-provider/README.md

Learn how to use the FriendliAI provider for the Vercel AI SDK.

# @friendliai/ai-provider

Learn how to use the FriendliAI provider for the Vercel AI SDK.

## Installation

You can install the package via npm:

```bash
npm i @friendliai/ai-provider
```

## Credentials

The tokens required for model usage can be obtained from the [Friendli suite](https://suite.friendli.ai/).

To use the provider, you need to set the `FRIENDLI_TOKEN` environment variable with your personal access token.

```bash
export FRIENDLI_TOKEN="YOUR_FRIENDLI_TOKEN"
```

Check the [FriendliAI documentation](https://friendli.ai/docs/guides/personal_access_tokens) for more information.

## Provider Instance

```ts
import { friendli } from '@friendliai/ai-provider'
```

## Language Models

You can create [FriendliAI models](https://friendli.ai/docs/guides/serverless_endpoints/text_generation#model-supports) using a provider instance.
The first argument is the model id, e.g. `meta-llama-3.1-8b-instruct`.

```ts
const model = friendli('meta-llama-3.1-8b-instruct')
```

### Example: Generating text

You can use FriendliAI language models to generate text with the `generateText` function:

```ts
import { friendli } from "@friendliai/ai-provider";
import { generateText } from 'ai'

const { text } = await generateText({
  model: friendli('meta-llama-3.1-8b-instruct')
  prompt: 'What is the meaning of life?',
})
```

### Example: Using Enforcing Patterns (Regex)

Specify a specific pattern (e.g., CSV), character sets, or specific language characters (e.g., Korean Hangul characters) for your LLM's output.

```ts
import { friendli } from '@friendliai/ai-provider'
import { generateText } from 'ai'

const { text } = await generateText({
  model: friendli('meta-llama-3.1-8b-instruct', {
    regex: '[\n ,.?!0-9\uac00-\ud7af]*',
  }),
  maxTokens: 40,
  prompt: 'who is the first king of the Joseon Dynasty?',
})

console.log(text)
```

### Example: Using built-in tools (Beta)

If you use `@friendliai/ai-provider`, you can use the [built-in tools](https://friendli.ai/docs/guides/serverless_endpoints/tools/built_in_tools) via the `tools` option.

Built-in tools allow models to use tools to generate better answers. For example, a `web:search` tool can provide up-to-date answers to current questions.

```ts highlight="1,8,9,10,11,12,13,14,15"
import { friendli } from '@friendliai/ai-provider'
import { convertToCoreMessages, streamText } from 'ai'

export async function POST(req: Request) {
  const { messages } = await req.json()

  const result = await streamText({
    model: friendli('meta-llama-3.1-8b-instruct', {
      tools: [
        { type: 'web:search' },
        { type: 'math:calculator' },
        { type: 'code:python-interpreter' }, // and more tools..!!
      ],
    }),
    messages: convertToCoreMessages(messages),
  })

  return result.toDataStreamResponse()
}
```

FriendliAI language models can also be used in the `streamText`, `generateObject`, `streamObject`, and `streamUI` functions.
(see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

## OpenAI Compatibility

We can also use `@ai-sdk/openai` with OpenAI compatibility.

```ts
import { createOpenAI } from '@ai-sdk/openai'

const friendli = createOpenAI({
  baseURL: 'https://api.friendli.ai/serverless/v1',
  apiKey: process.env.FRIENDLI_TOKEN,
})
```