@arizeai/phoenix-evals/docs/llm-evaluators.mdx

A library for running evaluations for AI use cases

---
title: "LLM Evaluators"
description: "Use LLM-backed evaluators in @arizeai/phoenix-evals"
---

The `llm` entrypoint provides reusable evaluator factories that call an AI SDK model and return structured evaluation results.

<section className="hidden" data-agent-context="relevant-source-files" aria-label="Relevant source files">
  <h2>Relevant Source Files</h2>
  <ul>
    <li><code>src/llm/index.ts</code></li>
  </ul>
</section>

## Example

```ts
import { openai } from "@ai-sdk/openai";
import { createFaithfulnessEvaluator } from "@arizeai/phoenix-evals";

const faithfulness = createFaithfulnessEvaluator({
  model: openai("gpt-4o-mini"),
});

const result = await faithfulness.evaluate({
  input: "What is the capital of France?",
  context: "France is a country in Europe. Paris is its capital city.",
  output: "The capital of France is Paris.",
});
```

## Built-In Evaluator Factories

- `createConcisenessEvaluator`
- `createCorrectnessEvaluator`
- `createDocumentRelevanceEvaluator`
- `createFaithfulnessEvaluator`
- `createRefusalEvaluator`
- `createClassificationEvaluator`
- `createToolSelectionEvaluator`
- `createToolInvocationEvaluator`
- `createToolResponseHandlingEvaluator`

```ts
import { openai } from "@ai-sdk/openai";
import {
  createCorrectnessEvaluator,
  createRefusalEvaluator,
} from "@arizeai/phoenix-evals";

const model = openai("gpt-4o-mini");

const correctness = createCorrectnessEvaluator({ model });
const refusal = createRefusalEvaluator({ model });
```

<section className="hidden" data-agent-context="source-map" aria-label="Source map">
  <h2>Source Map</h2>
  <ul>
    <li><code>src/llm/createClassificationEvaluator.ts</code></li>
    <li><code>src/llm/ClassificationEvaluator.ts</code></li>
    <li><code>src/llm/LLMEvaluator.ts</code></li>
    <li><code>src/llm/createFaithfulnessEvaluator.ts</code></li>
    <li><code>src/types/evals.ts</code></li>
  </ul>
</section>