@arizeai/phoenix-evals/docs/create-evaluator.mdx

A library for running evaluations for AI use cases

---
title: "Create Evaluator"
description: "Build custom evaluators with @arizeai/phoenix-evals"
---

Use `createEvaluator` when your evaluation logic is plain TypeScript and you want a reusable evaluator object with consistent metadata and telemetry.

<section className="hidden" data-agent-context="relevant-source-files" aria-label="Relevant source files">
  <h2>Relevant Source Files</h2>
  <ul>
    <li><code>src/helpers/createEvaluator.ts</code></li>
  </ul>
</section>

## Example

```ts
import { createEvaluator } from "@arizeai/phoenix-evals";

const exactMatch = createEvaluator(
  ({ output, expected }) => ({
    score: output === expected ? 1 : 0,
    label: output === expected ? "match" : "mismatch",
  }),
  {
    name: "exact-match",
    kind: "CODE",
  }
);

const result = await exactMatch.evaluate({
  output: "Paris",
  expected: "Paris",
});
```

## What You Get

- an evaluator name
- evaluator kind such as `CODE` or `LLM`
- optimization direction metadata
- optional OpenTelemetry spans around execution

## When To Use Code Evaluators

Code evaluators are the right fit when the scoring logic should stay deterministic, cheap, and fully under your control:

- regex or exact-match checks
- JSON structure validation
- latency and cost thresholds
- post-processing checks on existing model output

```ts
import { createEvaluator } from "@arizeai/phoenix-evals";

const lengthCheck = createEvaluator(
  ({ output }) => {
    const score = typeof output === "string" && output.length < 280 ? 1 : 0;
    return {
      score,
      label: score ? "fits-limit" : "too-long",
    };
  },
  {
    name: "response-length",
    kind: "CODE",
  }
);
```

## Related Helpers

- `asEvaluatorFn`
- `toEvaluationResult`

<section className="hidden" data-agent-context="source-map" aria-label="Source map">
  <h2>Source Map</h2>
  <ul>
    <li><code>src/helpers/createEvaluator.ts</code></li>
    <li><code>src/helpers/asEvaluatorFn.ts</code></li>
    <li><code>src/helpers/toEvaluationResult.ts</code></li>
    <li><code>src/core/FunctionEvaluator.ts</code></li>
    <li><code>src/core/EvaluatorBase.ts</code></li>
    <li><code>src/types/evals.ts</code></li>
  </ul>
</section>