UNPKG

eve

Version:

Filesystem-first framework for durable backend AI agents that run anywhere.

49 lines (33 loc) 3.2 kB
--- title: "Targets" description: "Point evals at a local dev server or a deployment with the same eval files." --- An eval target is always an HTTP URL. `eve eval` starts a local dev server, while `eve eval --url <url>` runs against an existing server or deployment. The same eval files work for both, which is what makes evals usable as end-to-end tests in CI. The runner polls `/eve/v1/health`, verifies `/eve/v1/info`, and exposes the live target as `t.target` inside the `test` function. ## Target helpers ```ts title="evals/heartbeat.eval.ts" import { defineEval } from "eve/evals"; export default defineEval({ async test(t) { const { sessionIds } = await t.target.dispatchSchedule("heartbeat"); await t.target.attachSession(sessionIds[0]!); t.completed(); t.calledTool("send_report"); }, }); ``` - `t.target.fetch(path, init)` performs an authenticated fetch against the target, useful for channel and webhook ingress. See [Authentication](#authentication) for how the runner authenticates. - `t.target.dispatchSchedule(id)` triggers a [schedule](../schedules) through the dev-only schedule route and returns the session ids it created. It works only against a target with dev routes enabled (the local `eve eval` dev server, or a deployment running in development mode), and throws otherwise. - `t.target.attachSession(sessionId, { startIndex? })` consumes one turn from a session created outside the eval, by a channel or a schedule, so its events feed the run-level assertions. `startIndex` skips events before that position, so a session already partway through its stream resumes from where you left off rather than replaying from the start. Sessions attached this way are full `EveEvalSession`s: you can keep driving them with `send` and read their event streams. The run-level assertions on `t` (`t.completed()`, `t.calledTool(...)`) read the whole run, including attached sessions. ## Authentication Local targets send no auth: `eve eval` owns the dev server it boots. For a remote `--url`, eve gets the expected Vercel owner and project from `VERCEL_ORG_ID` and `VERCEL_PROJECT_ID` when both are set. Otherwise it reads `.vercel/project.json`. Eve then asks Vercel to resolve the exact HTTPS origin and sends ambient credentials only when the project IDs match. An arbitrary URL remains anonymous. After verification, eve sends the available Vercel credentials: - The resolved OIDC token as both the bearer and Vercel trusted-IDP header. - `VERCEL_AUTOMATION_BYPASS_SECRET`, when set, as the Protection Bypass for Automation header. `EVE_EVAL_AUTH_TOKEN` is an explicit bearer override for targets whose auth is not Vercel OIDC. Credential-bearing clients do not follow redirects, so those headers cannot be forwarded to another origin. `t.target.fetch(path, init)` carries these same credentials, so channel and webhook ingress you exercise through it authenticates the same way the session protocol does. ## What to read next - [Running evals](./running): `--url` and the rest of the CLI in practice - [Schedules](../schedules): the surface `dispatchSchedule` drives - [Channels](../channels/overview): ingress you can exercise with `target.fetch`