workflow
Version:
Workflow SDK - Build durable, resilient, and observable workflows
318 lines (238 loc) • 15.1 kB
text/mdx
---
title: Idempotency
description: Make step retries safe and coordinate duplicate workflow starts with hook tokens.
type: conceptual
summary: Use step IDs for retry-safe external calls, and route duplicate workflow-start requests through deterministic hook tokens.
prerequisites:
- /docs/foundations/workflows-and-steps
related:
- /docs/foundations/errors-and-retries
- /docs/foundations/starting-workflows
- /docs/foundations/hooks
---
Idempotency is a property of an operation that ensures repeated attempts have the same effect as a single attempt.
In Workflow, idempotency shows up in two related places: step idempotency makes external calls safe when a step retries, and run idempotency coordinates duplicate requests that try to start the same workflow.
## Step idempotency
In distributed systems (calling external APIs), it is not always possible to ensure an operation has only been performed once by seeing if it succeeds.
Consider a payment API that charges the user $10, but due to network failures, the confirmation response is lost. When the step retries (because the previous attempt was considered a failure), it will charge the user again.
To prevent this, many external APIs support idempotency keys. An idempotency key is a unique identifier for an operation that can be used to deduplicate requests.
Every step invocation has a stable `stepId` that stays the same across retries.
Use it as the idempotency key when calling third-party APIs.
```typescript lineNumbers
import { getStepMetadata } from "workflow";
async function chargeUser(userId: string, amount: number) {
"use step";
const { stepId } = getStepMetadata(); // [!code highlight]
// Example: Stripe-style idempotency key
// This guarantees only one charge is created even if the step retries
await stripe.charges.create(
{
amount,
currency: "usd",
customer: userId,
},
{
idempotencyKey: stepId, // [!code highlight]
}
);
}
```
Why this works:
- **Stable across retries**: `stepId` does not change between attempts.
- **Globally unique per step**: Fulfills the uniqueness requirement for an idempotency key.
## Run idempotency
Step idempotency protects side effects **inside** a workflow run. Run idempotency answers a different question: if the same API request is sent twice, should it create one workflow run or two?
Because [hooks](/docs/foundations/hooks) already ensure globally unique active tokens, Workflow can use the same mechanism to coordinate duplicate requests while a run is active.
Use a hook token as the idempotency key for an active workflow run. Hook tokens are globally unique while they are active: if another run tries to create a hook with the same token, the runtime records a conflict, `hook.getConflict()` resolves with a `Run` handle for the run that owns the token, and the hook rejects with [`HookConflictError`](/docs/errors/hook-conflict) when the workflow awaits or iterates its payload.
The token should come from your domain, such as an order ID, invoice ID, import ID, or request ID. Create the hook near the beginning of the workflow and check `await hook.getConflict()` before doing duplicate-sensitive work that depends on owning the active token. Calling `createHook()` alone does not register the hook; awaiting `getConflict()` suspends the workflow to commit the registration. Check it before calling any step: steps the workflow calls before it suspends are started alongside the hook's registration, so a duplicate run that only learns of the conflict later (for example, by awaiting the hook and letting `HookConflictError` end the run) may already have started them. See [Registering a hook before a step uses it](/docs/api-reference/workflow/create-hook#registering-a-hook-before-a-step-uses-it).
```typescript lineNumbers
import { createHook } from "workflow";
type OrderRequest = { confirmed: boolean };
type OrderResult =
| { status: "processed" | "cancelled" }
| { status: "duplicate"; runId: string };
declare function chargeOrder(orderId: string): Promise<void>; // @setup
export async function processOrder(orderId: string): Promise<OrderResult> {
"use workflow";
using request = createHook<OrderRequest>({ // [!code highlight]
token: `order:${orderId}`, // [!code highlight]
}); // [!code highlight]
const conflict = await request.getConflict(); // [!code highlight]
if (conflict) { // [!code highlight]
// Another active run already owns this order's token. // [!code highlight]
return { status: "duplicate" as const, runId: conflict.runId }; // [!code highlight]
} // [!code highlight]
const { confirmed } = await request;
if (!confirmed) {
return { status: "cancelled" as const };
}
await chargeOrder(orderId);
return { status: "processed" as const };
}
```
The runtime creates the hook atomically. At most one hook can own `order:${orderId}`, so duplicate workflow runs converge on one owner. A duplicate run observes `getConflict()` resolving with the owner's `Run` and returns before it reaches `chargeOrder()`. The conflicting run's accessors (`status`, `returnValue`, `cancel()`, …) are durable steps, so the duplicate run can do more than report the owner. See [conflict-handling strategies](#conflict-handling-strategies) below.
Outside the workflow, try to resume the hook first. If the hook is not registered yet, start the workflow and retry the resume until the new run creates the hook:
```typescript lineNumbers
import { resumeHook, start } from "workflow/api";
import { HookNotFoundError } from "workflow/errors";
import { processOrder } from "./workflows/process-order";
type OrderRequest = { confirmed: boolean };
async function resumeOrder(token: string, payload: OrderRequest) {
for (let attempt = 0; attempt < 5; attempt++) {
try {
return await resumeHook(token, payload); // [!code highlight]
} catch (error) {
if (!HookNotFoundError.is(error)) throw error;
await new Promise((resolve) => setTimeout(resolve, 100));
}
}
throw new Error("Order workflow did not register its hook in time");
}
export async function POST(request: Request) {
const { orderId, confirmed } = await request.json();
const token = `order:${orderId}`;
const payload = { confirmed };
try {
const hook = await resumeHook(token, payload); // [!code highlight]
return Response.json({ runId: hook.runId, reused: true });
} catch (error) {
if (!HookNotFoundError.is(error)) throw error;
}
const run = await start(processOrder, [orderId]); // [!code highlight]
const resumed = await resumeOrder(token, payload);
// A concurrent request's run may have won the race between `start()` // [!code highlight]
// and hook registration. The resume always reaches the actual active // [!code highlight]
// owner, so compare run IDs instead of waiting for this run to finish. // [!code highlight]
return Response.json({ // [!code highlight]
runId: resumed.runId, // [!code highlight]
reused: resumed.runId !== run.runId, // [!code highlight]
}); // [!code highlight]
}
```
<Callout type="warn">
This avoids creating a new run only after the first run has registered its hook. Because `start()` returns before the run body executes and calls `createHook()`, two concurrent requests can both observe "no hook yet" and each call `start()`. The race is resolved inside the workflow body, where the losing run observes `getConflict()` resolving with the active owner and returns without doing duplicate-sensitive work, and the route detects it by comparing the resumed hook's `runId` against the run it just started, without waiting for either run to finish. A native API for atomically starting a run and registering a hook is in the works. Until then, model recovery inside the workflow by checking `hook.getConflict()`.
</Callout>
This coordinates active runs by default: the token becomes available when its workflow ends. Set `experimental_minRetention` to keep it unavailable to late duplicates. After the workflow ends, the Hook can still be found with `getHookByToken()` until retention ends, but it cannot be resumed. See [`createHook()` minimum retention](/docs/api-reference/workflow/create-hook#keep-a-token-unavailable-after-the-run-ends) for examples and supported values.
### Conflict-handling strategies
Some workflow systems resolve duplicate IDs with a fixed, pre-declared policy, typically a static choice between rejecting the new execution, deferring to the existing one, or terminating it. Workflow has no policy enum. `hook.getConflict()` hands the duplicate run the conflicting `Run` itself, and the policy is ordinary code, including policies that inspect state before deciding, which static configuration can't express.
The example above implements **reject the duplicate**: return the owner's `runId` and let the caller decide. Other common strategies:
**Adopt the owner's result.** Wait for the active run to finish and return its result, so callers cannot tell which run did the work:
```typescript lineNumbers
import { createHook } from "workflow";
type OrderRequest = { confirmed: boolean };
declare function processOwnedOrder(orderId: string): Promise<{ status: string }>; // @setup
export async function processOrder(orderId: string) {
"use workflow";
using request = createHook<OrderRequest>({
token: `order:${orderId}`,
});
const conflict = await request.getConflict();
if (conflict) {
// Callers get the same result regardless of which run did the work.
return await conflict.returnValue; // [!code highlight]
}
return await processOwnedOrder(orderId);
}
```
**Inspect the owner before deciding.** Reuse a completed owner's result, but reject other duplicates:
```typescript lineNumbers
import { createHook } from "workflow";
type OrderRequest = { confirmed: boolean };
declare function processOwnedOrder(orderId: string): Promise<{ status: string }>; // @setup
export async function processOrder(orderId: string) {
"use workflow";
using request = createHook<OrderRequest>({
token: `order:${orderId}`,
});
const conflict = await request.getConflict();
if (conflict) {
const status = await conflict.status; // [!code highlight]
if (status === "completed") {
return await conflict.returnValue;
}
return { status: "duplicate" as const, runId: conflict.runId };
}
return await processOwnedOrder(orderId);
}
```
**Signal the owner instead of doing the work.** A conflict can refer to a finished run when `experimental_minRetention` is set, so check its status before sending data to its Hook:
```typescript lineNumbers
import { createHook } from "workflow";
import { resumeHook } from "workflow/api";
type OrderRequest = { confirmed: boolean };
async function forwardToOwner(token: string, payload: OrderRequest) {
"use step";
await resumeHook(token, payload); // [!code highlight]
}
export async function processOrder(orderId: string, confirmed: boolean) {
"use workflow";
const token = `order:${orderId}`;
using request = createHook<OrderRequest>({ token });
const conflict = await request.getConflict();
if (conflict && ["pending", "running"].includes(await conflict.status)) {
await forwardToOwner(token, { confirmed }); // [!code highlight]
return { status: "forwarded" as const, runId: conflict.runId };
}
if (conflict) {
return { status: "duplicate" as const, runId: conflict.runId };
}
// ... own the token and do the work
}
```
**Supersede the owner.** Create the Hook with [`experimental_force: true`](/docs/api-reference/workflow/create-hook#take-over-a-token-another-run-holds) so the newest run takes the token over from the active owner. This also works for a finished run holding the token under `experimental_minRetention`:
```typescript lineNumbers
import { createHook } from "workflow";
type OrderRequest = { confirmed: boolean };
declare function chargeOrder(orderId: string): Promise<void>; // @setup
export async function processOrderNewestWins(orderId: string) {
"use workflow";
using request = createHook<OrderRequest>({
token: `order:${orderId}`,
experimental_force: true, // [!code highlight]
});
// This run now owns the token.
const { confirmed } = await request;
if (confirmed) {
await chargeOrder(orderId);
}
return { status: "processed" as const };
}
```
The previous owner's `await request` rejects with [`HookForceClaimedError`](/docs/api-reference/workflow-errors/hook-force-claimed-error), and every `resumeHook()` for the token reaches the new run from then on. Any run of this workflow can be superseded by a later one, so catch the error and exit cleanly:
```typescript lineNumbers
import { createHook } from "workflow";
import { HookForceClaimedError } from "workflow/errors";
type OrderRequest = { confirmed: boolean };
declare function chargeOrder(orderId: string): Promise<void>; // @setup
export async function processOrderNewestWins(orderId: string) {
"use workflow";
using request = createHook<OrderRequest>({
token: `order:${orderId}`,
experimental_force: true,
});
try {
const { confirmed } = await request;
if (confirmed) {
await chargeOrder(orderId);
}
return { status: "processed" as const };
} catch (error) {
if (HookForceClaimedError.is(error)) { // [!code highlight]
// A newer run for this order owns the token now. Stop here.
return { status: "superseded" as const, runId: error.claimedByRunId }; // [!code highlight]
}
throw error;
}
}
```
A run started at a Workflow spec version below 8, including runs started by older SDK releases, can't be taken from. In that case the forced Hook rejects with `HookConflictError`, as it would without `experimental_force`.
If duplicate requests should only reuse the active run without sending data, use [`getHookByToken()`](/docs/api-reference/workflow-api/get-hook-by-token) as an advisory pre-check before calling `start()`. The workflow should still check `hook.getConflict()`, because the lookup and `start()` are not atomic.
Because this pattern uses hooks for idempotency, duplicate requests can also inject additional data and steer the existing run. The route example above uses `resumeHook()` for that: if the hook already exists, the duplicate request resumes the active workflow; if the hook is not registered yet, the route starts the workflow and retries `resumeHook()` so the payload is not dropped.
## Related docs
- Learn about retries in [Errors & Retrying](/docs/foundations/errors-and-retries)
- API reference: [`getStepMetadata`](/docs/api-reference/workflow/get-step-metadata)
- API reference: [`createHook()`](/docs/api-reference/workflow/create-hook)
- API reference: [`getHookByToken()`](/docs/api-reference/workflow-api/get-hook-by-token)
- API reference: [`resumeHook()`](/docs/api-reference/workflow-api/resume-hook)
- API reference: [`start()`](/docs/api-reference/workflow-api/start)
- Learn about deterministic hook tokens in [Hooks](/docs/foundations/hooks)