UNPKG

workflow

Version:

Workflow SDK - Build durable, resilient, and observable workflows

160 lines (115 loc) • 9.56 kB
--- title: Sequential & Parallel Execution description: Compose steps with familiar async/await patterns, sequential await, Promise.all, and Promise.race. type: guide summary: Workflows are plain async functions, so all the standard composition primitives (await, Promise.all, Promise.race) apply unchanged, including racing webhooks against durable sleeps. related: - /docs/foundations/workflows-and-steps - /cookbook/common-patterns/timeouts - /cookbook/common-patterns/scheduling --- <CopyPrompt text="Compose these workflow steps with standard async/await patterns. In the exported &quot;use workflow&quot; function, chain dependent &quot;use step&quot; calls with sequential `await`; run independent steps concurrently by starting them without `await` and awaiting `Promise.all([...])`; and use `Promise.race([...])` to act on whichever promise settles first. These compose with durable primitives: race a step or a webhook from `createWebhook()` against `sleep()` from `workflow` for deadlines. Keep every step input and output serializable, and remember `Promise.race` does not cancel the losing branch (it keeps running), so side-effectful losers need idempotency keys. When the fan-out is wide, consider keeping concurrency bounded by chunking the array into batches or bundling several items per step rather than one flat `Promise.all` over everything. Verify sequential ordering, parallel execution, and both race outcomes." /> Workflows are written in plain async/await: there's no new control-flow API to learn. Sequential awaits chain steps that depend on each other, `Promise.all` runs independent steps in parallel, and `Promise.race` returns whichever finishes first. These compose with workflow primitives like [`sleep()`](/docs/api-reference/workflow/sleep) and [`createWebhook()`](/docs/api-reference/workflow/create-webhook) since those are also promises. ## When to use this - **Pipelines**: Each step depends on the previous step's output (validate → process → store). - **Independent fan-out**: Fetch multiple resources or perform multiple actions that don't depend on each other. - **Race conditions**: Return as soon as one operation completes, such as for a timeout, first responder, or deadline. - **Mixing primitives**: Run steps, sleeps, and webhooks side by side in the same control-flow expression. ## Pattern ### Sequential The simplest way to orchestrate steps is to execute them one after another, where each step depends on the previous step's output. ```typescript lineNumbers declare function validateData(data: unknown): Promise<string>; // @setup declare function processData(data: string): Promise<string>; // @setup declare function storeData(data: string): Promise<string>; // @setup export async function dataPipelineWorkflow(data: unknown) { "use workflow"; const validated = await validateData(data); const processed = await processData(validated); const stored = await storeData(processed); return stored; } ``` ### Parallel with `Promise.all` When steps don't depend on each other, run them concurrently with `Promise.all`. The workflow waits until all of them resolve. ```typescript lineNumbers declare function fetchUser(userId: string): Promise<{ name: string }>; // @setup declare function fetchOrders(userId: string): Promise<{ items: string[] }>; // @setup declare function fetchPreferences(userId: string): Promise<{ theme: string }>; // @setup export async function fetchUserData(userId: string) { "use workflow"; const [user, orders, preferences] = await Promise.all([ // [!code highlight] fetchUser(userId), // [!code highlight] fetchOrders(userId), // [!code highlight] fetchPreferences(userId), // [!code highlight] ]); // [!code highlight] return { user, orders, preferences }; } ``` ### Race with `Promise.race` `Promise.race` resolves as soon as the first promise settles. Since [`sleep()`](/docs/api-reference/workflow/sleep) and [`createWebhook()`](/docs/api-reference/workflow/create-webhook) return promises, they compose naturally. For example, waiting for a webhook callback with a deadline: ```typescript lineNumbers import { sleep, createWebhook } from "workflow"; declare function executeExternalTask(webhookUrl: string): Promise<void>; // @setup export async function runExternalTask(userId: string) { "use workflow"; const webhook = createWebhook(); await executeExternalTask(webhook.url); await Promise.race([ // [!code highlight] webhook, // [!code highlight] sleep("1 day"), // [!code highlight] ]); // [!code highlight] console.log("Done"); } ``` For racing operations against deadlines specifically (timeouts), see the dedicated [Timeouts](/cookbook/common-patterns/timeouts) recipe, which covers result discrimination, `FatalError` semantics, and the "loser keeps running" caveat. ### Combining sequential, parallel, and durable primitives Most workflows combine all three patterns. The following simplified version of the [birthday card generator demo](https://github.com/vercel/workflow-examples/tree/main/birthday-card-generator) uses sequential card generation, parallel invitation-response fan-out, non-blocking webhook collection, and a durable sleep until the birthday: ```typescript lineNumbers import { createWebhook, sleep, type Webhook } from "workflow"; declare function makeCardText(prompt: string): Promise<string>; // @setup declare function makeCardImage(text: string): Promise<string>; // @setup declare function sendRSVPEmail(friend: string, webhook: Webhook): Promise<void>; // @setup declare function sendBirthdayCard(text: string, image: string, rsvps: unknown[], email: string): Promise<void>; // @setup export async function birthdayWorkflow( prompt: string, email: string, friends: string[], birthday: Date ) { "use workflow"; const text = await makeCardText(prompt); // [!code highlight] const image = await makeCardImage(text); // [!code highlight] const webhooks = friends.map(() => createWebhook()); await Promise.all( // [!code highlight] friends.map((friend, i) => sendRSVPEmail(friend, webhooks[i])) // [!code highlight] ); // [!code highlight] const rsvps: unknown[] = []; webhooks.map((webhook) => webhook.then((req) => req.json()).then(({ rsvp }) => rsvps.push(rsvp)) ); await sleep(birthday); // [!code highlight] await sendBirthdayCard(text, image, rsvps, email); return { text, image, status: "Sent" }; } ``` ## How it works 1. **`await` is durable**: When the workflow awaits a step, the runtime persists the step's input, suspends the workflow, runs the step, and replays the workflow with the step's result on resume. The same applies to `sleep()` and `createWebhook()`. 2. **`Promise.all` runs steps concurrently**: Each promise in the array is suspended on its own, and the workflow resumes only when all have settled. If any promise rejects, the whole `Promise.all` rejects. 3. **`Promise.race` resolves when the first promise settles**: The losing promises keep running in the background, but the workflow discards their results. 4. **All primitives are promises**: `sleep("1 day")` and `createWebhook()` return promises, so they compose with `Promise.all` and `Promise.race` like steps do. This behavior enables patterns such as racing a webhook against a 24-hour deadline. ## Adapting to your use case - **Replace `Promise.all` with `Promise.allSettled`**: Use this option when partial failures shouldn't abort the remaining operations. You'll get an array of `{ status, value | reason }` instead of an error on the first rejection. - **Bound the parallelism**: `Promise.all` over a large array fans out one concurrent step per item, all in the same run, and each item's events land in that one log. When concurrency gets high, batch the array into chunks or bundle several items into each step (see [Batching](/cookbook/common-patterns/batching)) so fewer, larger units of work are in flight. Splitting a wide fan-out across [child workflows](/cookbook/advanced/child-workflows) keeps each log shorter and isolates failures, but it does not by itself narrow the fan-out. Both the log and the step count are capped per run: see [Vercel World limits](/worlds/vercel#per-run-limits), and split before a run would grow past a few thousand events. - **Add a deadline to any race**: Pair the operation with `sleep("30s").then(() => "timeout" as const)` and check the discriminated result. See [Timeouts](/cookbook/common-patterns/timeouts). - **Mix steps and hooks in a race**: Wait for an external signal, a deadline, or a step result in the same `Promise.race`. The first promise to resolve wins. ## Key APIs - [`"use workflow"`](/docs/foundations/workflows-and-steps): Marks the orchestrator function. - [`"use step"`](/docs/foundations/workflows-and-steps): Marks functions with full Node.js access. - [`sleep()`](/docs/api-reference/workflow/sleep): Provides durable sleep that survives restarts. - [`createWebhook()`](/docs/api-reference/workflow/create-webhook): Provides a webhook URL that the workflow can race against. - [`Promise.all()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all): Waits for all promises. - [`Promise.race()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/race): Waits for the first promise to settle. - [`Promise.allSettled()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled): Waits for all promises, including failures.