UNPKG

workflow

Version:

Workflow SDK - Build durable, resilient, and observable workflows

509 lines (330 loc) • 33.2 kB
--- title: Postgres World description: Self-hosted reference world using PostgreSQL for storage and graphile-worker for job processing. type: integration summary: Deploy workflows to your own infrastructure using PostgreSQL and graphile-worker. prerequisites: - /docs/deploying related: - /worlds/local - /worlds/vercel --- The Postgres World is a self-hosted backend that uses PostgreSQL for durable storage and [graphile-worker](https://github.com/graphile/worker) for reliable job processing. Use the Postgres World to deploy workflows on your own infrastructure outside Vercel, such as a Docker container, Kubernetes cluster, or any cloud that supports long-running servers. <Callout type="warn"> The Postgres World is a **reference implementation**: it implements the complete World spec and is tested, but it is not optimized for scale, speed, or security, and it ships with **no authentication** on the workflow HTTP routes. For production use-cases we recommend cloning it and adapting it to your persistence, network stack, scale, and security requirements. If you deploy it as is, you must restrict those routes yourself. Read [Security](#security) before you expose the app. </Callout> ## Quickstart These steps take an existing workflow app (see [Getting Started](/docs/getting-started)) from the Local World to the Postgres World. Everything after this section is for production and tuning, and none of it is needed to get a first run going. <Steps> <Step> ### Start a PostgreSQL database Any PostgreSQL database works, self-hosted or managed. For local development, this container matches the World's default connection string, `postgres://world:world@localhost:5432/world`: ```bash docker run -d --name workflow-postgres -p 127.0.0.1:5432:5432 \ -e POSTGRES_USER=world -e POSTGRES_PASSWORD=world -e POSTGRES_DB=world \ postgres:18-alpine ``` </Step> <Step> ### Install the package ```package-install @workflow/world-postgres ``` Keep `workflow` and `@workflow/world-postgres` on the same major version and release cycle, including prereleases. A mismatch fails before starting a run, with an error that names the spec versions the runtime supports and the one the World declares. </Step> <Step> ### Point your app at the World ```bash title=".env" WORKFLOW_TARGET_WORLD="@workflow/world-postgres" WORKFLOW_POSTGRES_URL="postgres://world:world@localhost:5432/world" ``` `WORKFLOW_POSTGRES_URL` falls back to `DATABASE_URL`, then to the default above. See [Configuration](#configuration) for everything else you can set. </Step> <Step> ### Create the tables Run the migrations against the same database. The command reads `WORKFLOW_POSTGRES_URL` or `DATABASE_URL`: ```bash tab="npm" npx --package=@workflow/world-postgres bootstrap ``` ```bash tab="pnpm" pnpm dlx --package @workflow/world-postgres bootstrap ``` ```bash tab="Yarn" yarn dlx --package @workflow/world-postgres bootstrap ``` ```bash tab="Bun" bunx --package @workflow/world-postgres bootstrap ``` The command is idempotent, so you can run it on every deploy. </Step> <Step> ### Start the World when your server starts The Postgres World runs a background worker that pulls jobs from graphile-worker, so your app has to start it once when the server boots: <TabsWithChildren tabs={["Next.js","SvelteKit","Nitro"]}> <TabContent order={1}> Create an `instrumentation.ts` file in your project root: ```ts title="instrumentation.ts" lineNumbers export async function register() { if (process.env.NEXT_RUNTIME !== "edge") { const { getWorld } = await import("workflow/runtime"); const world = await getWorld(); await world.start?.(); } } ``` <Callout type="info"> Learn more about [Next.js Instrumentation](https://nextjs.org/docs/app/guides/instrumentation). </Callout> </TabContent> <TabContent order={2}> Create a `src/hooks.server.ts` file: ```ts title="src/hooks.server.ts" lineNumbers import type { ServerInit } from "@sveltejs/kit"; export const init: ServerInit = async () => { const { getWorld } = await import("workflow/runtime"); const world = await getWorld(); await world.start?.(); }; ``` <Callout type="info"> Learn more about [SvelteKit Hooks](https://svelte.dev/docs/kit/hooks). </Callout> </TabContent> <TabContent order={3}> Create a plugin to start the world on server initialization: ```ts title="plugins/start-pg-world.ts" lineNumbers import { defineNitroPlugin } from "nitro/~internal/runtime/plugin"; export default defineNitroPlugin(async () => { const { getWorld } = await import("workflow/runtime"); const world = await getWorld(); await world.start?.(); }); ``` Register the plugin in your config: ```ts title="nitro.config.ts" import { defineNitroConfig } from "nitropack"; export default defineNitroConfig({ modules: ["workflow/nitro"], plugins: ["plugins/start-pg-world.ts"], }); ``` <Callout type="info"> Learn more about [Nitro Plugins](https://v3.nitro.build/docs/plugins). </Callout> </TabContent> </TabsWithChildren> <Callout type="info"> Only Worlds with a background worker need this hook. If the same app also deploys with the [Vercel World](/worlds/vercel), skip it there: `start()` does nothing on that World, and importing `workflow/runtime` from a startup hook puts the whole runtime on the cold-start path. </Callout> </Step> <Step> ### Run a workflow Start your dev server and trigger a workflow as usual. Then check that the run landed in PostgreSQL, from the terminal or in the web UI: ```bash # List workflow runs npx workflow inspect runs --backend @workflow/world-postgres # Launch the web UI npx workflow web --backend @workflow/world-postgres ``` </Step> </Steps> ## Deploying to production The Postgres World needs a long-lived process to run its worker, so it does not work on serverless platforms. Deploy your application to any cloud that supports long-running servers: - Docker containers - Kubernetes clusters - Virtual machines - Platform-as-a-service (PaaS) providers, such as Railway, Render, and Fly.io Before you go live, make sure that: - The app can reach your PostgreSQL database, and `WORKFLOW_TARGET_WORLD` and `WORKFLOW_POSTGRES_URL` are set in its runtime environment - `bootstrap` runs on every deploy, for example as a post-deployment lifecycle script (see [Upgrading](#upgrading)) - `start()` is called on server initialization - `/.well-known/workflow/v1/flow` is restricted so untrusted clients cannot reach it (see [Security](#security)) - If more than one process shares the database, you have set [`WORKFLOW_POSTGRES_JOB_LOCK_STALE_SECONDS`](#workflow_postgres_job_lock_stale_seconds) to the same value on every process, so another process picks up the jobs of one that dies <Callout type="info"> The Postgres World is not compatible with Vercel deployments. On Vercel, workflows automatically use the [Vercel World](/worlds/vercel) with zero configuration. </Callout> ## Security Unlike the [Vercel World](/worlds/vercel), which authenticates and [encrypts](/docs/how-it-works/encryption) workflow traffic automatically, the Postgres World does **not** automatically authenticate requests that drive workflow execution. Adding that is your responsibility and should be in place before an app using this World is used in production and reachable by untrusted clients. We also recommend [enabling encryption](/docs/how-it-works/encryption#custom-world-implementations). ### Protect the queue route `POST /.well-known/workflow/v1/flow` is where the worker delivers workflow orchestration and queued step invocations. It is mounted in your application like any other route, so it is publicly reachable by default, and it accepts any request whose headers and body are well-formed: there is no signature, shared secret, or caller check. Anyone who can reach it can forge or replay a workflow or step invocation, including steps your application would only reach after its own gating. Restrict it before you expose the app. The runtime does not check deliveries to this route with a [queue key](/worlds/building-a-world#authenticating-deliveries): the Postgres World declares `queueAuthMode: 'skip'` until its messages are delivered in-process and the route is denied ([#5258](https://github.com/vercel/workflow/issues/5258)). The other routes under `/.well-known/workflow/v1/` differ: - `webhook/:token`, created by [`createWebhook()`](/docs/api-reference/workflow/create-webhook), is authorized by the token in the URL and nothing else. Use [`createHook()`](/docs/api-reference/workflow/create-hook) behind your own authenticated route and [`resumeHook()`](/docs/api-reference/workflow-api/resume-hook) when you need more than that. - `manifest.json` responds with `404` unless the app was built with `WORKFLOW_PUBLIC_MANIFEST=1`. That variable is read at build time, so unsetting it in the runtime environment of an already-built deployment does not withdraw the manifest. Leave it unset outside of testing, because the manifest lists your workflow and step names. A queue consumer that registered its own route with [`registerRoute()`](#queue-consumers-other-than-the-workflow-sdk) receives deliveries the same way, so protect that route like the flow route. ### Bringing your own auth Workflow does not prescribe an auth mechanism for self-hosted Worlds, so restrict the flow route at the network edge rather than inside the application: - **Keep the flow route unreachable from outside.** By default the worker delivers to a loopback address (`http://localhost:{PORT}`, or `WORKFLOW_LOCAL_BASE_URL` when set), so in the common single-process topology nothing outside the container needs to reach it. Blocking external requests to `/.well-known/workflow/v1/flow` at your ingress, reverse proxy, or firewall costs you nothing, because loopback delivery never traverses that layer. Do not block the whole `/.well-known/workflow/` prefix if you use [`createWebhook()`](/docs/api-reference/workflow/create-webhook): its `webhook/:token` route sits under the same prefix and has to stay reachable by whoever calls it. - **Authenticate at the proxy when the routes must cross hosts.** If your web tier and workers are separate deployments, require mTLS or a shared-secret header at the proxy in front of the application, and strip any client-supplied copy of that header at the edge. - **Do not gate these paths in framework middleware.** The [Next.js setup guide](/docs/getting-started/next) tells you to exclude `/.well-known/workflow/*` from your proxy matcher, because a handler that consumes the internal request body breaks execution. Adding the paths back in order to gate them reintroduces that failure mode. - **Do not expect the World to present a credential.** It does not sign its delivery requests or attach a secret to them, so an in-application check that requires one will reject the World's own deliveries. ### Data at rest The Postgres World does not currently implement `getEncryptionKeyForRun()`, so it does not participate in Workflow's [end-to-end encryption](/docs/how-it-works/encryption): workflow and step inputs and return values, hook payloads and metadata, and stream chunks are all stored in your database in readable form. A World derived from this one can opt in by implementing that one method; see [Custom World implementations](/docs/how-it-works/encryption#custom-world-implementations). Until then, protect the database, its credentials, and its backups accordingly. The [observability UI](/docs/observability) has no authentication of its own either. If you self-host `@workflow/web` against your Postgres database, put it behind your own auth layer, because everyone who can reach it can read every run. ## Observability Use the Workflow CLI to inspect workflows stored in PostgreSQL: ```bash # Set your database URL export WORKFLOW_POSTGRES_URL="postgres://user:password@host:5432/database" # List workflow runs npx workflow inspect runs --backend @workflow/world-postgres # Launch the web UI npx workflow web --backend @workflow/world-postgres ``` If `WORKFLOW_POSTGRES_URL` is not set, the CLI defaults to `postgres://world:world@localhost:5432/world`. Learn more in the [Observability](/docs/observability) documentation. ## Configuration The defaults work for a single process on a local database, so most apps set only `WORKFLOW_TARGET_WORLD` and `WORKFLOW_POSTGRES_URL`. You can set every option through environment variables, or programmatically through `createWorld()` (see [Programmatic configuration](#programmatic-configuration)). | Variable | Default | Controls | | --- | --- | --- | | [`WORKFLOW_POSTGRES_URL`](#workflow_postgres_url) | `DATABASE_URL`, then `postgres://world:world@localhost:5432/world` | Database connection | | [`WORKFLOW_POSTGRES_JOB_PREFIX`](#workflow_postgres_job_prefix) | `workflow_` | Job name prefix, for sharing a database between apps | | [`WORKFLOW_POSTGRES_WORKER_CONCURRENCY`](#workflow_postgres_worker_concurrency) | `50` | Concurrent jobs per process | | [`WORKFLOW_POSTGRES_POLL_INTERVAL_MS`](#workflow_postgres_poll_interval_ms) | `500` ms | Idle job polling | | [`WORKFLOW_POSTGRES_MAX_POOL_SIZE`](#workflow_postgres_max_pool_size) | `10` | Connection pool size | | [`WORKFLOW_POSTGRES_APPLICATION_MANAGED_SHUTDOWN`](#workflow_postgres_application_managed_shutdown) | off | Who coordinates shutdown | | [`WORKFLOW_POSTGRES_HEADERS_TIMEOUT_MS`](#workflow_postgres_headers_timeout_ms) | `0` ms (no deadline) | Hung delivery detection | | [`WORKFLOW_POSTGRES_BODY_TIMEOUT_MS`](#workflow_postgres_body_timeout_ms) | `0` ms (no deadline) | Stalled response detection | | [`WORKFLOW_POSTGRES_JOB_LOCK_STALE_SECONDS`](#workflow_postgres_job_lock_stale_seconds) | unset: 300 s for spec 9 named queues only | Redelivery after a process dies | | [`WORKFLOW_POSTGRES_HOOK_RETENTION_LIMIT_DAYS`](#workflow_postgres_hook_retention_limit_days) | `30` days | Maximum hook retention | | [`WORKFLOW_QUEUE_NAMESPACE`](#workflow_queue_namespace) | unset | Queue topic namespace | | [`WORKFLOW_STREAM_FLUSH_INTERVAL_MS`](#workflow_stream_flush_interval_ms) | `0` ms | Stream write batching | | [`WORKFLOW_POSTGRES_INVOKE`](#experimental-invocation-delivery) | off | Experimental: hook resumes only | ### `WORKFLOW_POSTGRES_URL` PostgreSQL connection string used by the runtime World. Precedence: `WORKFLOW_POSTGRES_URL` > `DATABASE_URL` > `postgres://world:world@localhost:5432/world` The `bootstrap` migration command uses the same precedence. ### `WORKFLOW_POSTGRES_JOB_PREFIX` Prefix for graphile-worker queue job names. Default: `workflow_` Give each application that shares a database its own prefix. A World claims jobs only under its own prefix and stamps that prefix on every run it creates. On `start()`, it re-enqueues only the active runs stamped with its prefix, so restarting one application does not hand another application's runs to its worker. `WORKFLOW_QUEUE_NAMESPACE` does not separate applications that share a database. Set the prefix through `WORKFLOW_POSTGRES_JOB_PREFIX` rather than only in code. The CLI and the observability UI build their World from the environment, so a run they create, such as a re-run, is stamped with the prefix they see, and only a World with that prefix runs and recovers it. ### `WORKFLOW_POSTGRES_WORKER_CONCURRENCY` Number of concurrent workers polling for jobs. Default: `50`. A parent workflow awaiting `childRun.returnValue` of a child it started does not hold a worker slot: it suspends on a system hook the child resumes when it finishes (see [Child Workflows](/cookbook/advanced/child-workflows#how-the-parent-waits)). A `returnValue` that falls back to polling still does, for example one awaited inside a step, or any await while `WORKFLOW_CHILD_RUN_COMPLETION_HOOK=0` is set. Each of those holds a worker slot until its child run terminates, so raise this ceiling above the peak number of such polls in flight. ### `WORKFLOW_POSTGRES_POLL_INTERVAL_MS` Milliseconds between idle job fetches per worker. Each worker polls on its own, so an idle process runs about `queueConcurrency × 1000 / pollInterval` fetches per second. Default: `500`. ### `WORKFLOW_POSTGRES_MAX_POOL_SIZE` Maximum size of the internal `pg.Pool` used when `createWorld()` constructs the pool. Default: the `pg` default (`10`). For higher worker concurrency, Graphile Worker recommends setting `maxPoolSize` to `10` or `queueConcurrency + 2`, whichever is larger. ### `WORKFLOW_POSTGRES_APPLICATION_MANAGED_SHUTDOWN` Set to `1` when the application or framework coordinates shutdown and awaits `world.close()` before closing its workflow HTTP server and any caller-owned pool. Default: unset (`false`). ### `WORKFLOW_POSTGRES_HEADERS_TIMEOUT_MS` Maximum time in milliseconds a queue delivery waits for the workflow handler to begin responding before the delivery is failed and Graphile Worker redelivers the message. Default: `0` (no deadline). A delivery executes the workflow body inline, so the handler responds only once that work is done. A deadline shorter than your longest inline step declares a healthy delivery crashed and redelivers it while the original is still running, executing the same steps twice. Set this only if you would rather a hung handler be redelivered than hold its worker slot until the process restarts, and set it above the longest inline work you expect. ### `WORKFLOW_POSTGRES_BODY_TIMEOUT_MS` Maximum gap in milliseconds between response body chunks from the workflow handler before the delivery is failed and redelivered. Default: `0` (no deadline). The same caution as `WORKFLOW_POSTGRES_HEADERS_TIMEOUT_MS` applies. ### `WORKFLOW_POSTGRES_JOB_LOCK_STALE_SECONDS` Seconds a running delivery's job lock may go unrenewed before another process releases the job for redelivery. Default: unset (see below). `30` is a good value when more than one process shares the database. Programmatic option: `jobLockStaleSeconds`. A running delivery renews its lock every quarter of the window, and at least every 10 seconds. When a process dies mid-delivery, another process redelivers its jobs within about one and a half windows (about 45 seconds at `30`), instead of after Graphile Worker's fixed 4 hour stale-lock reset or the next restart. Unset, every delivery still renews its lock, and its worker registers as renewing in `workflow.workflow_job_lock_holders` (migration 0030, which `bootstrap` applies). Then only the jobs of a spec 9 run's named queue (its orchestrator deliveries, and invoke mode's executor lane) are released, after 300 seconds unrenewed, and only those whose holder registered. A spec 9 run whose worker was killed therefore continues within about 8 minutes (one and a half windows) instead of 4 hours. A job held by a process on an earlier version, which does not register, is never released this way, so the default is safe during a rolling upgrade. `0` turns renewal and release off. Use the same value on every process that shares the database, and keep it well above the longest a process can go without running its timers (synchronous work that blocks the event loop) or getting a pool connection. A delivery whose renewals stall past the window is redelivered while it still runs. Setting it releases every job of this World, whoever holds it, so set it only after a rolling upgrade has finished. A process on an earlier version doesn't renew its locks, so a process that has this set would redeliver any of that process's deliveries that run longer than the window. When the earlier-version process then finishes such a delivery, its acknowledgement, which isn't fenced on the lock holder, can delete the redelivered job's row, and the redelivery loses its retry. To set it, use `WORKFLOW_POSTGRES_JOB_LOCK_STALE_SECONDS=30` (or `jobLockStaleSeconds: 30`) on every process that shares the database, once all of them run a version that includes this option. ### `WORKFLOW_POSTGRES_HOOK_RETENTION_LIMIT_DAYS` Maximum [`experimental_minRetention`](/docs/api-reference/workflow/create-hook#keep-a-token-unavailable-after-the-run-ends) accepted by the Postgres World, in days. Default: `30`. Set this to the same limit as your production World so oversized values fail during development. ### `WORKFLOW_QUEUE_NAMESPACE` Queue topic namespace shared by build output and the Postgres World. Default: unset. For example, `custom` changes the queue topic prefix from `__wkf_workflow_` to `__custom_wkf_workflow_`. The value must be lowercase alphanumeric and start with a letter. ### `WORKFLOW_STREAM_FLUSH_INTERVAL_MS` Group-commit window, in milliseconds, for the leading chunk of an idle stream. Default: `0` (dispatch immediately). A positive value holds the first chunk up to that long to collect a group, trading first-chunk latency for fewer requests. Chunks arriving while a request is in flight always coalesce into the next group regardless. ## Upgrading Run `bootstrap` again whenever you upgrade `@workflow/world-postgres`: a release can add migrations that the upgraded World needs before it handles workflows. Upgrade `workflow` alongside it so both stay on the same release cycle. <Callout type="warn"> Migration 0027 converts every timestamp column to `timestamp with time zone`. On a server whose time zone is UTC (the default on most hosted providers) it only changes the catalog and rebuilds one small index, and the upgrade order does not matter. On a server in another time zone it also rewrites each `created_at` value from local time to the instant it recorded, about one rewrite of each table, holding the tables' locks until it finishes. There, stop the previous version's workers, run `bootstrap`, then start the upgraded app: older versions read the converted columns shifted by the server's UTC offset, so you cannot roll back past this migration either. If your app's connection pool set a session time zone other than the server's, pass it to the migration with `?options=-c%20workflow.legacy_timezone%3DEurope%2FBerlin` in the connection string, or with `ALTER DATABASE ... SET workflow.legacy_timezone = 'Europe/Berlin'` where your pooler drops startup `options`. The migration waits at most 10 seconds for its locks; if a long transaction holds them, `bootstrap` fails and can be run again. </Callout> ### Start Hooks Migration `0028_add_start_hook_admissions` adds [`start({ experimental_hook })`](/docs/api-reference/workflow-api/start#prevent-duplicate-starts) support. It is required for every Postgres World on this version, because every Hook insert writes its `claims_token` column. It takes table locks on `workflow_hooks` and then `workflow_runs` (building the new unique index scans `workflow_hooks` first), and gives up after 10 seconds (unless the session sets its own `lock_timeout`), leaving the migration to retry, rather than stall traffic. Apply it with the rest of the migrations before deploying this version. Stop workers running a previous version before your app passes `experimental_hook` to `start()`. The queue does not route by deployment, so an older worker that picks up a candidate's queued message would create its run without the start Hook, and the start would no longer be idempotent (`start()` reports that the World did not confirm the start Hook). An older runtime replaying a start Hook run can't bind the workflow's `createHook({ token })` to the start Hook either. For the same reason, don't roll back past migration `0028_add_start_hook_admissions` while start Hooks are in use. ## Advanced Most apps don't need anything in this section. It covers building your own World module, sharing the queue with other consumers, coordinating shutdown yourself, and an experimental delivery mode. ### Programmatic configuration {/*@skip-typecheck: incomplete code sample*/} ```typescript title="my-world.ts" lineNumbers import { createWorld } from "@workflow/world-postgres"; export default createWorld({ connectionString: process.env.WORKFLOW_POSTGRES_URL ?? process.env.DATABASE_URL!, jobPrefix: "myapp_", namespace: "myapp", queueConcurrency: 50, maxPoolSize: 52, // overrides WORKFLOW_POSTGRES_MAX_POOL_SIZE streamFlushIntervalMs: 10, }); ``` Options passed to `createWorld()` take precedence over the environment variables above. Export the World from a module and point `WORKFLOW_TARGET_WORLD` at that module: ```bash title=".env" WORKFLOW_TARGET_WORLD="./my-world.ts" ``` You can also pass an existing `pg.Pool` as `pool` instead of a connection string. The World then leaves that pool's `'error'` events to you. A connection the server ends, in a restart, a failover or an idle timeout, emits one, and an `'error'` event with no listener ends the process. Listen on each client the pool connects as well as on the pool: pg-pool listens only on idle clients, and the World holds a checked-out client across a transaction's statements. {/*@skip-typecheck: requires pg, which the docs type-check does not install*/} ```typescript title="my-world.ts" lineNumbers import { createWorld } from "@workflow/world-postgres"; import { Pool } from "pg"; const pool = new Pool({ connectionString: process.env.DATABASE_URL }); pool.on("connect", (client) => client.on("error", (error) => console.error(error.message)) ); pool.on("error", () => {}); // An idle client's error, logged just above export default createWorld({ pool }); ``` A pool the World creates from `connectionString` already has these listeners. ### Queue consumers other than the Workflow SDK The worker delivers every job over HTTP, and by default to the Workflow SDK's flow route, `/.well-known/workflow/v1/flow`. A library that consumes messages of its own on the same World, such as a durable agent harness, uses its own queue prefix (`__{namespace}_wkf_workflow_`) and registers where those go: `world.registerRoute(prefix, "/api/agent/queue")` to deliver them to its own route, or `world.registerHandler(prefix, handler)` to deliver them inside the process that took the job, without HTTP. A job keeps the prefix it was enqueued under and the route its producer had registered, so a worker in another process delivers it to the same place. Register before calling `start()`, which re-enqueues each active run to the queue namespace it recorded at creation (`executionContext.queueNamespace`). ### Application-managed shutdown Graphile Worker responds automatically when the application is asked to shut down. If your application or framework already coordinates a broader shutdown sequence, set `WORKFLOW_POSTGRES_APPLICATION_MANAGED_SHUTDOWN=1` when selecting the package directly with `WORKFLOW_TARGET_WORLD`, or set `applicationManagedShutdown: true` in a programmatic World. Use the application's normal shutdown hook. This prevents Graphile Worker's default handler from terminating the process as soon as its queue stops. The hook must handle cleanup errors and await resources in this order: 1. `world.close()` 2. The workflow HTTP server 3. Any caller-owned `pg.Pool` Closing the world stops new queue claims and waits for active jobs. After Graphile Worker's grace period, a pending workflow HTTP request is aborted. Graphile Worker unlocks that same row through its normal failure handling. The already-claimed delivery consumes a Graphile attempt and is retried only if its attempt budget remains; a one-attempt or final-attempt job is not retried. The shutdown handler does not insert a successor row. Because a client abort does not prove the server handler stopped, workflow and step handlers must tolerate at-least-once execution. ### Experimental invocation delivery Invocation delivery is off by default, and you don't need it to run the Postgres World. Normally [`resumeHook()`](/docs/api-reference/workflow-api/resume-hook) writes the hook's event itself and then enqueues the run to wake up. With invocation delivery on, `resumeHook()` instead sends the payload to the run's executor and resolves once the executor has validated the input and persisted the hook event. It covers hook resumes only for now. To turn it on, set `WORKFLOW_POSTGRES_INVOKE=1` or pass `enableInvoke: true` to `createWorld()`. Apply the migrations and upgrade producers and workers together before enabling it. Workflow user code can consume the event later. The World stores inputs and responses in `workflow.workflow_invocations`. Inserting an input and enqueueing a workflow execution job share one transaction. Retries also enqueue an execution job, including when a retained response already exists, so committed events can be replayed after a runner failure. With the default job prefix, Graphile uses `workflow_flows_executor` for workflow execution and `workflow_flows` for step jobs and health checks. Execution jobs use a named queue per run, `workflow_flows:<runId>:executor`. Graphile permits one active job on each named queue across workers. Other runs, steps, and health checks can run concurrently. `queueConcurrency` limits total active jobs per worker process and defaults to 50. A custom job prefix replaces `workflow_` in these names. The HTTP receiver checks each execution request against its active Graphile job before processing pending inputs. Updated workers and receivers move legacy orchestration requests to the run's named queue. Before acknowledging an execution job, the World finishes any input already being processed and replays newly committed events. These entry checks do not stop an old handler from writing after its job is reclaimed. The caller receives a stored return value or a restored Workflow error. Terminal errors are retained as outcomes; transient or unrecognized failures leave inputs pending for Graphile to retry. Migration 0022 preserves the meaning of responses stored by earlier previews. Failure to store or read a response leaves the processing outcome unknown. Durable resume identities deduplicate the hook event when response storage must be retried. A shared `LISTEN/NOTIFY` connection signals changes to pending inputs and responses. Notifications contain identifiers, and readers fetch data from the table. Use a session-capable connection for `LISTEN`. Readers fall back to polling at 1s intervals if notifications fail, and connection retries use a 1s backoff. Notification failure and recovery are logged once per transition without payloads or connection details. The response-wait timeout defaults to 30s and can be set with `timeoutMs`. Encoded inputs and outcomes are limited to 1 MiB each. A timeout does not cancel processing. `$retention: 0` purges invocation data and prevents late writes from restoring it. Callers whose result has expired receive a 410 error with code `INVOCATION_DATA_EXPIRED`, even if the hook event committed earlier. Automatic cleanup of other results, stopping writes from reclaimed handlers, and routing requests across incompatible worker versions remain unimplemented. ## How it works The Postgres World uses PostgreSQL as a durable backend: - **Storage**: Workflow runs, events, steps, and hooks are stored in PostgreSQL tables. - **Job queue**: [graphile-worker](https://github.com/graphile/worker) handles reliable job processing with retries. - **Crash recovery**: The next `start()` re-enqueues active runs. A running delivery also renews its job lock. If its process dies, any other process redelivers a spec 9 run's orchestrator job once its lock has gone unrenewed for 300 seconds, and with [`WORKFLOW_POSTGRES_JOB_LOCK_STALE_SECONDS`](#workflow_postgres_job_lock_stale_seconds) set, any job of this World once its lock has gone unrenewed for that many seconds. - **Streaming**: PostgreSQL NOTIFY/LISTEN enables real-time event distribution. If the database drops the `LISTEN` connection, the World reconnects every second and re-reads persisted chunks for open stream readers, so readers don't miss chunks written during the outage. This architecture ensures workflows survive application restarts with all state reliably persisted. For implementation details, see the [source code](https://github.com/vercel/workflow/tree/main/packages/world-postgres). ## Testing & compatibility <WorldTestingPerformance worldId="postgres" /> ## Limitations - **Reference implementation**: Tested and implements the complete World spec, but is not optimized for scale, speed, or security; a production deployment typically clones it and adapts it, or runs workers in separate processes with a more robust queuing system - **No built-in authentication**: The workflow HTTP routes accept any request that reaches them; you must restrict them yourself (see [Security](#security)) - **No encryption**: Workflow and step data is stored unencrypted; the World does not currently implement [end-to-end encryption](/docs/how-it-works/encryption) - **Requires long-running process**: Must call `start()` on server initialization; not compatible with serverless platforms - **PostgreSQL infrastructure**: Requires a PostgreSQL database (self-hosted or managed) - **Not compatible with Vercel**: Use the [Vercel World](/worlds/vercel) for Vercel deployments For local development, use the [Local World](/worlds/local) which requires no external services.