UNPKG

quelda

Version:

PDF & Illustrator security scanner (WebAssembly): files + policy in, verdict out

491 lines (379 loc) • 23.5 kB
# quelda — PDF & Illustrator security scanner **English** | [日本語](https://github.com/tranhuucanh/quelda/blob/main/packages/quelda-js/README.ja.md) Stops malicious **PDF and Adobe Illustrator (`.ai`) files** right in the browser, before they are uploaded. It never renders or previews the file. | Why quelda | | |---|---| | 🌐 **One engine, browser + server** | Same rules, same verdict here and in the [server version](https://github.com/tranhuucanh/quelda) | | ⚡ **WebAssembly in the browser, near-native speed** | 500 PDF files in 0.82 s, in a Web Worker: the page never freezes, files never leave the machine | | 🛡️ **Tested on real attacks** | ~32,000 real PDF files (malware, exploits, clean documents): 98.9 % of attacks rejected in strict mode, 0 clean files blocked by mistake ([results](#tested-on-real-attacks)) | | 🧱 **No crash on complex PDFs** | Tested on 900,000 objects, and 505,000 objects in 5,000 compressed object streams | | 🪶 **Fast and light** | One 1 GB PDF in 0.8 s with 2.4 MB of engine memory: files are read in place, never loaded whole ([numbers](#-speed-and-memory)) | | 📦 **Self-contained** | No external tools, services or network calls: the engine ships inside the package, with no dependencies | | 🔎 **Clear reasons** | Every rejected file says which rule it broke, where in the file, and why | | 🎚️ **Easy to tune** | Pick the mode that fits you (`strict` · `balanced` · `permissive`), then switch any single rule on or off: `{ rules: { 'EXTERNAL.URI': 'off' } }` ([modes](#-modes-strict-balanced-permissive)) | | 🧩 **Any frontend** | Vite, webpack, esbuild, plain HTML, CDN; React, Vue, Svelte… ([tested in a real browser](#-works-with)) | > **The browser is not a security boundary.** A check in the browser gives users instant > feedback and saves a pointless upload, but anyone can skip it and send a file straight to > your server. **Always scan again on the server** before a file reaches Ghostscript, > ImageMagick or any other PDF processor. quelda has a server version with the same engine and > the same verdicts: see the [project page](https://github.com/tranhuucanh/quelda). ## ⚡ Speed and memory Scan time and engine memory of the WebAssembly engine on an Apple M2 Pro (Node 22). ### One file | File | Size | Time | Engine memory | |---|---:|---:|---:| | Real-world brochure | 54 MB | 0.06 s | 2.4 MB | | Document with images and text | 50 MB | 0.04 s | 2.4 MB | | Document with images and text | 200 MB | 0.14 s | 2.4 MB | | Document with images and text | 1 GB | 0.8 s | 2.4 MB | ### Deeply nested folders The [server version](https://github.com/tranhuucanh/quelda) scans every file in a folder tree, including deeply nested subdirectories in parallel, one per CPU core, and reports each file by its full path; in the browser, files are scanned one at a time. | Folder | Jobs | Time | Peak memory | |---|---:|---:|---:| | 133 real-world PDF and `.ai` files, about 800 MB | 10 (default) | 0.55 s | 343 MB | | 5,820 files in nested folders, 8 GB (real documents and attack test files) | 10 (default) | 6.3 s | 370 MB | ### Heavy files A file is heavy because of its structure, not its size: many objects, deep nesting, page content that must be decompressed. That is exactly what attackers build to crash or stall a scanner. quelda was tested on the heaviest files we could build, far beyond any real document, and still answers in seconds, with bounded memory, without crashing: | Heavy file | Size | Time | Engine memory | |---|---:|---:|---:| | 900,000 objects | 62 MB | 1.9 s | 284 MB | | 505,000 objects in 5,000 compressed object streams | 10 MB | 1.1 s | 246 MB | | 10,000 objects nested 120 levels deep | 18 MB | 0.73 s | 126 MB | | 200,000 objects full of `#xx` escapes | 33 MB | 0.55 s | 112 MB | | 999 incremental updates, 100,000 objects | 7 MB | 0.20 s | 34 MB | | 4 page streams, 480 MiB once decompressed | 53 MB | 3.7 s | 271 MB | | Arrays nested 100,000 levels deep (parser crash attack) | 0.2 MB | < 0.01 s | 1.6 MB | ## 🔍 What quelda checks **56 rules in 11 categories**, plus 12 engine safeguards that refuse to judge what cannot be fully inspected. Full list with explanations: [docs/rules.md](https://github.com/tranhuucanh/quelda/blob/main/docs/rules.md). | Category | Rules | Catches, for example | |---|---|---| | `filetype` | 4 | PostScript/EPS/legacy `.ai` renamed to `.pdf` (even behind a BOM or a printer header), SVG/HTML/MVG/ZIP/executables disguised as PDF, PDF + ZIP/HTML polyglots | | `postscript` | 3 | PostScript code inside a PDF: the `PS` operator, PostScript XObjects, non-standard Type 4 functions | | `active_content` | 7 | JavaScript (anywhere, any encoding, XFA form scripts), Launch actions, actions run on open or on events, links to scripts, local files or network shares | | `embedded` | 4 | Attached files, file attachment annotations, portfolios, Flash/3D/multimedia | | `external` | 7 | Links, remote documents (SMB credential leaks), form submission to a URL, network file references, images read from outside the file (external streams, OPI) | | `forms` | 2 | AcroForm and XFA forms | | `xml` | 2 | DOCTYPE/ENTITY in XMP or XFA (XXE, billion laughs), XInclude and other external references, in UTF-8/16/32 | | `structure` | 14 | Tampered or ambiguous structure: hidden revisions, conflicting objects, objects that cannot be parsed, mismatched xref, page-tree and bookmark loops, data after `%%EOF`, multiple headers, oversized names, content that prints but is hidden on screen, files changed after signing | | `encryption` | 1 | Encrypted files whose content cannot be inspected | | `codecs` | 7 | JBIG2 (FORCEDENTRY), JPEG 2000, Type 1 fonts, out-of-spec decoder parameters, malformed fonts and ICC colour profiles | | `resource` | 5 | Pixel bombs, gigantic or broken pages, page-count bombs, drawing loops, huge colour function tables | ### Tested on real attacks | Dataset | From | Contents | |---|---|---| | [CIC-Evasive-PDFMal2022](https://www.unb.ca/cic/datasets/pdfmal-2022.html) | Canadian Institute for Cybersecurity | Real malware and clean documents | | [PDF Insecurity](https://pdf-insecurity.org/) (incl. PDF101) | Ruhr University Bochum | Attacks on PDF readers, signatures and encryption | | ~32,000 files | balanced (default) | strict | |---|---|---| | 🔴 Attack files | ✅ 95.4 % rejected | ✅ 98.9 % rejected | | 🟢 Clean files | ✅ 0 rejected by a rule mistake | | - **Clean files rejected on purpose:** those that carry JavaScript, attachments or encryption, which balanced blocks. Relax that one rule if your users need them. - **Attacks still accepted:** mostly phishing documents that are only a link (quelda cannot judge a URL) and signed files changed after signing (a warning in balanced). Every rule is also tested on generated files: attack variants, clean look-alikes that must not trigger it, damaged and disguised files. The test files are not published, because antivirus software may flag them; if you need them, email the maintainer (tranhuucanh39@gmail.com). ## 🎚 Modes: strict, balanced, permissive **Every rule runs in every mode.** A mode (preset) only decides what a finding does: **block** (the file is rejected), **warn** (reported, the file is accepted) or **off**. | Category | strict | balanced (default) | permissive | |---|---|---|---| | `filetype`, `postscript`, `encryption` | block | block | block | | `active_content`, `embedded`, `xml`, `resource` | block | block | warn | | `external`, `forms`, `structure`, `codecs` | block | warn | off | - **strict**: files from anyone go straight into Ghostscript, ImageMagick or a print RIP. Rejects anything unusual, including links, forms and damaged structure. - **balanced**: blocks everything that can execute code, carry a payload or overload a server; only warns about features common in normal documents (web links, form fields, edits). - **permissive**: files from sources you mostly trust. Locked rules (PostScript, Launch actions, files that are not PDF) block in every mode. Start from a mode, then change single rules: `{ "preset": "strict", "rules": { "EXTERNAL.URI": "warn" } }`. A few rules differ from their category; the action of every rule in every mode is in [rules.md](https://github.com/tranhuucanh/quelda/blob/main/docs/rules.md), and how to write a policy in [policy.md](https://github.com/tranhuucanh/quelda/blob/main/docs/policy.md). ## 📚 Documentation | Guide | Read it to | |---|---| | [**Rules**](https://github.com/tranhuucanh/quelda/blob/main/docs/rules.md) | See every rule: what it catches, and whether it blocks, warns or is off in each preset | | [**Policies**](https://github.com/tranhuucanh/quelda/blob/main/docs/policy.md) | Choose a preset, write your own policy, relax or tighten one rule, raise limits | | [**Server version**](https://github.com/tranhuucanh/quelda) | Scan the same files again on your server, with the same verdicts | ## 📦 Install ```bash npm install quelda ``` Or no install at all: load it from a CDN ([below](#no-build-step-cdn)). ## 🚀 Quick start ```ts import { createScanner } from 'quelda'; const scanner = await createScanner(); // starts a worker, loads the engine once const verdict = await scanner.scan(file, { preset: 'balanced' }); if (verdict.valid) { upload(file); } else { showError(verdict.violations.map((v) => v.message).join('\n')); } ``` Treat **anything but `verdict.valid === true` as invalid**. A verdict looks like this (shortened): ```json { "valid": false, "status": "REJECT", "violations": [ { "rule_id": "ACTIVE.JAVASCRIPT", "action": "block", "message": "Document contains JavaScript" } ], "warnings": [], "errors": [] } ``` `status` is `PASS` (valid), `REJECT` (a rule set to `block` fired) or `ERROR` (the file could not be fully inspected: damaged, over a limit, or the scan failed). Full format: [`verdict.schema.json`](https://github.com/tranhuucanh/quelda/blob/main/schemas/verdict.schema.json). ### A file input ```html <input type="file" id="upload" accept=".pdf,.ai,application/pdf" /> <p id="result"></p> <script type="module"> import { createScanner } from 'quelda'; const scanner = await createScanner(); document.getElementById('upload').addEventListener('change', async (event) => { const file = event.target.files[0]; if (!file) return; const verdict = await scanner.scan(file); document.getElementById('result').textContent = verdict.valid ? 'OK, uploading…' : `Refused: ${[...verdict.violations, ...verdict.errors].map((r) => r.message).join('; ')}`; }); </script> ``` (With a bundler, put the script in your app code. Without one, see [Works with](#-works-with).) ### Deeply nested folders `webkitdirectory` lets the user pick a folder; every file inside it, at any depth, comes with its path in `webkitRelativePath`: ```html <input type="file" id="folder" webkitdirectory multiple /> <ul id="results"></ul> <script type="module"> import { createScanner } from 'quelda'; const scanner = await createScanner(); document.getElementById('folder').addEventListener('change', async (event) => { const files = [...event.target.files] .filter((f) => /\.(pdf|ai)$/i.test(f.name)) .sort((a, b) => a.webkitRelativePath.localeCompare(b.webkitRelativePath)); const list = document.getElementById('results'); list.replaceChildren(); for (const file of files) { const verdict = await scanner.scan(file); // one at a time; the page stays responsive const item = document.createElement('li'); item.textContent = `${verdict.status} ${file.webkitRelativePath}`; list.append(item); } }); </script> ``` Only upload the folder if every verdict is valid. Scans run one after another in the worker (see [API](#scannerscaninput-policy-options)). ### React One scanner for the component's lifetime; it loads the engine once and scans files one after another. ```tsx import { useEffect, useRef, useState, type ChangeEvent } from 'react'; import { createScanner, type Scanner, type Verdict } from 'quelda'; export function PdfUpload({ onAccepted }: { onAccepted: (file: File) => void }) { const scanner = useRef<Promise<Scanner> | null>(null); const [verdict, setVerdict] = useState<Verdict | null>(null); useEffect(() => { const started = createScanner(); scanner.current = started; return () => { started.then((s) => s.terminate(), () => {}); }; }, []); async function onChange(e: ChangeEvent<HTMLInputElement>) { const file = e.target.files?.[0]; if (!file || !scanner.current) return; const v = await (await scanner.current).scan(file, { preset: 'balanced' }); setVerdict(v); if (v.valid) onAccepted(file); } return ( <> <input type="file" accept=".pdf,.ai,application/pdf" onChange={onChange} /> {verdict && !verdict.valid && ( <ul> {[...verdict.violations, ...verdict.errors].map((r, i) => ( <li key={i}>{r.message}</li> ))} </ul> )} </> ); } ``` Vue, Svelte, Angular and others work the same way: create the scanner once (on mount), call `scan(file)` on change, `terminate()` on unmount. ### No build step (CDN) ```html <script type="module"> import { createScanner } from 'https://cdn.jsdelivr.net/npm/quelda/dist/index.js'; const scanner = await createScanner(); // … same as above </script> ``` Public npm CDNs such as [jsDelivr](https://www.jsdelivr.com/package/npm/quelda) and [unpkg](https://unpkg.com/quelda/) serve every package published to npm automatically: nothing to set up, the URL works as soon as a version is on npm. Pin a version in production (`…/npm/quelda@X.Y.Z/dist/index.js`, the version from `npm view quelda version`) so a new release cannot change your verdicts unannounced. The package starts its worker from the CDN by itself. To serve the files from your own server instead, see [No bundler](#-works-with). ## 🧩 Works with The package is plain ESM. It ships the worker (`dist/worker.js`) and the engine (`dist/wasm/quelda_bg.wasm`, a separate file, not inlined) next to the API and finds them with `new URL(..., import.meta.url)`. Each setup below is built and run in headless Chrome on every release (clean PDF must pass, PostScript must be rejected, through the real worker and engine): | Setup | What to do | |---|---| | **Vite** | `optimizeDeps: { exclude: ['quelda'] }` and `worker: { format: 'es' }` in `vite.config.js` (below) | | **webpack 5** | nothing | | **esbuild** | bundle as usual (`--format=esm`), then copy `node_modules/quelda/dist/` next to the bundle | | **No bundler** | copy `node_modules/quelda/dist/` to your site (e.g. `/quelda/`) and `import { createScanner } from './quelda/index.js'` | | **Import map** | as above, plus `<script type="importmap">{ "imports": { "quelda": "./quelda/index.js" } }</script>` so `import … from 'quelda'` works | | **CDN** | [see above](#no-build-step-cdn) | ```js // vite.config.js: pre-bundling breaks new URL(..., import.meta.url) in dev mode export default { optimizeDeps: { exclude: ['quelda'] }, worker: { format: 'es' } }; ``` Any other bundler: if it does not copy `worker.js` and the `.wasm` file by itself, copy `dist/` next to your bundle as with esbuild, or tell the package where they are: ```ts const scanner = await createScanner({ wasm: '/static/quelda_bg.wasm', worker: () => new Worker('/static/quelda/worker.js', { type: 'module' }), }); ``` Working projects: [Vite](https://github.com/tranhuucanh/quelda/tree/main/packages/quelda-js/examples/vite), [webpack](https://github.com/tranhuucanh/quelda/tree/main/packages/quelda-js/examples/webpack). Serve `.wasm` files as `application/wasm` for the fastest load. **Content Security Policy:** WebAssembly needs `script-src 'wasm-unsafe-eval'`. The worker needs `worker-src 'self'` (or wherever you serve it); from a CDN, `worker-src blob:`, and the CDN in `script-src` and `connect-src`. **Browsers:** module workers and WebAssembly SIMD: Chrome/Edge 91+, Firefox 114+, Safari 16.4+. ## 📄 Supported files Files are identified by content, not by extension. **PDF** and **modern `.ai` files** (Illustrator 9 and later, which are PDF inside) are scanned. **PostScript, EPS and legacy `.ai`** (Illustrator 8 and earlier) are always rejected with `FILETYPE.POSTSCRIPT_MAGIC`, a locked rule: PostScript is a programming language and cannot be proven safe by a scan. ## 🔌 API ### `createScanner(options?)` Starts the worker and resolves once the engine is loaded. Rejects with a `QueldaError` (`code`, `message`) if it cannot start (Web Workers unavailable, `.wasm` not found, CSP, …). | Option | Default | | |---|---|---| | `timeoutMs` | `30000` | Per scan / `listRules` / worker start. `Infinity` disables it. | | `wasm` | the bundled `.wasm` | A URL (string or `URL`, resolved against the page) or a compiled `WebAssembly.Module`. | | `worker` | the bundled worker | Advanced: a function returning a `Worker` that runs `quelda/worker.js`. | ### `scanner.scan(input, policy?, options?)` - `input`: `File`, `Blob`, `ArrayBuffer`, or any typed array / `DataView` (`Uint8Array`, …). Only the bytes of a view are scanned. - `policy`: a policy object, or policy JSON text. Default `{}` = the `balanced` preset. See [Policy](#-policy). - `options.timeoutMs`: overrides the scanner's timeout for this scan. - `options.signal`: an `AbortSignal`. - `options.transfer`: see [Memory](#-memory-and-transfer). **`scan` never rejects.** Every failure becomes a verdict with `status: "ERROR"` and `valid: false` (fail-closed): | `errors[].code` | Made by | When | |---|---|---| | `LIMIT.*`, `ENGINE.*`, `POLICY.INVALID` | engine | a limit was exceeded, the file could not be inspected, the policy is invalid | | `LIMIT.TIMEOUT` | package | the scan took longer than `timeoutMs`; the worker was killed | | `ENGINE.ABORTED` | package | the `signal` fired, or `terminate()` was called | | `ENGINE.WORKER` | package | the worker or the engine crashed, failed to load, or replied with something malformed | | `ENGINE.INPUT` | package | the input is not a supported type, the file could not be read, or an option is invalid | | `ENGINE.READ` | engine | the `File` / `Blob` could not be read while scanning (e.g. it changed or was deleted) | | `POLICY.INVALID` | package | the policy object cannot be turned into JSON (cycle, `BigInt`, …) | The engine has no clock in WebAssembly, so the package fills `stats.duration_ms` with the time the engine call took, measured in the worker (whole milliseconds; queueing and file reading are not included). Scans run **one at a time**, in call order (WebAssembly is single-threaded). The timeout of a scan starts when the scan starts, not when it is queued. A timeout, an abort of a running scan, or a crash kills the worker; the next scan starts a fresh one. A worker whose engine threw is never reused, nor one whose WebAssembly memory grew past 64 MiB (it never shrinks; a fresh worker returns it to the system). ### `scanner.listRules(options?)` Every rule the engine knows (`id`, `category`, `severity`, `default_action`, `locked`, `stage`, `since`, `message`, `description`; see [`rules.schema.json`](https://github.com/tranhuucanh/quelda/blob/main/schemas/rules.schema.json)), sorted by `id`. Handy for a settings screen. Rejects with a `QueldaError`. `scanner.engineVersion` is the engine version. ### `scanner.terminate()` Kills the worker. Pending scans resolve with `ENGINE.ABORTED`; later calls fail. Safe to call twice. ## 📋 Policy **Every rule always runs**: a policy starts from a preset and changes the action (`block`, `warn`, `off`) of the categories and rules you name; everything else keeps its preset action. Full guide (presets, recipes, limits, engine errors): [policy.md](https://github.com/tranhuucanh/quelda/blob/main/docs/policy.md). Three levels; the most specific wins: **preset → categories → rules** ([`policy.schema.json`](https://github.com/tranhuucanh/quelda/blob/main/schemas/policy.schema.json)). ```ts await scanner.scan(file, { preset: 'balanced', // strict | balanced | permissive categories: { external: 'warn', codecs: 'off' }, // block | warn | off rules: { 'ACTIVE.OPEN_ACTION': 'off' }, limits: { max_file_bytes: 50 * 1024 * 1024 }, }); ``` What each preset does: [Modes](#-modes-strict-balanced-permissive). Defaults are strict on purpose. If a rule blocks files your users legitimately upload, relax **that rule** rather than the whole category or preset. For example, `EMBED.EMBEDDED_FILE` blocks files that carry attachments (some e-books and reports do): ```ts { rules: { 'EMBED.EMBEDDED_FILE': 'warn' } } // still reported in verdict.warnings ``` Locked rules (`locked: true` in `listRules()`: PostScript, Launch actions, files that are not PDF, …) stay `block` whatever the policy says, unless the policy also sets `unsafe_allow_unlock: true` (reported in `verdict.policy`; do not do this for untrusted files). The engine parses the policy **strictly**: unknown fields, `null`, wrong types, unknown categories and **rule IDs this engine version does not know** give an `ERROR` verdict (`POLICY.INVALID`) instead of being ignored, because a silently ignored typo could switch a rule off. ### Keep the browser and the server on the same rule set A policy written for a newer engine (naming a rule an older engine does not have) makes every scan of the older engine `ERROR`. Either use the same quelda version in the browser and on the server, or drop the rules the engine does not know before scanning: ```ts const known = new Set((await scanner.listRules()).map((r) => r.id)); const policy = { ...serverPolicy, rules: Object.fromEntries( Object.entries(serverPolicy.rules ?? {}).filter(([id]) => known.has(id))) }; ``` (The server must keep using the full policy.) ## 🧠 Memory and transfer Prefer passing the `File` / `Blob` itself (limit: `limits.max_file_bytes`, default 256 MiB). - `File` / `Blob` (recommended): **never loaded whole.** The worker reads it in place with `FileReaderSync`: one pass in 1 MiB blocks, then only the parts the engine must parse or decode (images and other opaque data are not read again; fonts and colour profiles are read once more to check their structure). The engine's memory stays a few MiB whatever the file size, plus the data it must decompress. Your object is untouched; the file must not change until the scan resolves (if it does: `ENGINE.READ`). - `ArrayBuffer` / typed array: **copied when `scan` is called**, then copied whole into the engine's memory; changing your buffer afterwards does not affect the scan. - `transfer: true`: hands your `ArrayBuffer` (or the buffer of a typed array that spans it whole) to the worker without a copy. **Your buffer is detached (length 0) as soon as `scan` is called.** A view of part of a buffer is still copied. ## 💻 Node The package targets browsers. Node (≥ 20) has no Web Workers, so `createScanner()` does not work there out of the box; on a server, use the server version from the [project page](https://github.com/tranhuucanh/quelda). ## 📜 License [MIT](https://github.com/tranhuucanh/quelda/blob/main/LICENSE). Security issues: please report them privately (see [SECURITY.md](https://github.com/tranhuucanh/quelda/blob/main/SECURITY.md)).