UNPKG

@wdio/visual-service

Version:

Image comparison / visual regression testing for WebdriverIO

1,143 lines (709 loc) 77.3 kB
# @wdio/visual-service ## 10.1.0 ### Minor Changes - b194642: fix: ignore\* option parity with resemble (pixelmatch) After v10 switched to pixelmatch, the public `ignore*` API did not fully match resemble.js preset behaviour. Combined modes such as `ignoreLess` with the default `ignoreAntialiasing: true` still inherited AA forgiveness, and `ignoreColors` used BT.601 grayscale instead of resemble brightness-only comparison. This release also adds `compareOptions.pixelmatch` so you can pass pixelmatch settings directly instead of using `ignore*` presets. **What changed** - Multiple `ignore*` flags now follow resemble last-wins ordering (`ignoreAlpha` → `ignoreAntialiasing` → `ignoreColors` → `ignoreLess` → `ignoreNothing`) instead of composing independently - `ignoreLess`, `ignoreAlpha`, `ignoreColors`, and `ignoreNothing` now apply their own threshold and AA rules when active; they no longer inherit default `ignoreAntialiasing: true` forgiveness - `ignoreColors` now compares brightness only using resemble luma weights (`0.3/0.59/0.11`), matching resemble v9 behaviour - WDIO logs a warning when multiple `ignore*` flags are enabled, naming which preset wins - New `compareOptions.pixelmatch` object for direct pixelmatch control (`threshold`, `includeAA`, `diffColor`, `aaColor`, `diffColorAlt`, `alpha`, `diffMask`, `checkerboard`) **Preset reference** | Active preset | threshold | AA forgiven | | ------------------------------ | --------- | -------------------- | | `ignoreNothing` | 0 | no | | `ignoreLess` | ~16/255 | no | | `ignoreColors` | ~16/255 | no (brightness only) | | `ignoreAlpha` | ~16/255 | no | | `ignoreAntialiasing` (default) | ~32/255 | yes | **Using `compareOptions.pixelmatch`** Set it in your service config or on a single `check*` call. Do not put `ignore*` keys and `pixelmatch` on the same options object; that throws, even when an `ignore*` flag is `false`. Service config and method options are separate objects, so a method call can override the service compare mode for that check (a warning is logged when the mode switches). Service config: ```js // wdio.conf.js services: [ [ "visual", { compareOptions: { pixelmatch: { threshold: 0.063, includeAA: true, }, }, }, ], ]; ``` Method override when the service uses `ignore*` presets: ```js await browser.checkScreen("homepage", { pixelmatch: { threshold: 0.05 }, }); ``` Method override when the service uses `pixelmatch`: ```js await browser.checkScreen("homepage", { ignoreLess: true, }); ``` Invalid (throws): ```js compareOptions: { ignoreLess: false, pixelmatch: { threshold: 0.063 }, } ``` See [pixelmatch](https://github.com/mapbox/pixelmatch) for option details. **What you need to do** - No change needed if you use a single `ignore*` flag or rely on defaults (`ignoreAntialiasing: true`) - Set `ignoreAntialiasing: false` when anti-aliased pixels should count as differences - If you combine multiple `ignore*` flags, review your tests; last-wins ordering now matches resemble v9 - If you use `ignoreColors`, results may differ slightly from early v10 but align with resemble v9 - To tune pixelmatch directly, add `compareOptions.pixelmatch` in your service config or pass `pixelmatch` on individual `check*` calls ### Patch Changes - b194642: chore: dependency updates Updated dependencies to their latest compatible versions: - `@wdio/visual-service`: `expect-webdriverio` to `^5.7.0` - `@wdio/visual-reporter`: `sharp` to `^0.35.3` - Dev tooling: `@typescript-eslint/*` to `^8.63.0`, `vitest` to `^3.2.7`, `eslint` to `^9.39.5`, plus minor bumps for `postcss`, `react-icons`, and `isbot` in the reporter package No functional or API changes. ### Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [b194642] - @wdio/image-comparison-core@2.1.0 ## 10.0.1 ### Patch Changes - 8cbb294: fix: wire ignoreAntialiasing to pixelmatch AA forgiveness toggle In v10 the pixelmatch engine always ran with anti-aliasing forgiveness enabled, even when `ignoreAntialiasing` was `false`. The option had no effect on comparison behaviour. **What changed** - `ignoreAntialiasing` now toggles pixelmatch's AA handling: `true` forgives anti-aliased pixels, `false` counts them as mismatches. - The default is now `ignoreAntialiasing: true`, matching the forgiving pixelmatch behaviour users already get in v10. - `ignoreLess` and `ignoreNothing` keep their own threshold behaviour; AA forgiveness is controlled independently via `ignoreAntialiasing`. **Migration** - No action needed if you rely on the current forgiving defaults, comparison behaviour stays the same. - If you explicitly set `ignoreAntialiasing: true` today, that remains redundant but harmless. - Set `ignoreAntialiasing: false` when you need strict comparison where anti-aliased pixels count as differences. ### Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [8cbb294] - @wdio/image-comparison-core@2.0.1 ## 10.0.0 ### Major Changes - d2758ce: ### 💥 Breaking change: new image comparison engine We replaced the engine that powers every visual comparison. This is a breaking change, so please read the migration note below before upgrading. **The problem** Visual tests were flaky. Tests failed on differences that are impossible to see by eye, like sub-pixel font rendering, 1px anti-aliasing on edges and small shadow shifts between runs. The old engine (resemble.js) compared raw RGB values, which does not match how human vision works, and on larger screenshots it quietly skipped about a third of the pixels. So you got failures that were not real, and in some cases real changes that could slip through. On top of that, all the image handling (decode, crop, composite, rotate, resize) ran through [`jimp`](https://github.com/jimp-dev/jimp), a large dependency that we only used a small slice of and that is no longer actively maintained. **The solution** Two things changed under the hood: 1. The comparison engine is now [pixelmatch](https://github.com/mapbox/pixelmatch). It compares images the way the eye perceives them (in the YIQ colour space) and detects anti-aliasing by checking both images at once. Invisible rendering noise now passes, and real regressions still fail. 2. `jimp` has been removed completely. PNG decode and encode now go through the small [`fast-png`](https://github.com/image-js/fast-png) library, and the handful of image operations we still need (crop, composite, canvas, opacity, rotate, resize) live in a tiny internal helper. The bundled resemble file is gone too. The net effect is a much lighter dependency footprint with no loss in functionality. **What you need to do** - Your public API does not change. `checkScreen`, `checkElement`, `checkFullPageScreen` and the matchers all work exactly as before, and the same `ignore` options are supported. - Because the new engine measures differences differently, mismatch percentages will not match the old numbers exactly. You should re-run your suite once and re-accept your baselines so they are generated with the new engine. After that your tests should be noticeably more stable. **Also fixed in this release** - **Top-row artifact on full page screenshots:** Jimp's `contain()` centred the image, shifting content by 1px and creating a false diff across the top row. Replaced with buffer-level padding that anchors content at (0,0). - **Ignored region 1px under-coverage:** The device-pixel size of an ignored region used `Math.floor`, which could drop a pixel when `cssSize * DPR` had a fractional part. Width and height now use `Math.ceil` so the full element is always covered. Position still uses `Math.floor`. - **Comparison sensitivity matches what you were used to:** Switching engines meant retuning how strict a comparison is. The pixelmatch threshold is now aligned with the old resemble tolerances, so a difference that used to fail still fails and one that used to pass still passes. The diff highlight also uses a single consistent colour instead of varying per run. - **Different image sizes no longer crash the comparison:** When a baseline and the actual screenshot had slightly different dimensions, the old flow threw an error and you lost the result. Both images are now normalised to the same size before they are compared, so a size change is reported as a visual difference you can review instead of a hard failure. - **More reliable ignore regions with WebDriver BiDi:** With BiDi the calculated element bounds can be off by a pixel or two, which sometimes left part of an ignored element just outside the ignored area and caused a false diff. The BiDi emulated flow now uses a larger `ignoreRegionPadding` so the whole element stays covered. ### Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ### Patch Changes - 6cd5742: ### Dependency updates Updated dependencies across all packages to their latest compatible versions. This includes the WebdriverIO toolchain (`webdriverio`, `@wdio/*`) to `9.29.1`, the TypeScript ESLint plugins to `8.62.0`, Vitest to `3.2.6`, and various other packages such as `sharp`, `@remix-run/*`, `fuse.js` and `expect-webdriverio`. There are no functional or API changes. ### Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [d2758ce] - Updated dependencies [6cd5742] - @wdio/image-comparison-core@2.0.0 ## 9.2.4 ### Patch Changes - 60997df: fix: prevent false emulation detection when checkElement is called inside an iframe after switchFrame ### Committers: 1 - Taro.Nonoyama([@n2-freevas](https://github.com/n2-freevas)) - Updated dependencies [60997df] - @wdio/image-comparison-core@1.2.4 ## 9.2.3 ### Patch Changes - c56e1ae: ## #1146 Fix BiDi element screenshots missing composited layers (scrollbars, fixed/sticky overlays) ### Root cause When `checkElement` / `saveElement` is used with the WebDriver BiDi protocol, the screenshot was taken with `browsingContext.captureScreenshot` using `origin: 'document'`. This renders the document layout independently of the browser's compositor, which means **composited layers are never included** — element-level scrollbars, `position: fixed` / `position: sticky` overlays, and elements with a `will-change` CSS property all render as invisible or without their correct visual state. The switch to `origin: 'document'` was introduced in an earlier fix (commit `227f10a`) to avoid a `zero dimensions` error that occurred when `origin: 'viewport'` was used for elements that were outside the visible viewport. That fix was correct for out-of-viewport elements, but it also silently broke composited-layer capture for all elements. ### Fix: new `biDiOrigin` method option A new **method-level** option `biDiOrigin` has been added to `saveElement` / `checkElement`. It is BiDi-only and ignored for the legacy WebDriver screenshot path. | Value | Behaviour | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `'document'` _(default)_ | Previous behaviour — works for any element position but composited layers (scrollbars, overlays, `will-change`) are not captured | | `'viewport'` | Captures the composited frame as the browser painted it — scrollbars, fixed/sticky overlays and `will-change` layers are included. The element must be visible in the viewport; descriptive errors are thrown when it is not | #### Usage ```ts // Capture an element with its scrollbar / overlay visible: await browser.checkElement(element, "myTag", { biDiOrigin: "viewport" }); await browser.saveElement(element, "myTag", { biDiOrigin: "viewport" }); ``` #### Error messages when `biDiOrigin: 'viewport'` cannot produce a valid screenshot **Element larger than the viewport** — must fall back to `'document'`: ``` [BiDi viewport screenshot] The element dimensions (1400x800px) exceed the viewport (1280x720px). You must use the default `biDiOrigin: 'document'` for this element. Note: with `'document'` origin, composited layers such as scrollbars, fixed/sticky overlays, and elements using `will-change` may not appear in the screenshot. ``` **Element not in the viewport at all** — needs scrolling: ``` [BiDi viewport screenshot] The element is not in the viewport (element: x=0, y=900, 300x200px; viewport: 1280x720px). Call `element.scrollIntoView()` before taking the screenshot, or set `autoElementScroll: true`. ``` **Element partially outside the viewport but fits** — needs to be scrolled fully into view: ``` [BiDi viewport screenshot] The element is not fully visible in the viewport (element: x=-20, y=100, 300x200px; viewport: 1280x720px). The element fits within the viewport — scroll it fully into view by calling `element.scrollIntoView()` or setting `autoElementScroll: true`. ``` ### Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [c56e1ae] - @wdio/image-comparison-core@1.2.3 ## 9.2.2 ### Patch Changes - db33fa7: #### `@wdio/image-comparison-core` and `@wdio/ocr-service` Security: update jimp (CVE in `file-type` transitive dep) Bumped `jimp` to the latest version to resolve a reported vulnerability in its `file-type` transitive dependency (see [#1130](https://github.com/webdriverio/visual-testing/issues/1130), raised by [@denis-sokolov](https://github.com/denis-sokolov), thank you!). **Actual impact on these packages** `file-type` is used by `@jimp/core` solely to detect image MIME types when reading a buffer. In both `@wdio/image-comparison-core` and `@wdio/ocr-service`, every image passed to jimp originates from either WebDriver screenshots (browser-controlled base64 data) or local files written by the framework itself. There is no code path where untrusted external input is fed directly into jimp, which removes the exploitability that the CVE describes. That said, the reputational and compliance risk was real, security scanners flag the package as vulnerable, enterprise users hit audit failures, and some organisations block installation of packages with known CVEs. The update addresses all of that. #### `@wdio/visual-reporter` and `@wdio/visual-service` Updated internal dependencies to pick up the jimp bump in `@wdio/image-comparison-core`. ### Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [db33fa7] - @wdio/image-comparison-core@1.2.2 ## 9.2.1 ### Patch Changes - d5afb54: ## #1129 Fix `TypeError: element.getBoundingClientRect is not a function` when a `ChainablePromiseElement` is passed to `checkElement` When `checkElement` (or `saveElement`) was called with a `ChainablePromiseElement`, the lazy promise-based element reference that WebdriverIO's `$()` returns, the element was passed directly as an argument to `browser.execute()` without being awaited first. `browser.execute()` serializes its arguments for transfer to the browser context and cannot handle a pending Promise, so it arrived in the browser as a plain empty object `{}` instead of a WebElement reference. This caused `element.getBoundingClientRect is not a function` because the browser-side `scrollElementIntoView` script received `{}` rather than a DOM element. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [d5afb54] - @wdio/image-comparison-core@1.2.1 ## 9.2.0 ### Minor Changes - 994f4da: ## #857 Support ignore regions for web screenshots Add `ignore` support to all web screenshot methods (`saveScreen`/`checkScreen`, `saveElement`/`checkElement`, `saveFullPageScreen`/`checkFullPageScreen`) so that specified elements can be blocked out during visual comparison. This brings web parity with the native-app ignore-region support that already existed. ### Changes - **Ignore regions for full-page screenshots:** new `determineWebFullPageIgnoreRegions` function that calculates ignore-region rectangles for full-page screenshots, including a `fullPageCropTopPaddingCSS` correction for mobile scroll-and-stitch scenarios where the address-bar shadow padding shifts element positions - **Consolidated `ignoreRegionPadding`:** moved `ignoreRegionPadding` into `BaseWebScreenshotOptions` so it is inherited by all web methods instead of being duplicated per method - **Fix `isAndroidNativeWebScreenshot` type:** ensure `nativeWebScreenshot` is always a boolean (was accidentally an object for LambdaTest capabilities), preventing ignore-region DPR scaling failures - **Fix viewport rounding for mobile:** restore `Math.round()` in `injectWebviewOverlay` and remove `Math.min` clamping in `getMobileViewPortPosition` to prevent 1-pixel crop shifts during full-page stitching - **Fix `scrollElementIntoView` for scrolled pages:** account for `currentPosition` (existing scroll offset) when computing the target scroll position, so elements are scrolled into view correctly when the page is already scrolled - **Dismiss Chrome Start Surface on Android:** when Chrome's tab-overview UI blocks the webview overlay, automatically press the Android Back button (up to 4 retries) to restore the active tab before measuring the viewport - **Add hybrid status bar blockout:** on hybrid apps the statusbar was not blocked out which could result in flaky tests regarding battery and reception # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ### Patch Changes - Updated dependencies [994f4da] - @wdio/image-comparison-core@1.2.0 ## 9.1.6 ### Patch Changes - 0a19d78: Fix `clearRuntimeFolder` clearing the actual and diff folders after each spec/feature execution instead of once before all workers start. This caused only the last spec's visual data to be present in the output when running multiple specs. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - ed0bea6: Fix `EISDIR` error when using `resolveSnapshotPath` with the visual service. The service now uses `dirname()` of the resolved path as the baseline folder, preventing it from creating a directory at a path that `expect-webdriverio`'s snapshot service expects to be a file. Fixes #984. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - cbf1d22: Fix incomplete `wdio-ics:options` type augmentation on `WebdriverIO.Capabilities`. The global type declaration now uses the `WdioIcsOptions` interface directly, ensuring all supported properties (`logName`, `name`) are available to TypeScript users in both standalone and multiremote configurations. Fixes #732. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [0a19d78] - Updated dependencies [ce74703] - @wdio/image-comparison-core@1.1.4 ## 9.1.5 ### Patch Changes - 6ed0469: ## Fix: support `appium:options` nested capability format and `avd` fallback (#1118) Appium caps need to be prefixed with `appium:`, but this can feel redundant when you have a lot of caps. So you can also put them inside the `appium:options`-object. This was not supported by the visual module and was reported in #1118. It is now supported. The following capabilities are now correctly read from both `appium:`-prefixed top-level format and the nested `appium:options` format: - `deviceName` - `nativeWebScreenshot` - `avd` (new, see below) Second issue that is fixed is that for Android the `deviceName` could be left away and the `avd` could be provided. This is now also supported where `deviceName` takes priority over `avd` if both are provided. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ## 9.1.4 ### Patch Changes - a3bc7a4: ## #1115 Respect `alwaysSaveActualImage: false` for `checkScreen` methods When using visual matchers like `toMatchScreenSnapshot('tag', 0.9)` with `alwaysSaveActualImage: false`, the actual image was still being saved even when the comparison passed within the threshold. The root cause was that the matcher's expected threshold was not being passed to the core comparison logic. The core used `saveAboveTolerance` (defaulting to 0) to decide whether to save images, while the matcher used the user-provided threshold to determine pass/fail - these were disconnected. This fix ensures: - When `alwaysSaveActualImage: false` and `saveAboveTolerance` is not explicitly set, actual images are never saved (respecting the literal meaning of the option) - When `saveAboveTolerance` is explicitly set (like matchers do internally), actual images are saved only when the mismatch exceeds that threshold # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - a3bc7a4: ## Fix: `save*` methods now always save files regardless of `alwaysSaveActualImage` setting Previously, when `alwaysSaveActualImage: false` was set in the configuration, `save*` methods (`saveScreen`, `saveElement`, `saveFullPageScreen`, `saveAppScreen`, `saveAppElement`) were not saving files to disk, causing test failures. The `alwaysSaveActualImage` option is intended to control whether actual images are saved during `check*` methods (comparison operations), not `save*` methods. Since `save*` methods are explicitly designed to save screenshots, they should always save files regardless of this setting. This fix ensures: - `save*` methods always save files to disk, even when `alwaysSaveActualImage: false` is set in the config - `alwaysSaveActualImage: false` continues to work correctly for `check*` methods (as intended for issue #1115) - The behavior is now consistent: `save*` = always save, `check*` = respect `alwaysSaveActualImage` setting **Implementation details:** - The visual service overrides `alwaysSaveActualImage: true` when calling `save*` methods directly from the browser API - `save*` methods respect whatever `alwaysSaveActualImage` value is passed to them (no special logic needed) - `check*` methods pass through the config value (which may be `false`), so `save*` methods respect it when called internally - This clean separation ensures `save*` methods work correctly when called directly while still respecting `alwaysSaveActualImage` for `check*` methods # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [a3bc7a4] - Updated dependencies [a3bc7a4] - @wdio/image-comparison-core@1.1.3 ## 9.1.3 ### Patch Changes - 2a518ff: # 🐛 Bugfixes ## #1111 Pass matcher threshold to core as saveAboveTolerance When using visual matchers like `toMatchScreenSnapshot('tag', 0.9)` with `alwaysSaveActualImage: false`, the actual image was still being saved even when the comparison passed within the threshold. The root cause was that the matcher's expected threshold was not being passed to the core comparison logic. The core used `saveAboveTolerance` (defaulting to 0) to decide whether to save images, while the matcher used the user-provided threshold to determine pass/fail - these were disconnected. This fix ensures the matcher passes the expected threshold to the core as `saveAboveTolerance`, so images are only saved when the mismatch actually exceeds the user's acceptable threshold. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ## 9.1.2 ### Patch Changes - 0a2b6d0: ## #1111 Respect saveAboveTolerance when deciding to save actual images when alwaysSaveActualImage is false. When `alwaysSaveActualImage` is `false`, the actual image is no longer written to disk if the mismatch is below the configured tolerance, avoiding extra actuals when the comparison still passes. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [0a2b6d0] - @wdio/image-comparison-core@1.1.2 ## 9.1.1 ### Patch Changes - 340fbe6: # 🐛 Bugfixes ## #1098 Improve error message when baseline is missing and both flags are false When `autoSaveBaseline = false` and `alwaysSaveActualImage = false` and a baseline image doesn't exist, the error message now provides clear guidance suggesting users set `alwaysSaveActualImage` to `true` if they need the actual image to create a baseline manually. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - e4e5b5c: # 🐛 Bugfixes ## #1085 autoSaveBaseline collides with the new alwaysSaveActualImage flag When `autoSaveBaseline` is `true` and `alwaysSaveActualImage` is `false`, actual images were still saved. This patch should fix that # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - ddf68fe: # 🐛 Bugfixes ## #1084 expect(...).toMatch\*Snapshot methods do not have Promise return types The methods should be typed as promises # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [340fbe6] - Updated dependencies [e4e5b5c] - @wdio/image-comparison-core@1.1.1 ## 9.1.0 ### Minor Changes - bde4851: This PR will implement FR #1077 which is asking not to create the actual image on success. This should create a better performance because no files are writing to the system and should make sure that there's not a lot of noise in the actual folder. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ### Patch Changes - Updated dependencies [bde4851] - @wdio/image-comparison-core@1.1.0 ## 9.0.2 ### Patch Changes - 8ff1bc3: # 🐛 BugFix ## #1078: Cursor inside shadow is shown, even with disableBlinkingCursor Fix option "disableBlinkingCursor" to also work within shadowdom # Committers: 1 - Carlo Jeske ([@plusgut](https://github.com/plusgut)) - Updated dependencies [8ff1bc3] - @wdio/image-comparison-core@1.0.2 ## 9.0.1 ### Patch Changes - 79d2b1d: # 🐛 Bugfixes ## #1073 Normalize Safari desktop screenshots by trimming macOS window corner radius and top window shadow Safari desktop screenshots included the macOS window mask at the bottom and a shadow at the top. These artifacts caused incorrect detection of the viewable area for full page screenshots, which resulted in misaligned stitching. The viewable region is now calculated correctly by trimming these areas. # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - 782b98a: # 🐛 Bugfixes ## #1000 fix incorrect cropping and stitching of last image for fullpage screenshots on mobile The determination of the position of the last image in mobile fullpage webscreenshots was incorrect. This was mostly seen with iOS, but also had some impact on Android. This is now fixed # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - 2c109b3: # 🐛 Bugfixes ## #1038 fix incorrect determination of ignore area Ignore regions with `left: 0` and `right:0` lead to an incorrect width which lead to an incorrect ignore area. This is now fixed # Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [79d2b1d] - Updated dependencies [782b98a] - Updated dependencies [2c109b3] - @wdio/image-comparison-core@1.0.1 ## 9.0.0 ### Major Changes - 1326e99: ## 💥 Major Release: New @wdio/image-comparison-core Package ### 🏗️ Architectural Refactor This release introduces a **completely new core architecture** with the dedicated `@wdio/image-comparison-core` package, replacing the generic `webdriver-image-comparison` module with a WDIO-specific solution. #### What was the problem? - The old `webdriver-image-comparison` package was designed for generic webdriver usage - Complex integration between generic and WDIO-specific code - Limited test coverage (~58%) making maintenance difficult - Mixed responsibilities between core logic and service integration #### What changed? ✅ **New dedicated core package**: `@wdio/image-comparison-core` - purpose-built for WebdriverIO ✅ **Cleaner architecture**: Modular design with clear separation of concerns ✅ **Enhanced test coverage**: Improved from ~58% to ~90% across all metrics ✅ **Better maintainability**: Organized codebase with comprehensive TypeScript interfaces ✅ **WDIO-specific dependencies**: Only depends on `@wdio/logger`, `@wdio/types`, etc. ### 🧪 Testing Improvements - **100% branch coverage** on critical decision points - **Comprehensive unit tests** for all major functions - **Optimized mocks** for complex scenarios - **Better test isolation** and reliability | Before/After | % Stmts | % Branch | % Funcs | % Lines | | ------------------ | ------- | -------- | ------- | ------- | | **Previous** | 58.59 | 91.4 | 80.71 | 58.59 | | **After refactor** | 90.55 | 96.38 | 93.99 | 90.55 | ### 🔧 Service Integration The `@wdio/visual-service` now imports from the new `@wdio/image-comparison-core` package while maintaining the same public API and functionality for users. ### 📈 Performance & Quality - **Modular architecture**: Easier to maintain and extend - **Type safety**: Comprehensive TypeScript coverage - **Clean exports**: Well-defined public API - **Internal interfaces**: Proper separation of concerns ### 🔄 Backward Compatibility ✅ **No breaking changes** for end users ✅ **Same public API** maintained ✅ **Existing configurations** continue to work ✅ **All existing functionality** preserved ### 🎯 Future Benefits This refactor sets the foundation for: - Easier addition of new features - Better bug fixing capabilities - Enhanced mobile and native app support - More reliable MultiRemote functionality ### 📦 Dependency Updates - Updated most dependencies to their latest versions - Improved security with latest package versions - Better compatibility with current WebdriverIO ecosystem - Enhanced performance through updated dependencies *** **Note**: This is an architectural improvement that modernizes the codebase while maintaining full backward compatibility. All existing functionality remains unchanged for users. *** ### Patch Changes - be4272c: fix: [983](#983) in multiremote, commands are now executed on the requested instances - Updated dependencies [74df53b] - Updated dependencies [1326e99] - @wdio/image-comparison-core@1.0.0 ## Committers: 2 - P-Courteille ([@P-Courteille](https://github.com/P-Courteille)) - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ## 8.0.4 ### Patch Changes - d88d8dd: Optimize Mobile and Emulated device support ## 🐛 Bugfixes ### #969 Fix layer injection on mobile devices On some devices the layer injection, to determine the exact position of the webview, was failing. It exceeded the appium timeout and returned an error like ```logs [1] [0-0] 2025-05-23T08:04:11.788Z INFO webdriver: COMMAND getUrl() [1] [0-0] 2025-05-23T08:04:11.789Z INFO webdriver: [GET] https://hub-cloud.browserstack.com/wd/hub/session/xxxxx/url [1] [0-0] 2025-05-23T08:04:12.036Z INFO webdriver: RESULT about:blank [1] [0-0] 2025-05-23T08:04:12.038Z INFO webdriver: COMMAND navigateTo("data:text/html;base64,CiAgICAgICAgPG .... LONG LIST OF CHARACTERS=") [1] [0-0] 2025-05-23T08:04:12.038Z INFO webdriver: [POST] https://hub-cloud.browserstack.com/wd/hub/session/xxxx/url [1] [0-0] 2025-05-23T08:04:12.038Z INFO webdriver: DATA { [1] [0-0] url: 'data:text/html;base64,CiAgICAgICAgPGh0bWw.... LONG LIST OF CHARACTERS=' [1] [0-0] } [1] [0-0] 2025-05-23T08:05:42.132Z ERROR @wdio/utils:shim: Error: WebDriverError: The operation was aborted due to timeout when running "url" with method "POST" and args "{"url":"data:text/html;base64,CiAgICAgICAgPGh0b.... LONG LIST OF CHARACTERS="}" [1] [0-0] at FetchRequest._libRequest (file:///xxxxxxx/node_modules/webdriver/build/node.js:1836:13) [1] [0-0] 2025-05-23T08:05:42.132Z DEBUG @wdio/utils:shim: Finished to run "before" hook in 91147ms [1] [0-0] at process.processTicksAndRejections (node:internal/process/task_queues:95:5) [1] [0-0] at async FetchRequest._request (file:///C:/xxxxxx/node_modules/webdriver/build/node.js:1846:20) [1] [0-0] at Browser.wrapCommandFn (c:/Projects/xxxxxx/node_modules/@wdio/utils/build/index.js:907:23) [1] [0-0] at Browser.url (c:/Projects/xxxxxxx/node_modules/webdriverio/build/node.js:5682:3) [1] [0-0] at Browser.wrapCommandFn (c:/Projects/xxxxxx/node_modules/@wdio/utils/build/index.js:907:23) [1] [0-0] at async loadBase64Html (file:///C:/Projects/xxxxxx/node_modules/webdriver-image-comparison/dist/helpers/utils.js:377:5) [1] [0-0] at async getMobileViewPortPosition (file:///C:/Projects/xxxxxx/node_modules/webdriver-image-comparison/dist/helpers/utils.js:417:9) [1] [0-0] at async getMobileInstanceData (file:///C:/Projects/xxxxxx/node_modules/@wdio/visual-service/dist/utils.js:58:28) [1] [0-0] at async getInstanceData (file:///C:/Projects/xxxxxxx/node_modules/@wdio/visual-service/dist/utils.js:189:77) [1] [0-0] 2025-05-23T08:05:42.144Z INFO @wdio/browserstack-service: Update job with sessionId xxxxx ``` This was caused by the `await url(`data:text/html;base64,${base64Html}`)` that injected the layer. Some browsers couldn't handle the `data:text/html;base64`. We now fixed that with a different injection. It was tested on Android/iOS and on Sims/Emus/Real Devices and it worked ### Improve determining if a device is emulated In a previous release we added a function to determine if a device was emulated. This resulted in incorrect screen sizes that were used for the files names for devices. This caused or failing baselines, or new files to be created because the screen sizes were not available We now improved the check and the correct screen sizes are added again to the file names and made sure that the previous generated base line could be used again. ## Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [d88d8dd] - webdriver-image-comparison@9.0.4 ## 8.0.3 ### Patch Changes - 2f9ec42: ## 🐛 Bug-fixes ### #967: Emulated device crops with `enableLegacyScreenshotMethod` set to `true` are not correct When a screenshot of an emulated device is taken, but the browser was initially started as a "desktop" session, so not with emulated caps, and the `enableLegacyScreenshotMethod` property is set to `true`, the DPR of the emulated device is taken. This resulted in incorrect crop. We now store the original dpr and use that for the crop when it's an emulated device and started as a desktop browser session. ## BiDi Fullpage screenshots for emulated device are broken The BiDi fullpage screenshot for an emulated device is broken in the driver. We now fallback to the legacy screenshot method for BiDi and emulated devices ## 💅 Polish - Updated the multiple interfaces to use JS-Doc for better docs - When `createJsonReportFiles` is set to `true` and there are a lot of differences we kept waiting. We now limited that to check a max of 5M diff-pixels or a diff threshold of 20%. If it's bigger the report will show a full coverage and extra logs are shown in the WDIO logs, something like this ```logs [0-0] 2025-05-24T06:02:18.887Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Processing diff pixels started [0-0] 2025-05-24T06:02:18.888Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Processing 20143900 diff pixels [0-0] 2025-05-24T06:02:19.770Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Total pixels in image: 52,184,160 [0-0] 2025-05-24T06:02:19.770Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Number of diff pixels: 20,143,900 [0-0] 2025-05-24T06:02:19.770Z INFO @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Diff percentage: 38.60% [0-0] 2025-05-24T06:02:19.770Z ERROR @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Too many differences detected! Diff percentage: 38.60%, Diff pixels: 20,143,900 [0-0] 2025-05-24T06:02:19.771Z ERROR @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: This likely indicates a major visual difference or an issue with the comparison. [0-0] 2025-05-24T06:02:19.771Z ERROR @wdio/visual-service:webdriver-image-comparison:pixelDiffProcessing: Consider checking if the baseline image is correct or if there are major UI changes. ``` ## Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [2f9ec42] - webdriver-image-comparison@9.0.3 ## 8.0.2 ### Patch Changes - 9363467: ## 🐛 Bug-fixes - #946: Visual Regression Changes in WDIO v9 - Fixed screen size detection in emulated mode for filenames. Previously used incorrect browser window size. - Fixed screenshot behavior when `enableLegacyScreenshotMethod: true`, now correctly captures emulated screen instead of complete screen. - Fixed emulated device handling for Chrome and Edge browsers, now properly sets device metrics based on `deviceMetrics` or `deviceName` capabilities. ## Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [9363467] - webdriver-image-comparison@9.0.2 ## 8.0.1 ### Patch Changes - 5c6c6e2: Fix capturing element screenshots with BiDi This release fixes #919 where an element screenshot, that was for example from an overlay, dropdown, popover, tooltip, modal, was returning an incorrect screenshot ## Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) - Updated dependencies [5c6c6e2] - webdriver-image-comparison@9.0.1 ## 8.0.0 ### Major Changes - bfe6aca: ## 💥 BREAKING CHANGES ### 🧪 Web Screenshot Strategy Now Uses BiDi by Default #### What was the problem? Screenshots taken via WebDriver's traditional protocol often lacked precision: - Emulated devices didn't reflect true resolutions - Device Pixel Ratio (DPR) was often lost - Images were cropped or downscaled #### What changed? All screenshot-related methods now use the **WebDriver BiDi protocol** by default (if supported by the browser), enabling: ✅ Native support for emulated and high-DPR devices ✅ Better fidelity in screenshot size and clarity ✅ Faster, browser-native screenshots via [`browsingContext.captureScreenshot`](https://w3c.github.io/webdriver-bidi/#command-browsingContext-captureScreenshot) The following methods now use BiDi: - `saveScreen` / `checkScreen` - `saveElement` / `checkElement` - `saveFullPageScreen` / `checkFullPageScreen` #### What’s the impact? ⚠️ **Existing baselines may no longer match.** Because BiDi screenshots are **sharper** and **match device settings more accurately**, even a small difference in resolution or DPR can cause mismatches. > If you rely on existing baseline images, you'll need to regenerate them to avoid false positives. #### Want to keep using the legacy method? You can disable BiDi screenshots globally or per test using the `enableLegacyScreenshotMethod` flag: **Globally in `wdio.conf.ts`:** ```ts import { join } from "node:path"; export const config = { services: [ [ "visual", { baselineFolder: join(process.cwd(), "./localBaseline/"), debug: true, formatImageName: "{tag}-{logName}-{width}x{height}", screenshotPath: join(process.cwd(), ".tmp/"), enableLegacyScreenshotMethod: true, // 👈 fallback to W3C-based screenshots }, ], ], }; ``` **Or per test:** ```ts it("should compare an element successfully using legacy screenshots", async function () { await expect($(".hero__title-logo")).toMatchElementSnapshot( "legacyScreenshotLogo", { enableLegacyScreenshotMethod: true } // 👈 fallback to W3C-based screenshots ); }); ``` ## 🐛 Bug Fixes - ✅ [#916](https://github.com/webdriverio/visual-testing/issues/916): Visual Testing Screenshot Behavior Changed in Emulated Devices ## Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ### Patch Changes - Updated dependencies [bfe6aca] - webdriver-image-comparison@9.0.0 ## 7.0.0 ### Major Changes - 42956e4: ## 💥 BREAKING CHANGES ### 🔍 Viewport Screenshot Logic Reworked for Mobile Web & Hybrid Apps #### What was the problem? Screenshots for mobile devices were inconsistent due to platform differences. iOS captures the entire device screen (including status and address bars), while Android (using ChromeDriver) only captures the webview, unless the capability `"appium:nativeWebScreenshot": true` is used. #### What changed? We’ve reimplemented the logic to correctly handle both platforms by default. This fix addresses [[#747](https://github.com/webdriverio/visual-testing/pull/747)](https://github.com/webdriverio/visual-testing/pull/747). 💡 Credit to [Benjamin Karran (@ebekebe)](https://github.com/ebekebe) for pointing us in the right direction to improve this logic! #### What’s the advantage? ✅ More **accurate full-page and element screenshots** on both Android and iOS. ⚠️ But this change may **break your current baselines**, especially on Android and iOS. *** ### 🍏 iOS Element Screenshot Strategy Changed #### What was the problem? iOS element screenshots were previously cut from full-device screenshots, which could lead to misalignment or off-by-a-few-pixels issues. #### What changed? We now use the element screenshot endpoint directly. #### What’s the advantage? ✅ More accurate iOS element screenshots. ⚠️ But again, this may affect your existing baselines. *** ### 🖥️ New Full-Page Screenshot Strategy for **Desktop Web** #### What was the problem? The "previous" scroll-and-stitch method simulated user interaction by scrolling the page, waiting, taking a screenshot, and repeating until the entire page was captured. This works well for **lazy-loaded content**, but it is **slow and unstable** on other pages. #### What changed? We now use WebDriver BiDi’s [`[browsingContext.captureScreenshot](https://webdriver.io/docs/api/webdriverBidi#browsingcontextcapturescreenshot)`] to capture **full-page screenshots in one go**. This is the new **default strategy for desktop web browsers**. 📌 **Mobile platforms (iOS/Android)** still use the scroll-and-stitch approach for now. #### What’s the advantage? ✅ Execution time reduced by **50%+** ✅ Logic is greatly simplified ✅ More consistent and stable results on static or non-lazy pages 📸 ![Example](https://github.com/user-attachments/assets/394ad1d6-bbc7-42dd-b93b-ff7eb5a80429) **Still want the old scroll-and-stitch behavior or need fullpage screenshots for pages who have lazy-loading?** Use the `userBasedFullPageScreenshot` option to simulate user-like scrolling. This remains the **better choice for pages with lazy-loading**: ```ts // wdio.conf.ts services: [ [ "visual", { userBasedFullPageScreenshot: true, }, ], ]; ``` Or per test: ```ts await expect(browser).toMatchFullPageSnapshot("homepage", { userBasedFullPageScreenshot: true, }); ``` *** ## 💅 Polish ### ⚠️ Deprecated Root-Level Compare Options #### What was the problem? Compare options were allowed at the root level of the service config, making them harder to group or discover. #### What changed? You now get a warning if you still use root-level keys. Please move them under the `compareOptions` property instead. **Example warning:** ```log WARN The following root-level compare options are deprecated and should be moved under 'compareOptions': - blockOutStatusBar - ignoreColors In the next major version, these options will be removed from the root level. ``` 📘 See: [[compareOptions docs](https://webdriver.io/docs/visual-testing/service-options#compare-options)](https://webdriver.io/docs/visual-testing/service-options#compare-options) *** ## 🐛 Bug Fixes - ✅ [[#747](https://github.com/your-repo/issues/747)](https://github.com/your-repo/issues/747): Fixed incorrect mobile webview context data. *** ## 🔧 Other - 🆙 Updated dependencies - 🧪 Improved test coverage - 📸 Refreshed image baselines *** ## Committers: 1 - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ### Patch Changes - Updated dependencies [42956e4] - webdriver-image-comparison@8.0.0 ## 6.4.0 ### Minor Changes - 7f859aa: Add `additionalSearchParams` to the Storybook Runner API - 307fbec: Add `getStoriesBaselinePath` to Storybook Runner API, enabling custom file paths (e.g. files with a flat hierarchy in the baselines folder) ### Patch Changes - 3d232d1: Fix compareOptions not passed from config to the storybook runner tests - Updated dependencies [7f859aa] - Updated dependencies [307fbec] - webdriver-image-comparison@7.4.0 ### Committers: 2 - Fábio Correia [@fabioatcorreia](https://github.com/fabioatcorreia) - alcpereira ([@alcpereira](https://github.com/alcpereira)) ## 6.3.3 ### Patch Changes - 77bc764: Removed unused `node-fetch` dependency ### Committers: 1 - alcpereira ([@ alcpereira](https://github.com/alcpereira)) ## 6.3.2 ### Patch Changes - 09dbc2d: update deps - 09dbc2d: don't check for runner - Updated dependencies [09dbc2d] - webdriver-image-comparison@7.3.2 ### Committers: 2 - Christian Bromann ([@christian-bromann](https://github.com/christian-bromann)) - Wim Selles ([@wswebcreation](https://github.com/wswebcreation)) ## 6.3.1 ### Patch Changes - 69d25fe: Multiple fixes: - update deps - fix #728: Missing Type Information in Service Configuration - fix #745: The image comparison process stalls when utilizing the checkElement method to compare screenshots of two different elements - Updated dependencies [69d25fe] - webdriver-image-comparison@7.3.1 ## 6.3.0 ### Minor Changes - 2d033e8: Add LambdaTest support - #691 Add option to ignore blinking cursors / carets by providing the option `disableBlinkingCursor` - #692 Add support for `appium:options` ### Patch Changes - Updated dependencies [2d033e8] - webdriver-image-comparison@7.3.0 ## 6.2.5 ### Patch Changes - 4a4adf1: Fix resize dimensions for mobile - 4a4adf1: update deps - Updated dependencies [4a4adf1] - Updated dependencies [4a4adf1] - webdriver-image-comparison@7.2.2 ## 6.2.4 ### Patch Changes - 36541dd: Fix issue 679 ## 6.2.3 ### Patch Changes - cc2b0fb: Fix issue 656 ## 6.2.2 ### Patch Changes - a34dd5d: Update of deps - d751b8c: fix android app determination ## 6.2.1 ### Patch Changes - 1df5350: # Improve iPhone support ## 💅 Polish @wdio/visual-reporter - Mobile: support iOS 18 and the iPhone 16 series for the blockouts ## 🐛 Bugs fixed @wdio/visual-reporter - Mobile: don't use the device blockouts for element screenshot - Mobile: when the blockouts had the value `{x: 0, y: 0, width: 0, height: 0}` then Resemble picked this up as a full blockout. This caused false positives for iOS - Updated dependencies [1df5350] - webdriver-image-comparison@7.2.1 ## 6.2.0 ### Minor Changes - 6e20f06: Upgrade dependencies ## 6.1.0 ### Minor Changes #### ea08748 Fix [495](https://github.com/webdriverio/visual-testing/issues/495): module system issue when using with vite and storybook This fix only allows `runner` to be the `local` one, if not, it will throw this error ```logs pnpm test.local.desktop.storybook ─╯ > @wdio/visual-testing-monorepo@ test.local.desktop.storybook /Users/wswebcreation/Git/wdio/visual-testing > wdio tests/configs/wdio.local.desktop.storybook.conf.ts --storybook --numShards=10 --url=https://govuk-react.github.io/govuk-react/ --skipStories="/.*(loading-box|spinner).*/" Execution of 0 workers started at 2024-09-22T05:50:20.875Z SevereServiceError in "onPrepare" SevereServiceError: A service failed in the 'onPrepare' hook SevereServiceError: Running `@wdio/visual-service` is only supported in `local` mode. at VisualLauncher.onPrepare (file:///Users/wswebcreation/Git/wdio/visual-testing/packages/visual-service/dist/storybook/launcher.js:22:19) at file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@wdio+cli@9.0.7/node_modules/@wdio/cli/build/index.js:1209:32 at Array.map (<anonymous>) at runServiceHook (file:///Users/wswebcreation/Git/wdio/visual-testing/node_modules/.pnpm/@wdio+cli@9.0.7/node_modules/@wdio/cli/build/index.js:1206:31) at Launcher.run (file:///Use