@cerner/terra-toolkit-docs/src/terra-dev-site/tool/terra-functional-testing/About.1.tool.mdx

Contains documentation for packages in the terra-toolkit monorepo

import { Notice } from '@cerner/terra-docs';
import { Badge } from '@cerner/terra-functional-testing/package.json?dev-site-package';

<Badge />

# Terra Functional Testing

The terra-functional-testing library is a utility for developing automation tests. The library extends [WebdriverIO](https://v6.webdriver.io/) to facilitate automating accessibility and functional testing for Terra projects.

## System Requirements

- [Node.js](http://nodejs.org/) Install at least v10.24.0. Due to requirements of fibers used within the project install no greater than lts/erbium (v12).
- [Docker](https://www.docker.com/) Install docker. Docker is used to setup and run tests in a containerized environment.

We strongly recommend using [nvm](https://github.com/nvm-sh/nvm) for installing node versions.

## Installation

Install @cerner/terra-functional-testing and @cerner/terra-cli as development dependencies with npm:

```bash
npm install --save-dev @cerner/terra-functional-testing @cerner/terra-cli
```

## Usage

A test runner is provided for local development. The test runner is invoked via the [terra-cli](/dev_tools/cerner-terra-toolkit-docs/terra-cli/about).

package.json
```json
"scripts": {
  "test:wdio": "terra wdio"
}
```

### Options

|            Option            |  Type   |                    Default                    |                                                                  Description                                                                   |
| ---------------------------- | ------- | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| --asset-server-port          | number  | 8080                                          | The port to run the webpack and express asset services on.                                                                                     |
| --browsers                   | array   |                                               | A list of browsers for the test run.                                                                                                           |
| --config, -c                 | string  |                                               | A file path to the test runner configuration.                                                                                                  |
| --disable-server             | boolean | false                                         | A flag to disable the internal webpack or express server. An external server must be running in order for the test runner to run successfully. |
| --disable-selenium-service   | boolean | false                                         | A flag to disable the selenium docker service.                                                                                                 |
| --ignore-screenshot-mismatch | boolean | false                                         | A flag to ignore screenshot mismatch.                                                                                                          |
| --external-host              | string  |                                               | The host address the testing environment is connected to.                                                                                      |
| --external-port              | number  |                                               | The port mapping from the host to the container.                                                                                               |
| --form-factors               | array   |                                               | A list of form factors for the test run. One of tiny, small, medium, large, huge, or enormous                                                  |
| --grid-url                   | string  |                                               | The remote selenium grid address.                                                                                                              |
| --locales                    | array   | en                                            | A list of language locales for the test run.                                                                                                   |
| --selenium-service-port      | number  | 4444                                          | The port mapping for the selenium service or the external selenium grid.                                                                       |
| --selenium-service-url       | string  | localhost                                     | The address for the selenium service.                                                                                                          |
| --site                       | string  |                                               | A file path to a static directory of assets. When defined, an express server will launch to serve the assets and disable running webpack.      |
| --spec                       | array   |                                               | A list of spec file paths.                                                                                                                     |
| --suite                      | array   |                                               | Overrides specs and runs only the defined suites.                                                                                              |
| --themes                     | array   | theme specified in terra-theme.config.js file | A list of themes for the test run.                                                                                                             |
| --update-screenshots, -u     | boolean | false                                         | Updates all reference screenshots with the latest screenshots.                                                                                 |
| --use-https                  | boolean | false                                         | A flag to turn on secure HTTP for the webpack server when a gridUrl is provided.                                                               |

Note: the arguments can also be passed as camelCase e.g. `--updateScreenshots`.

The following example will run the test suite a total of four times. Once for each permutation of the specified locales and form factors.

```json
"scripts": {
  "test:wdio": "terra wdio --locales en fr --form-factors tiny huge"
}
```

## Browser Capabilities

The following browsers are supported:

- Chrome
- Firefox
- Internet Explorer

Chrome is enabled by default when running locally against the docker container.

To target a list of browsers that are different than the default use the `browsers` cli option:

<Notice variant="important" ariaLevel="3">

  Internet Explorer can only be enabled when running against a remote selenium grid. Internet Explorer is not available within the docker container.

</Notice>

```json
"scripts": {
  "test:wdio": "terra wdio --browsers firefox"
}
```

To run against a remote selenium grid:

<Notice variant="important" ariaLevel="3">

  Chrome, Firefox, and Internet Explorer are all enabled by default when running against a remote selenium grid.

</Notice>

```json
"scripts": {
  "test:wdio": "terra wdio --grid-url grid.test.example.com"
}
```

## Form Factors

Tests can be executed in the following form factors:

| Size     | Width | Height |
|----------|-------|--------|
| tiny     | 470   | 768    |
| small    | 622   | 768    |
| medium   | 838   | 768    |
| large    | 1020  | 768    |
| huge     | 1300  | 768    |
| enormous | 1500  | 768    |

To specify a list of form factors use the cli option:

<Notice variant="important" ariaLevel="3">

  If no form factor is specified all tests will be run against the huge form factor.

</Notice>

```json
"scripts": {
  "test:wdio": "terra wdio --form-factors tiny small enormous"
}
```

## Configuration

The test runner ships with a default [wdio.conf.js](https://github.com/cerner/terra-toolkit/blob/main/packages/terra-functional-testing/src/config/wdio.conf.js). Options can be configured by extending the default configuration file.

To extend the default configuration create a `wdio.conf.js` file at the root of your project and apply the desired options.

wdio.conf.js
```js
const { config } = require('@cerner/terra-functional-testing');

// Stop the test run if there is a test failure.
config.bail = 1;

exports.config = config;
```

## Service Options

Service options are configurable by extending the wdio.conf.js file. These options are applied for every test run.

### selector

Specifies the default element to be captured when taking a screenshot.

Type: `string`

Required: `false`

Default: `[data-terra-test-content] *:first-child`

<Notice variant="important" ariaLevel="4">

  This selector is used as the default screenshot selector for Terra.validates.element and Terra.validates.screenshot.

</Notice>

```js
const { config } = require('@cerner/terra-functional-testing');

config.serviceOptions = {
  selector: '#root',
};

exports.config = config;
```

### seleniumVersion

Specifies the [docker selenium](https://github.com/SeleniumHQ/docker-selenium/tree/selenium-3) version used within the docker container.

Type: `string`

Required: `false`

Default: `3.14.0-helium`

<Notice variant="caution" ariaLevel="4">

  The selenium version is only applied when running tests against a docker container. This option does not change the version on a remote selenium grid. Keep this in mind if your tests run against a remote selenium grid on a CI system.

</Notice>

```js
const { config } = require('@cerner/terra-functional-testing');

config.serviceOptions = {
  seleniumVersion: '3.141.59-20210311',
};

exports.config = config;
```

## Test Utilities

Test utilities are available to help write tests. These utilities are accessed using the `Terra` global object.

### Describe Helpers

The describe helpers are an alias for the top level `describe` block used within spec files. The describe helpers provide useful features for limiting a set of tests to specific form factors, locales, and themes.

#### Terra.describeViewports

<Notice variant="important" ariaLevel="5">

  We recommended using Terra.describeTests instead of Terra.describeViewports. The same features in Terra.describeViewports can be achieved with Terra.describeTests.

</Notice>

The `Terra.describeViewports` utility defines a list of form factors the tests are enabled for. By default, all tests will be run against each provided viewport. If a form factor is specified via the cli the describe viewpoint helper works as a filter and only allows the tests to execute if they match the current form factor.

```js
/**
 * Executes all tests for each defined viewport.
 * @param {string} title - The describe block title.
 * @param {string[]} viewports - A list of viewports. [tiny, small, medium, large, huge, enormous]
 * @param {function} fn - The block of tests to execute against each viewport.
 */
Terra.describeViewports(title, [viewports], fn);
```

<Notice variant="caution" ariaLevel="5">

  Describe helpers should not be nested within other describe helpers.

</Notice>

```js
// The following example will scope the tests to run only for the `tiny` and `huge` form factors. The tests will not run for any other form factor.

Terra.describeViewports('Describe viewports title', ['tiny', 'huge'], () => {
  it('should test tiny and huge screens', () => {
    browser.url('/testing/route/');

    Terra.validates.screenshot('describe viewports');
  });
});

// Multiple describe helpers can be used sequentially in the same spec file as long as they are not nested.

Terra.describeViewports('Describe viewports title', ['small'], () => {
  it('should test small screens', () => {
    browser.url('/testing/route/small/');

    Terra.validates.screenshot('describe viewports');
  });
});
```

#### Terra.describeTests

The `Terra.describeTests` helper extends the `Terra.describeViewports` helper by additionally filtering tests by locale and theme. Tests within this helper will only be executed if each of the form factors, locales, and themes listed in the options match those defined in the configurations for the current test run. If no form factors, locales, or themes are provided as options, then all tests under this helper will qualify to run in any form factor, locale, or theme.

```js
/**
 * Executes all tests for each defined set of form factors, locales, and themes.
 * @param {string} title - The describe block title.
 * @param {Object} options - An object containing arrays of formFactors, locales, and themes that the block of tests will only qualify to execute in.
 * @param {string} options.formFactors - The form factors that the block of tests only execute in.
 * @param {string} options.locales -  The language locales that the block of tests only execute in.
 * @param {string} options.themes - The themes that the block of tests only execute in.
 * @param {function} fn - The block of tests to execute based on the defined form factor, locale, and theme.
 */
Terra.describeTests(title, options, fn);
```

The following tests will only be ran for the locales, themes, and form factors provided:

<Notice variant="caution" ariaLevel="5">

  The describe helper should not be nested inside of another describe helper.

</Notice>

```js
const testOptions = {
  formFactors: ['tiny', 'huge'],
  locales: ['en', 'fr'],
  themes: ['terra-default-theme', 'orion-fusion-theme']
}

Terra.describeTests('Describe tests', testOptions, () => {
  it('should execute only if form factor is tiny or huge, locale is en or fr, and theme is terra-default-theme or orion-fusion-theme', () => {
    browser.url('/testing/route/');
    Terra.validates.element('describe tests');
  });
});
```

### Accessibility Testing

The testing library integrates [axe-core](https://github.com/dequelabs/axe-core) into the testing framework to help automate accessibility testing along side functional testing.

Axe will analyze the entire document when run and report accessibility violations. The following [tags](https://github.com/dequelabs/axe-core/blob/develop/doc/API.md#axe-core-tags) are enabled: `wcag2a`, `wcag2aa`, `wcag21aa`, and `section508`. Each tag has an associated list of [rules](https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md) that will be checked against the document when axe is run.

<Notice variant="important" ariaLevel="4">

  Please note that not all accessibility testing can be automated. Axe provides a lightweight static analysis of the entire document to catch common accessibility violations, but it is the responsibility of each team and application to do thorough accessibility and functional testing manually when necessary.

</Notice>

#### Terra.validates.accessibility

The validates accessibility command will run accessibility checks on the entire document. If accessibility violations are found the test step will fail. It is recommended to run accessibility checks at various steps in a functional workflow to check for violations at different stages of the application.

<Notice variant="caution" ariaLevel="4">

  The accessibility assertion must be used within an `it` block.

</Notice>

```js
it('should report no accessibility violations', () => {
  browser.url('/testing/route/');

  Terra.validates.accessibility(); // Fails if accessibility violations are found anywhere on the document.
});
```

The accessibility assertion accepts [options](https://github.com/dequelabs/axe-core/blob/develop/doc/API.md#options-parameter) that will be passed to axe for the document analysis.

```js
it('should override a rule configuration', () => {
  browser.url('/testing/route/');

  // Rule override for this specific test.
  const options = { rules: { 'color-contrast': { enabled: false } } };

  Terra.validates.accessibility(options);
});
```

### Screenshot Testing

A screenshot can be captured at any given point inside an `it` block in a test. The element(s) being captured in the screenshot is determined by the provided selector option or the default `[data-terra-test-content]` selector if one is not provided. If part or all of the selector or any of its children are rendered outside the bounds of the current viewport, only what is within the viewport will be captured and what is outside the viewport will be clipped. The captured screenshot will be used as a reference artifact. Element validation is done by comparing the reference screenshot against the screenshot captured in future test runs at the same point in the test.

#### Terra.validates.screenshot

Invoking the assertion will capture a screenshot at the time it is invoked. If no reference screenshot exists, one will be created in the `reference` folder with the given screenshot name. If such reference screenshot already exists, the new screenshot will be compared with the reference screenshot to validate visually that they are within the mismatch tolerance. If the mismatch exceeds the tolerance, the test step will fail and a screenshot showing the difference will be generated.

<Notice variant="caution" ariaLevel="5">

  The screenshot assertion must be used within an `it` block and a screenshot name must be provided.

</Notice>

```js
it('should validate the screenshot', () => {
  browser.url('/testing/route/');

  Terra.validates.screenshot('screenshot-name'); // Fails if the new screenshot doesn't visually match the reference screenshot.
});
```

The screenshot naming pattern is as follows:

```js
base_directory/test_spec_directory/test_spec_name/__snapshots__/(reference|latest|diff)/theme/locale/browserName_terraViewportName/screenshot-name.png
```

The screenshot assertion accepts an optional second argument with the following keys.
  * `selector` - The element selector to use to capture the element for screenshot comparison. The default selector is `data-terra-test-content`.
  * `mismatchTolerance` - The number between 0 and 100 that defines the degree of mismatch to consider two images as identical. Increasing this value will decrease level of test coverage. The default mismatch tolerance is 0.01.

```js
it('should validate the screenshot with options', () => {
  browser.url('/testing/route/');

  const options = {
    selector: '#element-id',
    mismatchTolerance: 10,
  };

  Terra.validates.screenshot('screenshot-name', options);
});
```

#### Terra.validates.element

The validates element assertion performs both accessibility and screenshot assertions.

<Notice variant="caution" ariaLevel="5">

  The validates element assertion must be used within an `it` block and a screenshot name must be provided.

</Notice>

```js
it('should validate the element', () => {
  browser.url('/testing/route/');

  Terra.validates.element('screenshot-name'); // Fails if the new screenshot doesn't visually match the reference screenshot.
});
```

The element assertion accepts an optional second argument with the following keys.
  * `rules` - The accessibility [rules](https://github.com/dequelabs/axe-core/blob/master/doc/rule-descriptions.md) to use as overrides if necessary.
  * `selector` - The element selector to use to capture the element for screenshot comparison. The default selector is `data-terra-test-content`.
  * `mismatchTolerance` - The number between 0 and 100 that defines the degree of mismatch to consider two images as identical. Increasing this value will decrease level of test coverage. The default mismatch tolerance is 0.01.

```js
it('should validate the element with options', () => {
  browser.url('/testing/route/');

  const options = {
    rules: { 'color-contrast': { enabled: false } },
    selector: '#element-id',
    mismatchTolerance: 10,
  };

  Terra.validates.element('screenshot-name', options);
});
```

### Additional Utilities

#### Terra.hideInputCaret

An editable text field in focus will have a blinking caret. Often times this blinking caret causes inconsistent test failures due to the blinking of the caret during screenshot capture. This situation can be avoided by using `Terra.hideInputCaret` to set the CSS caret color to of the element to be transparent. This method must be placed in a `before`, `beforeEach`, or `it` block or it will not be ran. This method accepts a selector string as an argument and will only apply to the first selector if multiple are found. The caret will automatically be hidden for body every time the page loads or refreshes.

```js
it('should hide the caret', () => {
  Terra.hideInputCaret('#inputID');
  const element = browser.element('#inputID');

  expect(element.getCSSProperty('caretColor').value).toEqual('rgba(0,0,0,0)');
});
```

#### Terra.setApplicationLocale

Use `Terra.setApplicationLocale` to update a terra application's locale on a case by case basis; particularly useful for testing locale changes on a deployed application. This method accepts a string containing the `locale` to update to.

```js
it('should update an application's locale, () => {
  Terra.setApplicationLocale('en');
});
```

## Remote Screenshots Migration

<Notice variant="important" ariaLevel="3">

  The Remote Screenshots Mirgation feature is currently only supported by internal projects built using Jenkins at Cerner Corporation. Open source projects that utilize other CI such as Travis are not supported.

</Notice>

Follow this [consumption guide](https://pages.github.cerner.com/orion/docs/doc/docs/nexus-wdio-screenshots-mirgation/about) to utilize this feature.