@apica-io/asm-pm-runner
Version:
Run a postman collection in Apica ASM.
331 lines (308 loc) • 14 kB
Markdown
- [Introduction](
- [Apica ASM Integration](
- [Installation](
- [Change log summary](
- [Command line Options](
- [Samples](
- [Example 1 - Running some http requests on http-bin.](
- [Example 2 - Testing certificate on https://badssl.com](#example-2---testing-certificate-on-httpsbadsslcom)
- [The command line](
- [The console output from the command](
- [Reporting of result](
- [Steps and requests in the report](
- [Returning a custom value as the result](
- [Returning variables in the result](
- [Reporting Variables in json result report](
- [The generated json report](
- [Encryption of secret data](
- [Installation of cryptify](
- [Example of encrypt a file](
- [Example with encrypted postman_environment](
__asm-pm-runner__ runs postman collections and export results to crs format for usage in Apica ASM.
The command line tool is based on the standard Postman Newman utility. See [Newman npm at](https://www.npmjs.com/package/newman).
The main enhancement to Newman is support for decryption of data files and certificates. Encryption/decryption is based on the [cryptify](https://www.npmjs.com/package/cryptify) npm module.
Apica ASM (Synthetics) can run Postman checks based on __asm-pm-runner__. You can contact [Apica](https://www.apica.io/) about this feature/functionality.
## Installation
```console
npm install -g @apica-io/asm-pm-runner
```
## Change log summary
Version| Summary changes |
-------|-----------------|
1.4.8 | Enhanced the return message to provide customized error feedback upon assertion failure.
1.3.7 | Customized return message and fixes for variables with 0 values
1.3.5 | Report error/exception in the result message
1.3.4 | Fixed custom return value with unit and variables in json report
## Command line Options
``` console
Usage: asm-pm-runner [options] <collection>
Options:
-e, --environment <path> Environment JSON file
-dk, --decryptKey <decryptKey> Decrypt key
-ev, --envVars [envVars...] Environment variables. Format name=value format
-v, --verbose Print collection information on stdout (default: false)
-r, --resultDir <dir> The result directory
-dd, --dataDir <dir> The data directory for certificates and test data
-l, --logLevel <logLevel> Log level in log4js (default: "info")
--sslClientCert <path> Specify the path to a client certificate (PEM)
--sslClientKey <path> Specify the path to a client certificate private key
--sslClientPassphrase <passphrase> Specify the client certificate passphrase (for protected key)
-V, --version output the version number
-h, --help display help for command
```
# Samples
You will find some runnable examples in the installation directory under _samples_ . See [NPM documentation](https://docs.npmjs.com/cli/v7/configuring-npm/folders) about where you can find the global npm directory.
The _package.json_ file contains scripts for running the samples with correct parameter settings.
## Example 1 - Running some http requests on http-bin.
```
$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -v -ev username=foo password=bar
```
This sample use encrypted certificates.
```console
$ asm-pm-runner -l debug -dk Encrypt.4.ssl --sslClientCert file.cert.pem --sslClientKey file.key.pem --dataDir samples/ec_badssl_certs -r results/ -v samples/BadSSL.postman_collection.json
```
```console
[] [INFO] @apica-io/asm-pm-runner - @apica-io/asm-pm-runner (1.0.2) started
[] [DEBUG] @apica-io/asm-pm-runner - collection=samples/BadSSL.postman_collection.json
[] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem
[] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem
itemGroup BadSSL
item ClientBadSSL
request GET https://client.badssl.com/
response OK 200 OK 1891 { body: 687, header: 283, total: 970 }
timingPhases {
secure_handshake_ms: 339.4973890000001,
prepare: 73.57351800000015,
socket_wait_ms: 59.02749899999981,
dns_time_ms: 1147.0008660000003,
tcp_connect_ms: 171.81406999999945,
time_to_first_byte_ms: 152.74416699999983,
download_time_ms: 19.97996400000011,
process_time_ms: 1.6743750000005093,
total_time_ms: 1965.3118480000003
}
assertion Check status code is 200 true
Collection BadSSL
run.stats {
iterations: { total: 1, pending: 0, failed: 0 },
items: { total: 1, pending: 0, failed: 0 },
scripts: { total: 1, pending: 0, failed: 0 },
prerequests: { total: 1, pending: 0, failed: 0 },
requests: { total: 1, pending: 0, failed: 0 },
tests: { total: 1, pending: 0, failed: 0 },
assertions: { total: 1, pending: 0, failed: 0 },
testScripts: { total: 1, pending: 0, failed: 0 },
prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
[] [DEBUG] NewmanRunner - Clean up temporary files [
'/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem',
'/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem'
]
Result: true Requests:1, tests:1, assertions:1
[] [DEBUG] NewmanRunner - Results written to /Users/janostgren/work/node/asm-pm-runner/results/BadSSL_res.json
```
The json report is stored in the results directory. The format is complaint with Check Result Service in Apica ASM.
The json report is divided into one to many steps. A step contains requests from the postman collection.
- Default only one step with all requests is generated.
- Several steps will be generated if the collection contains *folders*. One step for each folder will be generated and step name corresponds to folder name.
- Only one level of steps is supported. Folders in folders will be named folder/subfolder. Example *root folder/sub folder 1/sub folder 2*
**asm-pm-runner** can return a custom value as the return value when running a collection. This functionality is often used in ASM when you want return something else than the total response time as the main result of the check run. What you need to do in the collection is use 2 special collection parameters and assign values to them in a JavaScript test.
Collection Variable | Description
--------------------| -----------
_Apica_ReturnValue | A numeric value used as the check value (return value)
_Apica_ReturnUnit | The unit of the customized return value
_Apica_Message | A customized message return as the message in ASM
``` JavaScript
pm.test("Status code is 200", function () {
pm.response.to.have.status(200);
});
let eventCount =pm.response.json().length;
pm.collectionVariables.set("_Apica_ReturnValue",eventCount)
pm.collectionVariables.set("_Apica_ReturnUnit","events")
pm.collectionVariables.set("_Apica_Message",`Event count = ${eventCount}`)
```
You can also return variable values in the result report. If you want suppress a variable from being included in the result report use a name starting with the "_" character. Collection and Environment variables not changed during the run of collection will be typed as input variables.
``` javascript
let json =pm.response.json();
pm.collectionVariables.set("eventName",json.name)
pm.collectionVariables.set("category",json?.category?.description|| "")
pm.collectionVariables.set("mediaLink",json?.mediaItem?.url|| "")
```
``` json
"variables": [
{
"key": "baseUrl",
"type": "url",
"usage": "input",
"value": "http://ticketmonster.apicasystem.com"
},
{
"key": "eventName",
"type": "string",
"usage": "inResponse",
"value": "Champions League"
},
{
"key": "category",
"type": "string",
"usage": "inResponse",
"value": "Sport"
},
{
"key": "mediaLink",
"type": "string",
"usage": "inResponse",
"value": "http://img.uefa.com/imgml/2016/ucl/social/og-default.jpg"
}
]
```
The name of result file will be *{collection.name}*_res.json.
The environment variable *APICA_JOB_ID* can be used to set the basename of the report file.
```json
{
"name": "BadSSL",
"type": "Postman.Collection",
"result_version": "1.0.2",
"producer": "@apica-io/asm-pm-runner 1.0.2",
"value": 1891,
"unit": "ms",
"returncode": 0,
"success": true,
"message": "Requests:1, tests:1, assertions:1",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"dns_time_ms": 1147,
"download_time_ms": 20,
"prepare": 74,
"process_time_ms": 2,
"secure_handshake_ms": 339,
"socket_wait_ms": 59,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"total_time_ms": 1965
},
"variables": [],
"steps": [
{
"name": "BadSSL",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"dns_time_ms": 1147,
"download_time_ms": 20,
"prepare": 74,
"process_time_ms": 2,
"secure_handshake_ms": 339,
"socket_wait_ms": 59,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"total_time_ms": 1965
},
"requests": [
{
"name": "ClientBadSSL",
"duration_ms": 1891,
"content_length": 687,
"start_timestamp_ms": 1629274104025,
"end_timestamp_ms": 1629274105916,
"metrics": {
"secure_handshake_ms": 339,
"prepare": 74,
"socket_wait_ms": 59,
"dns_time_ms": 1147,
"tcp_connect_ms": 172,
"time_to_first_byte_ms": 153,
"download_time_ms": 20,
"process_time_ms": 2,
"total_time_ms": 1965
},
"url": "https://client.badssl.com/",
"method": "GET",
"status": 200,
"status_text": "OK",
"request_headers": {
"User-Agent": "PostmanRuntime/7.28.2",
"Accept": "*/*",
"Cache-Control": "no-cache",
"Postman-Token": "c0cfe5c6-d962-4dcc-a2bf-bef71a09f62c",
"Host": "client.badssl.com",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive"
},
"headers": {
"Server": "nginx/1.10.3 (Ubuntu)",
"Date": "Wed, 18 Aug 2021 08:08:24 GMT",
"Content-Type": "text/html",
"Last-Modified": "Tue, 23 Feb 2021 21:28:41 GMT",
"Transfer-Encoding": "chunked",
"Connection": "keep-alive",
"ETag": "W/\"60357389-2af\"",
"Cache-Control": "no-store",
"Content-Encoding": "gzip"
}
}
],
"success": true,
"assertions": [
{
"source": "ClientBadSSL",
"description": "Check status code is 200",
"status": "info",
"expression": "Check status code is 200",
"value": true
}
]
}
]
}
```
Encryption/Decryption of certificates and environment files is supported.
You use the NPM package [cryptify](https://www.npmjs.com/package/cryptify) for decrypting and encrypting the files.
You must supply a decryption key with -dk option. The file must been encrypted with the key using the command line version of *cryptify*. Install the package globally and use the tool.
``` console
$ npm install -g cryptify
$ cryptify
Usage: cryptify [options] [command]
Options:
-v, --version Display the current version
-l, --list List available ciphers
-h, --help Display help for the command
Commands:
encrypt [options] <file...> Encrypt files(s)
decrypt [options] <file...> Decrypt files(s)
help <command> Display help for the command
Example:
$ cryptify encrypt file.txt -p 'Secret123!'
$ cryptify decrypt file.txt -p 'Secret123!'
Password Requirements:
1. Must contain at least 8 characters
2. Must contain at least 1 special character
3. Must contain at least 1 numeric character
4. Must contain a combination of uppercase and lowercase
```
## Example of encrypt a file
``` console
$ cryptify encrypt Http_bin_environment.postman_environment.json --password Encrypt.4.ssl
```
## Example with encrypted postman_environment
```console
$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -r results -l debug -v -e samples/ec_env/Http_bin_environment.postman_environment.json -dk Encrypt.4.ssl
```