binaris
Version:
Binaris SDK & CLI
1,209 lines (1,104 loc) • 39.4 kB
YAML
- test: Test help output
steps:
- in: bn --help
out: |-
Binaris command line interface
Usage: bn <command> [options]
Commands:
bn create <runtime> <function> [options] Create a function from template
bn deploy <function> [options] Deploys a function to the cloud
bn remove <function> [options] Remove a previously deployed function
bn invoke <function> [options] Invoke a Binaris function
bn list [options] List all deployed functions
bn perf <function> [options] Measure invocation latency (experimental)
bn logs <function> [options] Print the logs of a function
bn show [config] Show Binaris account configuration
bn login Login to your Binaris account using an API key and account id
bn feedback <email> <message> Provide feedback on the Binaris product
Options:
--version Show version number [boolean]
--help, -h Show help [boolean]*
Tip:
You can export BINARIS_LOG_LEVEL=[silly|debug|verbose] to view debug logs
- test: Test list help output
steps:
- in: bn list --help
out: |-
Usage: bn list [options]
Options:
--version Show version number [boolean]
--help, -h Show help [boolean]
--json Output as JSON [boolean]
- test: Test show help output
steps:
- in: bn show --help
out: |-
Usage: bn show --all | <config>
Positionals:
config What to show [choices: "accountId", "apiKey"]
Options:
--version Show version number [boolean]
--help, -h Show help [boolean]
--all, -a Show it all [boolean]
- test: Test logs help output (good-path)
steps:
- in: bn logs --help
out: |-
Usage: bn logs <function> [options]
Positionals:
function Function name [string] [required]
Options:
--version Show version number [boolean]
--help, -h Show help [boolean]
--tail, -t Outputs logs in "tail -f" fashion [boolean]
--since, -s Outputs logs after the given ISO timestamp [string]
Examples:
// retrieve all logs
bn logs foo
// tail all logs
bn logs foo --tail
// ISO
bn logs foo --since 2018-03-09T22:12:21.861Z
// unix
bn logs foo --since 1520816105798
// offset format
bn logs foo --since 3d
bn logs foo --since 13hours
bn logs foo --since 9s*
- test: Test login (good-path)
setup:
- export FUNC_NAME=login$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' node8 $FUNC_NAME
out: |-
Created function % in /home/dockeruser/test
(use "bn deploy %" to deploy the function)
- in: echo $BINARIS_API_KEY | bn login
out: |-
Please enter your Binaris API key to deploy and invoke functions.
If you don't have a key, head over to https://binaris.com to request one
*? API Key: *
*Authentication Succeeded*
(use "bn create node8 hello" to create a Node.js template function in your CWD)
- in: bn deploy $FUNC_NAME
out: |-
Deployed function %
Invoke with one of:
"bn invoke %"
"curl -H X-Binaris-Api-Key:% https://%"
- test: Test show (good-path)
steps:
- in: BINARIS_ACCOUNT_ID=1066 bn show accountId
out: "1066"
- in: BINARIS_API_KEY=17890505 bn show apiKey
out: "17890505"
- in: BINARIS_ACCOUNT_ID=1066 BINARIS_API_KEY=17890505 bn show --all
out: |-
apiKey: 17890505
accountId: 1066
- test: Test show with unknown option (bad-path)
steps:
- in: bn show accountId --json
err: |-
Usage: bn show --all | <config>
Positionals:
config What to show [choices: "accountId", "apiKey"]
Options:
--version Show version number [boolean]
--help, -h Show help [boolean]
--all, -a Show it all [boolean]
Unknown argument: json
exit: 1
- test: Test create (good-path)
steps:
- in: bn create node8 create_snowmanolympics
out: |-
Created function create_snowmanolympics in /home/dockeruser/test
(use "bn deploy %" to deploy the function)
- in: bn create node8 create_purplecannon -p /home/dockeruser/test/purplecannon
out: |-
Created function create_purplecannon in /home/dockeruser/test/purplecannon
(use "bn deploy -p /home/dockeruser/test/purplecannon create_purplecannon" to deploy the function)
- test: Test deploy (good-path)
setup:
- export FUNC_NAME=deploy$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn deploy $FUNC_NAME
out: |-
Deployed function %
Invoke with one of:
"bn invoke %"
"curl -H X-Binaris-Api-Key:% https://%"
- test: Test public deploy (good-path)
setup:
- export FUNC_NAME=public_deploy$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn deploy $FUNC_NAME
out: |-
Deployed function %
Invoke with one of:
"bn invoke %"
"curl https://%"
- test: Test deploy with TTL (number)
setup:
- export FUNC_NAME=ttl_numeric$RANDOM$RANDOM
- bn create --config.ttl 10 node8 $FUNC_NAME
- bn deploy $FUNC_NAME
steps:
- in: bn invoke $FUNC_NAME
out: |-
"Hello World!"
- in: sleep 20
- in: bn invoke $FUNC_NAME
err: |-
RequestId: %
Sorry, your function is not ready yet.
Please try again.
If this keeps happening, please contact support@binaris.com with the request ID.
exit: 1
- test: Test deploy with TTL (string)
setup:
- export FUNC_NAME=ttl_string$RANDOM$RANDOM
- bn create --config.ttl '10000ms' node8 $FUNC_NAME
- bn deploy $FUNC_NAME
steps:
- in: bn invoke $FUNC_NAME
out: |-
"Hello World!"
- in: sleep 20
- in: bn invoke $FUNC_NAME
err: |-
RequestId: *
Sorry, your function is not ready yet.
Please try again.
If this keeps happening, please contact support@binaris.com with the request ID.
exit: 1
- test: Test deploy with invalid TTL string (bad-path)
setup:
- bn create --config.ttl 'invalidttlstring' node8 functionwithinvalidttl
steps:
- in: bn deploy functionwithinvalidttl
exit: 1
err: |-
RequestId: *
Invalid request parameters (see errors field)
- test: Test feedback (good-path)
setup:
- export FEEDBACK_EMAIL=test@binaris.com
- export FEEDBACK_MESSAGE=ZL3FtCMyqeRQjP5GESTRbDRwMEjshHhjYAXHiMRZg3EOtutyWvEWcYJN7704ch3W
steps:
- in: bn feedback "$FEEDBACK_EMAIL" "$FEEDBACK_MESSAGE"
out: |-
Thank you!
- test: Test list
setup:
- export FUNC_NAME=listtestA$RANDOM$RANDOM
- export FUNC_NAME2=listtestB$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME -p 1
- bn deploy $FUNC_NAME -p 1
- bn create --config.ttl '2 hours' node8 $FUNC_NAME2 -p 2
- bn deploy $FUNC_NAME2 -p 2
cleanup:
- bn remove $FUNC_NAME
- bn remove $FUNC_NAME2
steps:
- in: bn list
out: |-
FUNCTION*LAST DEPLOYED*
listtestA*20#-#-#T#:#:#.#Z*
listtestB*20#-#-#T#:#:#.#Z*
- in: bn list --json
out: |-
*[*{"name":"listtestA%","lastDeployed":"20#-#-#T#:#:#.#Z","expiration":"20#-#-#T#:#:#.#Z"}*,*{"name":"listtestB%","lastDeployed":"20#-#-#T#:#:#.#Z","expiration":"20#-#-#T#:#:#.#Z"}*]*
- test: Test invoke (good-path)
setup:
- export FUNC_NAME=invoke$RANDOM$RANDOM
- |-
echo '{"name": "unguessable"}' > invoke.json
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
- bn deploy $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: cd /home/
- in: bn invoke $FUNC_NAME
out: |-
"Hello World!"
- in: |-
bn invoke $FUNC_NAME -d '{"name": "unguessable"}'
out: |-
"Hello unguessable!"
- in: bn invoke $FUNC_NAME -j ./dockeruser/test/invoke.json
out: |-
"Hello unguessable!"
- test: Block access to internal network
setup:
- export FUNC_NAME=internalnets$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
- npm i request-promise-native request
- |-
cat <<EOF >/home/dockeruser/test/function.js
exports.handler = async() => {
const rp = require('request-promise-native');
for (host of ['169.254.169.254', '172.17.0.1', '172.16.0.1']) {
try {
await rp.get('http://' + host);
throw new Error('should not be returned');
} catch (err) {
if (err.message.indexOf('ECONNREFUSED ' + host) === -1) {
throw err;
}
}
}
return true;
};
EOF
- bn deploy $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn invoke $FUNC_NAME
- test: Test logs (good-path)
setup:
- export FUNC_NAME=logs$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 32 | head -n 1)
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
- sed -i '7 a \ console.log(`Hello ${name}!`);' function.js
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn deploy $FUNC_NAME
out: |-
Deployed function %
Invoke with one of:
"bn invoke %"
"curl -H X-Binaris-Api-Key:%* https://%"
- in: bn invoke $FUNC_NAME
out: |-
"Hello World!"
- in: sleep 5
- in: bn logs $FUNC_NAME
out: |-
[20#-#-#T#:#:#.#Z] Hello World!
[20#-#-#T#:#:#.#Z] Function invocation took * us
- in: |-
bn invoke $FUNC_NAME -d '{"name": "again"}'
out: |-
"Hello again!"
- in: sleep 8
- in: bn logs $FUNC_NAME
out: |-
[20#-#-#T#:#:#.#Z] Hello World!
[20#-#-#T#:#:#.#Z] Function invocation took * us
[20#-#-#T#:#:#.#Z] Hello again!
[20#-#-#T#:#:#.#Z] Function invocation took * us
- in: sleep 15
- in: |-
bn invoke $FUNC_NAME -d '{"name": "for the last time"}'
out: |-
"Hello for the last time!"
- in: sleep 8
- in: bn logs $FUNC_NAME --since 10s
out: |-
[20#-#-#T#:#:#.#Z] Hello for the last time!
[20#-#-#T#:#:#.#Z] Function invocation took * us
- in: cd /home/
- in: bn logs $FUNC_NAME --since 40sec
out: |-
[20#-#-#T#:#:#.#Z] Hello again!
[20#-#-#T#:#:#.#Z] Function invocation took * us
[20#-#-#T#:#:#.#Z] Hello for the last time!
[20#-#-#T#:#:#.#Z] Function invocation took * us
- test: 'Test {PY_VERSION} E2E (good-path)'
foreach:
- PY_VERSION: python2
- PY_VERSION: pypy2
- PY_VERSION: python3
setup:
- export FUNC_NAME={PY_VERSION}$RANDOM$RANDOM
- bn create --config.ttl '2 hours' {PY_VERSION} $FUNC_NAME
- |-
cat > function.py <<EOF
def handler(body, req):
name = req.query.get('name') or body.get('name') or 'World'
out = 'Hello {}!'.format(name)
print(out)
return out
EOF
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn deploy $FUNC_NAME
out: |-
Deployed function %
Invoke with one of:
"bn invoke %"
"curl -H X-Binaris-Api-Key:% https://%"
- in: |-
bn invoke $FUNC_NAME -d '{"name": "Binaris"}'
out: |-
"Hello Binaris!"
- in: sleep 5
- in: bn logs $FUNC_NAME
out: |-
[20#-#-#T#:#:#.#Z] Hello Binaris!
[20#-#-#T#:#:#.#Z] Function invocation took * us
- test: Test deploy invoke remove cycle commands (good-path)
setup:
- export FUNC_NAME=cycle$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
steps:
- in: |-
echo 'exports.handler = () => { s = "Hello World!"; console.log(s); return s; };' >function.js
- in: bn deploy $FUNC_NAME
out: |-
Deployed function cycle#
Invoke with one of:
"bn invoke cycle#"
"curl -H X-Binaris-Api-Key:% https://%cycle%"
- in: bn invoke $FUNC_NAME
out: |-
"Hello World!"
- in: bn remove $FUNC_NAME
out: |-
Removed function %
- in: sleep 5
- in: bn logs $FUNC_NAME
out: |-
*[20#-#-#T#:#:#.#Z] Function % deployed (version digest:%)*
*[20#-#-#T#:#:#.#Z] Hello World!*
*[20#-#-#T#:#:#.#Z] Function % removed.*
- test: "envs: deploy invoke remove with secrets"
setup:
- export FUNC_NAME=secret$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' node8 $FUNC_NAME
- in: |-
cat <<EOF >/home/dockeruser/test/binaris.yml
functions:
$FUNC_NAME:
file: function.js
entrypoint: handler
runtime: node8
env:
FORWARD_ME:
DEFINED_HERE: value
BN_FUNCTION: "dont-override"
EOF
- in: |-
cat <<EOF >/home/dockeruser/test/function.js
exports.handler = () => [process.env.FORWARD_ME, process.env.DEFINED_HERE, process.env.BN_FUNCTION];
EOF
- in: FORWARD_ME=please bn deploy $FUNC_NAME
out: |-
Deployed function secret#
Invoke with one of:
"bn invoke secret#"
"curl -H X-Binaris-Api-Key:% https://%secret%"
- in: bn invoke $FUNC_NAME
out: |-
*["please","value","%secret%"]*
- test: "envs: fail to deploy if non-string in env"
steps:
- in: bn create node8 invalidenv
- in: |-
cat <<EOF >/home/dockeruser/test/binaris.yml
functions:
invalidenv:
file: function.js
entrypoint: handler
runtime: node8
env:
A_NUMBER: 6.66
EOF
- in: bn deploy invalidenv
exit: 1
err: "binaris.yml env var 'A_NUMBER' is not a string"
- test: "envs: fail to deploy if empty string in env"
steps:
- in: bn create node8 emptiness
- in: |-
cat <<EOF >/home/dockeruser/test/binaris.yml
functions:
emptiness:
file: function.js
entrypoint: handler
runtime: node8
env:
EMPTY: ""
EOF
- in: bn deploy emptiness
exit: 1
err: "Empty string env 'EMPTY' in binaris.yml is not supported"
- test: "concurrent execution model: deploy invoke multiple times"
setup:
- export FUNC_NAME=concurrent$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' node8 $FUNC_NAME -e concurrent
- in: |-
cat > function.js << EOL
function msleep(ms) {
return new Promise((resolve) => {
setTimeout(resolve, ms);
});
}
let duringFlight = false;
exports.handler = async () => {
if (duringFlight) {
return true;
} else {
duringFlight = true;
await msleep(10000);
duringFlight = false;
return false;
}
}
EOL
- in: bn deploy $FUNC_NAME
out: |-
Deployed function %
Invoke with one of:
"bn invoke %"
"curl -H X-Binaris-Api-Key:% https://%"
# 3 simultaneous invocations because testing has minBolts=2
- in: "(for i in `seq 1 3`; do bn invoke $FUNC_NAME & done) | grep true"
- test: "exclusive execution model: deploy invoke multiple times"
setup:
- export FUNC_NAME=exclusive$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' node8 $FUNC_NAME -e exclusive
- in: |-
cat > function.js << EOL
function msleep(ms) {
return new Promise((resolve) => {
setTimeout(resolve, ms);
});
}
let duringFlight = false;
exports.handler = async () => {
if (duringFlight) {
return true;
} else {
duringFlight = true;
await msleep(10000);
duringFlight = false;
return false;
}
}
EOL
- in: bn deploy $FUNC_NAME
out: |-
Deployed function %
Invoke with one of:
"bn invoke %"
"curl -H X-Binaris-Api-Key:% https://%"
# 3 simultaneous invocations because testing has minBolts=2
- in: "(for i in `seq 1 3`; do bn invoke $FUNC_NAME & done; wait) | grep -v true"
- test: Test login (bad-path)
steps:
- in: unset BINARIS_API_KEY
- in: echo 9239239 | bn login
err: Invalid API key
exit: 1
- test: Test login endpoint error (bad-path)
steps:
- in: unset BINARIS_API_KEY
- in: echo 9239239 | BINARIS_DEPLOY_ENDPOINT=fake.api.binaris.invalid. bn login
err: "Error: getaddrinfo ENOTFOUND fake.api.binaris.invalid. fake.api.binaris.invalid.:443"
exit: 1
- test: Test feedback empty message (bad-path)
setup:
- export FEEDBACK_EMAIL=test@binaris.com
steps:
- in: bn feedback "$FEEDBACK_EMAIL" ""
err: Not a valid message.
exit: 1
- test: Test feedback empty email (bad-path)
setup:
- export FEEDBACK_MESSAGE=ZL3FtCMyqeRQjP5GESTRbDRwMEjshHhjYAXHiMRZg3EOtutyWvEWcYJN7704ch3W
steps:
- in: bn feedback "" "$FEEDBACK_MESSAGE"
err: Not a valid email.
exit: 1
- test: Test create (bad-path)
steps:
- in: bn create node8 a*b*c*
err: "Invalid characters in function name 'a*b*c*'. Use only letters and digits"
exit: 1
- in: bn create node8 a_bשcd@e+
err: "Invalid characters in function name 'a_bשcd@e+'. Use only letters and digits"
exit: 1
- in: bn create node8 a=b~c,d.
err: "Invalid characters in function name 'a=b~c,d.'. Use only letters and digits"
exit: 1
- in: bn create node8 a:b?c!d-
err: "Invalid characters in function name 'a:b?c!d-'. Use only letters and digits"
exit: 1
- in: bn create
err: "*Not enough non-option arguments: got 0, need at least 2"
exit: 1
- in: bn create node8
err: "*Not enough non-option arguments: got 1, need at least 2"
exit: 1
- in: bn create abcd
err: "*Not enough non-option arguments: got 1, need at least 2"
exit: 1
- in: bn create cpp4096 abcd
err: '*Invalid values:*Argument: runtime, Given: "cpp4096", Choices:*'
exit: 1
- test: Test invoke (bad-path)
steps:
- in: bn create node8 invoke_bad -p /home/dockeruser/test/alloftheoptions
- in: bn invoke invoke_bad -j myFile.json -d data
err: Invoke flags --json(-j) and --data(-d) are mutually exclusive
exit: 1
- in: bn invoke invoke_bad -j myFile.json
err: "ENOENT: no such file or directory, open 'myFile.json'"
exit: 1
- in: bn invoke nosuchfunction
err: |-
RequestId: *
Sorry, your function is not ready yet.
Please try again.
If this keeps happening, please contact support@binaris.com with the request ID.
exit: 1
- test: Test user code timeout (bad-path)
setup:
- export FUNC_NAME=timeoutforsure$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' node8 $FUNC_NAME -e exclusive
- in: |-
cat > function.js << EOL
exports.handler = async (body, req) => {
const msleep = ms => new Promise(resolve => setTimeout(resolve, ms));
await msleep(100000);
};
EOL
- in: bn deploy $FUNC_NAME
- in: bn invoke $FUNC_NAME
err: |-
RequestId: *
Something went wrong, and it's not your fault.
If this keeps happening, please contact support@binaris.com with the request ID.
Apologies.
exit: 1
- test: Test bad user node8 code (bad-path)
setup:
- export FUNC_NAME=doomedtofailnode8_$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' node8 $FUNC_NAME
- in: |-
sed -i.bak "7i\ throw new Error('some error');" function.js
- in: bn deploy $FUNC_NAME
- in: bn invoke $FUNC_NAME
err: |-
some error
Error: some error
at Bolt.exports.handler [as userEntryPoint] (/code/function.js:7:9)
at *
at <anonymous>
exit: 1
- test: Test bad user python2 code (bad-path)
setup:
- export FUNC_NAME=doomedtofailpy2_$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' python2 $FUNC_NAME
- in: |-
sed -i.bak "2i\ sdasda" function.py
- in: bn deploy $FUNC_NAME
- in: bn invoke $FUNC_NAME
err: |-
global name 'sdasda' is not defined
File "/opt/binaris/py2/main.py", line *, in run_handler
*
, File "/code/function.py", line 2, in handler
sdasda
exit: 1
- test: Test bad user python3 code (bad-path)
setup:
- export FUNC_NAME=doomedtofailpy3_$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' python3 $FUNC_NAME
- in: |-
sed -i.bak "2i\ sdasda" function.py
- in: bn deploy $FUNC_NAME
- in: bn invoke $FUNC_NAME
err: |-
name 'sdasda' is not defined
File "/opt/binaris/py3/main.py", line *, in run_handler
*
, File "/code/function.py", line 2, in handler
sdasda
exit: 1
- test: Test bad user pypy2 code (bad-path)
setup:
- export FUNC_NAME=doomedtofailpypy2_$RANDOM$RANDOM
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn create --config.ttl '2 hours' pypy2 $FUNC_NAME
- in: |-
sed -i.bak "2i\ sdasda" function.py
- in: bn deploy $FUNC_NAME
- in: bn invoke $FUNC_NAME
err: |-
global name 'sdasda' is not defined
File "/opt/binaris/py2/main.py", line *, in run_handler
*
, File "/code/function.py", line 2, in handler
sdasda
exit: 1
- test: Test remove (bad-path)
setup:
- export FUNC_NAME=notdeployeddeployed_$RANDOM$RANDOM
steps:
- in: bn create --config.ttl '2 hours' node8 $FUNC_NAME
- in: bn remove $FUNC_NAME
err: |-
RequestId: %
Error: No such function
exit: 1
- in: bn deploy $FUNC_NAME
- in: bn remove $FUNC_NAME
out: "*Removed function %*"
- in: bn remove $FUNC_NAME
err: |-
RequestId: %
Error: No such function
exit: 1
- in: bn remove
err: "*Not enough non-option arguments: got 0, need at least 1"
out: |-
exit: 1
- test: Test logs (bad-path)
steps:
- in: bn create node8 endlessclue
- in: bn logs endlessclue --since h23sdh2
err: |-
Invalid time format "h23sdh2"
exit: 1
- in: bn logs endlessclue --since 231213daysf
err: |-
Invalid offset format, unknown unit "daysf"
exit: 1
- in: BINARIS_LOG_ENDPOINT=fake.logs.binaris.invalid. bn logs amillionamillion
err: "Error: getaddrinfo ENOTFOUND fake.logs.binaris.invalid. fake.logs.binaris.invalid.:443"
exit: 1
- test: Invalid name in binaris.yml (bad-path)
setup:
- bn create node8 pickypumpkin
# replace line 2 of binaris.yml(function name) with a bad function name
- sed -i '2 c \ inv@a-d:' binaris.yml
steps:
- in: bn deploy "inv@a-d"
err: "Invalid characters in function name 'inv@a-d'. Use only letters and digits"
exit: 1
- test: Unknown command (bad-path)
steps:
- in: bn alwaysbad
err: "*Unknown command: 'alwaysbad'"
exit: 1
- test: No permission
setup:
# aufs and fs-extra do not play nice together
- mkdir foo
- chmod -rwx foo
steps:
- in: bn create node8 orangehero -p foo/bar
err: "EACCES: permission denied, mkdir '/home/dockeruser/test/foo/bar'"
exit: 1
- test: Invocation stats
setup:
- export FUNC_NAME="invokestats$RANDOM$RANDOM"
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
- bn deploy $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn invoke $FUNC_NAME
- in: sleep 100 # Enough time to get minute-aggregated stats
- in: bn stats --since "3m"
out: |-
FUNCTION% METRIC% VALUE% SINCE% UNTIL*
invokestats% function_duration_us% #* #-#-#T#:#:#.#Z* #-#-#T#:#:#.#Z*
invokestats% function_invocations% #* #-#-#T#:#:#.#Z* #-#-#T#:#:#.#Z*
- in: bn stats --since "3m" --json
out: '{*"metrics":[*'
- test: Empty stats
steps:
- in: bn stats --until "1990-01-01T00:00:00Z"
out: No matching usage stats found
- test: Bad input to stats
steps:
- in: bn stats --since 'not a valid time'
exit: 1
err: Invalid time format "not a valid time"
# Disabled until the switch is made to yargs
#
# - test: Superfluous output(bad-path)
# steps:
# - in: bn create init init
# err: Argument "*" is not a valid input to *
# exit: 1
# - in: bn create sadmksad
# err: Argument "sadmksad" is not a valid input to create
# exit: 1
# - in: bn deploy init
# err: Argument "init" is not a valid input to deploy
# exit: 1
# - in: bn logs notlogs
# err: Argument "notlogs" is not a valid input to logs
# exit: 1
- test: No such path (bad-path)
steps:
- in: bn deploy livelyhall -p /home/dockeruser/test/bogus/comeon/really/hello.js
err: "ENOENT: no such file or directory, open '/home/dockeruser/test/bogus/comeon/really/hello.js/binaris.yml'"
exit: 1
- test: No API key or conf file (bad-path)
setup:
- unset BINARIS_API_KEY
steps:
- in: bn create node8 handsomelycalendar
- in: bn deploy handsomelycalendar
err: Binaris conf file could not be read and BINARIS_API_KEY is undefined, please use "bn login"
exit: 1
- test: Invalid API key (bad-path)
setup:
- export BINARIS_API_KEY=impossiblekey2313213
steps:
- in: bn create node8 perplexingpersimmon
- in: bn deploy perplexingpersimmon
err: |-
RequestId: *
Error: Invalid API key
exit: 1
- in: bn list
err: |-
RequestId: *
Error: Invalid API key
exit: 1
- in: BINARIS_API_KEY=boguskey bn invoke perplexingpersimmon
err: |-
RequestId: *
Unauthorized request. Make sure you're passing the right API key.
exit: 1
- test: Invalid characters in API key when invoking (bad-path)
setup:
- export BINARIS_API_KEY=יוניקוד
steps:
- in: bn invoke whatever
err: |-
Non-ASCII characters are not allowed in API key
exit: 1
- test: Invalid characters in API key when logging in (bad-path)
setup:
- export BINARIS_API_KEY=יוניקוד
steps:
- in: echo $BINARIS_API_KEY | bn login
err: |-
Non-ASCII characters are not allowed in API key
exit: 1
- test: Invalid characters in account ID from environment variable (bad-path)
setup:
- export BINARIS_ACCOUNT_ID=יוניקוד
steps:
- in: bn invoke whatever
err: |-
Non-ASCII characters are not allowed in account ID
exit: 1
- test: Deploy of 200MB function succeeds
setup:
- export FUNC_NAME=largeData200MB$RANDOM$RANDOM
- bn create --config.ttl '2 hours' python2 $FUNC_NAME
- dd if=/dev/urandom bs=1048576 count=200 of=large.data
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn deploy $FUNC_NAME
- in: bn invoke $FUNC_NAME
out: |-
"Hello World!"
- test: Deploy of 300MB function fails with significant error message (bad-path)
setup:
- export FUNC_NAME=largeData300MB$RANDOM$RANDOM
- bn create --config.ttl '2 hours' python2 $FUNC_NAME
- dd if=/dev/urandom bs=1048576 count=300 of=very.large.data
steps:
- in: bn deploy $FUNC_NAME
err: |-
RequestId: %
Error: Payload too large
exit: 1
- test: Test if imagemagick exists in runtime environment
setup:
- export FUNC_NAME=largeimagemagick$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
- npm i shelljs
- |-
cat <<EOF >/home/dockeruser/test/function.js
const shell = require('shelljs');
exports.handler = async() => {
return shell.exec('convert --version');
};
EOF
- bn deploy $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn invoke $FUNC_NAME
out: |-
"Version: ImageMagick %
- test: Test invalid option to command (bad-path)
steps:
- in: bn invoke hello --notarealarg
err: |-
Usage: bn invoke <function> [options]
Positionals:
function Function name [string] [required]
Options:
--version Show version number [boolean]
--help, -h Show help [boolean]
--json, -j Path to file containing JSON data [string]
--data, -d Data to send with invocation [string]
Examples:
// invoke a function
bn invoke foo
// invoke using JSON file data
bn invoke foo --json ./path/to/myfile.json
// invoke foo and send JSON data in the body
bn invoke foo --data '{ "name": "helloworld" }'
Unknown argument: notarealarg
exit: 1
- test: Stats help
steps:
- in: bn stats --help
out: |-
Usage: bn stats [options]
Options:
--version Show version number [boolean]
--help, -h Show help [boolean]
--since, -s Output statistics after given ISO timestamp (inclusive) [string]
--until, -u Output statistics until given ISO timestamp (non-inclusive) [string]
--json Output as JSON [boolean]
Examples:
// Retrieve all usage statistics of the account
bn stats
// Retrieve all statistics since the timestamp until now (~1 minute)
bn stats --since 2018-03-09T22:12:21.861Z
// Statistics over the last 24h
bn stats --since 1d
// Retrieve all statistics of a certain month
bn stats --since 2018-03-01T00:00:00Z --until 2018-04-01T00:00:00Z
- test: Test deploy of long name (good-path)
cleanup:
- bn remove $LONG_NAME
setup:
- export LONG_NAME=binarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinaris$RANDOM$RANDOM
steps:
- in: bn create --config.ttl '2 hours' node8 $LONG_NAME
- in: bn deploy $LONG_NAME
- in: bn invoke $LONG_NAME
out: |-
*"Hello World!"*
- test: Test deploy of somewhat long name (bad-path)
setup:
- export SERVER_LONG_NAME=binarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinaris
steps:
- in: bn create node8 $SERVER_LONG_NAME
- in: bn deploy $SERVER_LONG_NAME
exit: 1
err: "Function name cannot be longer than 200"
- test: Test remove of somewhat long name (bad-path)
steps:
- in: bn remove binarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinaris
exit: 1
err: "Function name cannot be longer than 200"
- test: Test deploy of very long name (bad-path)
setup:
- export LOCAL_LONG_NAME=binarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinarisbinaris1111
steps:
- in: bn create node8 $LOCAL_LONG_NAME
- in: bn deploy $LOCAL_LONG_NAME
exit: 1
err: "ENAMETOOLONG: name too long, open '/home/dockeruser/test/.binaris/binaris*1111.tgz'"
- test: Test perf 1 second (good-path)
setup:
- export FUNC_NAME=measure1sec$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
- bn deploy $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn perf -t 1 $FUNC_NAME
out: |-
Running performance test on function measure1sec#.
Executing 5000 invocations with 1 "thread" up to 1 second.
Stand by for results...
Perf summary
============
Total time % s*
Invocations %
Errors %
Rate % rps*
Latencies
=========
Mean % ms*
Min % ms*
Max % ms*
50% % ms*
90% % ms*
95% % ms*
99% % ms*
- test: Test perf 100 invocations (good-path)
setup:
- export FUNC_NAME=measure100_$RANDOM$RANDOM
- bn create --config.ttl '2 hours' node8 $FUNC_NAME
- bn deploy $FUNC_NAME
cleanup:
- bn remove $FUNC_NAME
steps:
- in: bn perf -n 100 $FUNC_NAME
out: |-
Running performance test on function measure100_#.
Executing 100 invocations with 1 "thread".
Stand by for results...
Perf summary
============
Total time * s*
Invocations *
Errors *
Rate * rps*
Latencies
=========
Mean * ms*
Min * ms*
Max * ms*
50% * ms*
90% * ms*
95% * ms*
99% * ms*
- test: empty env
cleanup:
- bn remove emptyenv
steps:
- in: bn create node8 emptyenv
- in: |-
cat <<EOF >>/home/dockeruser/test/binaris.yml
env:
nosuchenv:
EOF
- in: bn deploy emptyenv
out: |-
Non existing env var 'nosuchenv' is ignored*
- in: |-
nosuchenv= bn deploy emptyenv
err: |-
Empty existing env var 'nosuchenv' is not supported
exit: 1
- test: non dict env
steps:
- in: bn create node8 nondictenv
- in: |-
cat <<EOF >>/home/dockeruser/test/binaris.yml
env:
value
EOF
- in: bn deploy nondictenv
exit: 1
err: "binaris.yml section <env> is not a dictionary"
- test: empty binaris.yml
steps:
- in: |-
touch /home/dockeruser/test/binaris.yml
- in: bn deploy emptyyml
exit: 1
err: "binaris.yml is missing section <functions>"
- test: empty functions yml
steps:
- in: |-
echo "functions:" >/home/dockeruser/test/binaris.yml
- in: bn deploy emptyfunctions
exit: 1
err: "binaris.yml section <functions> is not a dictionary"
- test: numeric functions yml
steps:
- in: |-
echo "functions: 3" >/home/dockeruser/test/binaris.yml
- in: bn deploy numericfunctions
exit: 1
err: "binaris.yml section <functions> is not a dictionary"
- test: bad runtime yml
steps:
- in: |-
cat <<EOF >>/home/dockeruser/test/binaris.yml
functions:
noRuntime:
file: function.js
entrypoint: handler
runtime:
foo: 123
EOF
- in: bn deploy noRuntime
exit: 1
err: "binaris.yml field <runtime> is not a string"
- test: existing files
steps:
- in: touch binaris.yml
- in: bn create node8 alreadyExists
exit: 1
err: "/home/dockeruser/test/binaris.yml already exists"
- test: create configuration options (good-path)
steps:
- in: bn create node8 testConfOpts --config.executionModel=exclusive --config.env.PATH --config.env.FOO=bar
- in: |-
cat <<EOF >>/home/dockeruser/test/expected.yml
functions:
testConfOpts:
file: function.js
entrypoint: handler
executionModel: exclusive
runtime: node8
env:
PATH: null
FOO: bar
EOF
- in: diff binaris.yml expected.yml
- test: invalid create configuration options (bad-path)
steps:
- in: bn create node8 testBadConfOpts --config=bad
exit: 1
err: "Non object create configuration options: bad"
- in: bn create node8 testBadConfOpts --config
exit: 1
err: "Non object create configuration options: true"
- in: bn create node8 testBadConfOpts --config=bad --config.foo=bar
exit: 1
err: "Non object create configuration options: bad"
- in: bn create node8 testBadConfOpts --config --config.foo=bar
exit: 1
err: "Non object create configuration options: true"
- in: bn create node8 testArrayOpts --config=aaa --config=bbb
exit: 1
err: "Non object create configuration options: aaa,bbb"