@travetto/schema
Version:
Data type registry for runtime validation, reflection and binding.
391 lines (325 loc) • 16.9 kB
Markdown
<!-- This file was generated by @travetto/doc and should not be modified directly -->
<!-- Please modify https://github.com/travetto/travetto/tree/main/module/schema/DOC.tsx and execute "npx trv doc" to rebuild -->
# Schema
## Data type registry for runtime validation, reflection and binding.
**Install: @travetto/schema**
```bash
npm install @travetto/schema
# or
yarn add @travetto/schema
```
This module's purpose is to allow for proper declaration and validation of data types, in the course of running a program. The framework defined here, is leveraged in the [Configuration](https://github.com/travetto/travetto/tree/main/module/config#readme "Configuration support"), [Command Line Interface](https://github.com/travetto/travetto/tree/main/module/cli#readme "CLI infrastructure for Travetto framework"), [RESTful API](https://github.com/travetto/travetto/tree/main/module/rest#readme "Declarative api for RESTful APIs with support for the dependency injection module."), [OpenAPI Specification](https://github.com/travetto/travetto/tree/main/module/openapi#readme "OpenAPI integration support for the Travetto framework") and [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations.") modules. The schema is the backbone of all data transfer, as it helps to provide validation on correctness of input, whether it is a rest request, command line inputs, or a configuration file.
This module provides a mechanism for registering classes and field level information as well the ability to apply that information at runtime.
## Registration
The registry's schema information is defined by [Typescript](https://typescriptlang.org) AST and only applies to classes registered with the [@Schema](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/schema.ts#L14) decoration.
### Classes
The module utilizes AST transformations to collect schema information, and facilitate the registration process without user intervention. The class can also be described using providing a:
* `title` - definition of the schema
* `description` - detailed description of the schema
* `examples` - A set of examples as [JSON](https://www.json.org) or [YAML](https://en.wikipedia.org/wiki/YAML)
The `title` will be picked up from the [JSDoc](http://usejsdoc.org/about-getting-started.html) comments, and additionally all fields can be set using the [@Describe](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/common.ts#L15) decorator.
**Code: Sample User Schema**
```typescript
import { Schema } from '@travetto/schema';
@Schema()
export class User {
name: string;
age: number;
favoriteFood?: 'pizza' | 'burrito' | 'salad';
height?: `${number}${'m' | 'ft'}`;
}
```
From this schema, the registry would have the following information:
**Config: User schemas as a YAML file**
```yaml
User:
fields:
-
name: name
type": string
required: true
-
name: age
type: number
required: true
-
name: favoriteFood
type: string
required: false
allowedValues: ["pizza", "burrito", "salad" ]
```
### Fields
This schema provides a powerful base for data binding and validation at runtime. Additionally there may be types that cannot be detected, or some information that the programmer would like to override. Below are the supported field decorators:
* [@Field](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L25) defines a field that will be serialized.
* [@Required](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L65) defines a that field should be required
* [@Enum](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L72) defines the allowable values that a field can have
* [@Match](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L93) defines a regular expression that the field value should match
* [@MinLength](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L101) enforces min length of a string
* [@MaxLength](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L111) enforces max length of a string
* [@Min](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L101) enforces min value for a date or a number
* [@Max](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L111) enforces max value for a date or a number
* [@Email](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L138) ensures string field matches basic email regex
* [@Telephone](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L145) ensures string field matches basic telephone regex
* [@Url](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L152) ensures string field matches basic url regex
* [@Ignore](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L191) exclude from auto schema registration
* [@Integer](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L166) ensures number passed in is only a whole number
* [@Float](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L172) ensures number passed in allows fractional values
* [@Currency](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L184) provides support for standard currency
* [@Text](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L80) indicates that a field is expecting natural language input, not just discrete values
* [@LongText](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L85) same as text, but expects longer form content
* [@Readonly](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L52) defines a that field should not be bindable external to the class
* [@Writeonly](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L46) defines a that field should not be exported in serialization, but that it can be bound to
* [@Secret](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L58) marks a field as being sensitive. This is used by certain logging activities to ensure sensitive information is not logged out.
* [@Specifier](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L200) attributes additional specifiers to a field, allowing for more specification beyond just the field's type.
* [@SubTypeField](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/field.ts#L206) allows for promoting a given field as the owner of the sub type discriminator (defaults to `type`).
Additionally, schemas can be nested to form more complex data structures that are able to bound and validated.
Just like the class, all fields can be defined with
* `description` - detailed description of the schema
* `examples` - A set of examples as [JSON](https://www.json.org) or [YAML](https://en.wikipedia.org/wiki/YAML)
And similarly, the `description` will be picked up from the [JSDoc](http://usejsdoc.org/about-getting-started.html) comments, and additionally all fields can be set using the [@Describe](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/common.ts#L15) decorator.
### Parameters
Parameters are available in certain scenarios (e.g. [RESTful API](https://github.com/travetto/travetto/tree/main/module/rest#readme "Declarative api for RESTful APIs with support for the dependency injection module.") endpoints and [Command Line Interface](https://github.com/travetto/travetto/tree/main/module/cli#readme "CLI infrastructure for Travetto framework") main methods). In these scenarios, all of the field decorators are valid, but need to be called slightly differently to pass the typechecker. The simple solution is to use the `Arg` field of the decorator to convince Typescript its the correct type.
**Code: Sample Parameter Usage**
```typescript
import { Match, Min } from '@travetto/schema';
const NAME_REGEX = /[A-Z][a-z]+(\s+[A-Z][a-z]+)*/;
export class ParamUsage {
main(@Match(NAME_REGEX) name: string, @Min(20) age?: number) {
console.log('Valid name and age!', { name, age });
}
}
```
## Binding/Validation
At runtime, once a schema is registered, a programmer can utilize this structure to perform specific operations. Specifically binding and validation.
### Binding
Binding is a very simple operation, as it takes in a class registered as as [@Schema](https://github.com/travetto/travetto/tree/main/module/schema/src/decorator/schema.ts#L14) and a JS object that will be the source of the binding. Given the schema:
**Code: Sub Schemas via Address**
```typescript
import { Schema, Integer } from '@travetto/schema';
@Schema()
export class Address {
street1: string;
street2: string;
}
@Schema()
export class Person {
name: string;
@Integer() age: number;
address: Address;
}
```
A binding operation could look like:
**Code: Binding from JSON to Schema**
```typescript
import { Person } from './person';
export function Test(): Person {
return Person.from({
name: 'Test',
age: 19.999978,
address: {
street1: '1234 Fun',
street2: 'Unit 20'
}
});
}
```
and the output would be a `Person` instance with the following structure
**Terminal: Sample data output after binding**
```bash
$ trv main doc/person-output.ts
Person {
name: 'Test',
age: 19,
address: Address { street1: '1234 Fun', street2: 'Unit 20' }
}
```
**Note**: Binding will attempt to convert/coerce types as much as possible to honor the pattern of Javascript and it's dynamic nature.
### Validation
Validation is very similar to binding, but instead of attempting to assign values, any mismatch or violation of the schema will result in errors. All errors will be collected and returned. Given the same schema as above,
**Code: Sub Schemas via Address**
```typescript
import { Schema, Integer } from '@travetto/schema';
@Schema()
export class Address {
street1: string;
street2: string;
}
@Schema()
export class Person {
name: string;
@Integer() age: number;
address: Address;
}
```
But now with an invalid json object
**Code: Read Person, and validate**
```typescript
import { SchemaValidator } from '@travetto/schema';
import { Person } from './person';
export async function validate(): Promise<void> {
const person = Person.from({
name: 'Test',
age: 'abc',
address: {
street1: '1234 Fun'
}
});
await SchemaValidator.validate(Person, person);
}
```
would produce an exception similar to following structure
**Terminal: Sample error output**
```bash
$ trv main doc/person-invalid-output.ts
Validation Failed {
"message": "Validation errors have occurred",
"category": "data",
"type": "ValidationResultError",
"at": "2029-03-14T04:00:00.618Z",
"details": {
"errors": [
{
"kind": "type",
"type": "number",
"message": "age is not a valid number",
"path": "age"
},
{
"kind": "required",
"active": true,
"message": "address.street2 is required",
"path": "address.street2"
}
]
}
}
```
### Custom Validators
Within the schema framework, it is possible to add custom validators class level. This allows for more flexibility when dealing with specific situations (e.g. password requirements or ensuring two fields match)
**Code: Password Validator**
```typescript
import { Schema, Validator, ValidationError } from '@travetto/schema';
const passwordValidator = (user: User): ValidationError | undefined => {
const p = user.password;
const hasNum = /\d/.test(p);
const hasSpecial = /[!@#$%%^&*()<>?/,.;':"']/.test(p);
const noRepeat = !/(.)(\1)/.test(p);
if (!hasNum || !hasSpecial || !noRepeat) {
return {
kind: 'password-rules',
path: 'password',
message: 'A password must include at least one number, one special char, and have no repeating characters'
};
}
};
@Schema()
@Validator(passwordValidator)
class User {
password: string;
}
```
When the validator is executed, it has access to the entire object, and you can check any of the values. The validator expects an object of a specific structure if you are looking to indicate an error has occurred.
**Code: Validation Error Structure**
```typescript
export interface ValidationError {
/**
* The error message
*/
message: string;
/**
* The object path of the error
*/
path: string;
/**
* The kind of validation
*/
kind: ValidationKind;
/**
* The value provided
*/
value?: unknown;
/**
* Regular expression to match
*/
re?: string;
/**
* Number to compare against
*/
n?: number | Date;
/**
* The type of the field
*/
type?: string;
}
```
## Custom Types
When working with the schema, the basic types are easily understood, but some of [Typescript](https://typescriptlang.org)'s more complex constructs are too complex to automate cleanly.
To that end, the module supports two concepts:
### Type Adapters
This feature is meant to allow for simple Typescript types to be able to be backed by a proper class. This is because all of the typescript type information disappears at runtime, and so only concrete types (like classes) remain. An example of this, can be found with how the [Data Model Querying](https://github.com/travetto/travetto/tree/main/module/model-query#readme "Datastore abstraction for advanced query support.") module handles geo data.
**Code: Simple Custom Type**
```typescript
import { DataUtil } from '@travetto/schema';
/**
* @concrete #PointImpl
*/
export type Point = [number, number];
const INVALID = Symbol.for('invalid-point');
export class PointImpl {
static validateSchema(input: unknown): 'type' | undefined {
const ret = this.bindSchema(input);
return ret !== INVALID && ret && !isNaN(ret[0]) && !isNaN(ret[1]) ? undefined : 'type';
}
static bindSchema(input: unknown): [number, number] | typeof INVALID | undefined {
if (Array.isArray(input) && input.length === 2) {
const [a, b] = input.map(x => DataUtil.coerceType(x, Number, false));
return [a, b];
} else {
return INVALID;
}
}
}
```
What you can see here is that the `Point` type is now backed by a class that supports:
* `validateSchema` - Will run during validation for this specific type.
* `bindSchema` - Will run during binding to ensure correct behavior.
**Code: Simple Custom Type Usage**
```typescript
import { Schema } from '@travetto/schema';
import { Point } from './custom-type';
@Schema()
export class LocationAware {
name: string;
point: Point;
}
```
All that happens now, is the type is exported, and the class above is able to properly handle point as an `[x, y]` tuple. All standard binding and validation patterns are supported, and type enforcement will work as expected.
**Terminal: Custom Type Validation**
```bash
$ trv main doc/custom-type-output.ts
Validation Failed {
"message": "Validation errors have occurred",
"category": "data",
"type": "ValidationResultError",
"at": "2029-03-14T04:00:00.837Z",
"details": {
"errors": [
{
"kind": "type",
"type": "PointImpl",
"message": "point is not a valid PointImpl",
"path": "point"
}
]
}
}
```
## Data Utilities
Data utilities for binding values, and type conversion. Currently [DataUtil](https://github.com/travetto/travetto/tree/main/module/schema/src/data.ts#L11) includes:
* `deepAssign(a, b, mode ?)` which allows for deep assignment of `b` onto `a`, the `mode` determines how aggressive the assignment is, and how flexible it is. `mode` can have any of the following values:
* `loose`, which is the default is the most lenient. It will not error out, and overwrites will always happen
* `coerce`, will attempt to force values from `b` to fit the types of `a`, and if it can't it will error out
* `strict`, will error out if the types do not match
* `coerceType(input: unknown, type: Class<unknown>, strict = true)` which allows for converting an input type into a specified `type` instance, or throw an error if the types are incompatible.
* `shallowClone<T = unknown>(a: T): T` will shallowly clone a field
* `filterByKeys<T>(obj: T, exclude: (string | RegExp)[]): T` will filter a given object, and return a plain object (if applicable) with fields excluded using the values in the `exclude` input