dtl-js/docs/QuickStart.md

Data Transformation Language - JSON templates and data transformation

# DTL Quick Start

## What DTL is for

DTL, or *Data Transformation Language*, is built for transforming and
manipulating data structures. At its most basic level, **DTL can act as a
templating tool for JSON**, enabling straightforward data reformatting and
restructuring.  However, **DTL's capabilities extend far beyond simple
templating**. DTL is ideal for complex tasks such as data **extraction,
conversion, and preparation for analysis or reporting**. Its simple syntax and
robust functions facilitate the clear and concise expression of complex data
transformations, making DTL an **extremely useful tool** for a wide range of data
processing needs.

## Basic Structure

In DTL, transforms are generally defined within a **JSON object**, which
forms the transform library. Each key in this JSON object is a distinct
transform, functioning like a template or function for how input data should
be outputted. The `out` key is special as it's the default transform used by
`DTL.apply()`, akin to the `main` function in other languages. 

In most cases, the value of `out` will be an object that mirrors your
desired output structure, guiding the transformation of input data to this
format.  This library can have multiple transforms, each capable of
referencing and calling one another. When a transform is executed through
`DTL.apply()`, the provided data is represented as `$.` within the
transform.

## Syntax Essentials

### 1. Happy Tags

Every **DTL expression must be enclosed in happy tags**: `(: :)`. Happy tags
tell DTL to do something. Anything not in Happy tags is passed through
untouched. Happy tags are called that because they resemble happy emoji.

### 2. Accessing data

Reference data using **dot notation**, with the root represented as `$.`
For nested data access, use dot notation, e.g., `$user.address.street` 
Data can be referenced with or without a leading `.`, in other words, 
`$.user` and `$user` are equivalent.

### 3. No Commas 

In DTL expressions, **commas are treated like whitespace** and are not necessary.

### 4. Helper Functions and Transform Calls

Use **built-in helper functions** for complex operations, e.g.,
`math.round($number)`. To **call another transform** in the library, use
`($input_data -> transform_name)`. In this example, `$input_data` would be
sent to `transform_name` and the result would be the new data.

**Operations look like this**: `$data op $otherdata`, e.g., `$number1 + $number2`.

### 5. Parentheses for Order

Use **parentheses** to control the order that things should be done, e.g.,
`($number1 + $number2) * $number3`. DTL understands mathematical order of
operations, but parenthesis make it clearer.

### 6. Conditional Syntax

For conditionals, use `?(: $condition $truevalue $falsevalue)`.

### 7. Static and Dynamic Values

A mix of **static (literal values) and dynamic (DTL expressions)** is common in transforms. Only dynamic expressions use happy tags.

### 8. Iteration

You can loop or iterate over lists of data. In all helpers that iterate,
like `map` or `grep`, use **`$item`, `$index`, and `$all`** for referencing.
`$item` is the current item.  `$index` is the index of the current item and
`$all` is the entire list or object.

### 9. Scope

Transforms only have access to what was given to them (from `DTL.apply()` or
another transform in the library) and have no way to access information
other than that. Within a transform, `$.` refers to the input data 
of that transform.

### 10. The concept of empty

In DTL, the concept of **empty** is key. Empty in DTL means the data has no
meaningful value, e.g. undefined, null, an empty string, array, or object.
The `empty()` function returns true if a value is 'empty' 

Complementing this, the `fne()` (**First Non-Empty**) function scans a list of
values and returns the first one that is not considered empty, streamlining
the selection of valid data from multiple sources. It looks like this:
`fne($first_place_to_look $second_place_to_look defaultvalue)` 

## Example in JSON

```json
{
  "out": {
    "name": "(: fne( $display_name $first_name 'unknown user' ) :)",
    "calculatedValue": "(: $value1 + $value2 :)",
    "staticValue": "This is a static string",
    "address": "(: $. -> formatAddress :)"
  },
  "formatAddress": {
    "street": "(: $address1 :)",
    "unit": "(: $address2 :)",
    "city": "(: $addr_city :)",
    "state": "(: $addr_state :)",
    "postal": "(: $addr_zip :)"
  }
}
```

In this JSON example, `name` and `calculatedValue` are defined with DTL
expressions enclosed in happy tags, while `staticValue` is a static value.
The primary `out` transform delegates to the `formatAddress` transform for
structuring the `address` field, demonstrating how transforms can be used to
compose and define the desired output structure in DTL.