sai-language/docs/25.Pipers.md

An object-oriented language designed to afford code comprehension and maintenance. Transpiles in-place to Javascript.

# 

This file is automatically generated. _Do not edit._

Unknown Heading


### | ALL - _query a collection for all values_ - pipers - ^all

Checks a collection to see if it has _all_ of multiple values. If so, returns true.

    ... | all [expr]
    

The expression to the left is the _haystack_, it is where we are looking. The expression on the right is the
(set of) _needles_ that we are looking for.

The __all__ piper examines the haystack to see if it has _all_ of the needles. If so, it returns __true__.
Otherwise, it returns __undefined__.

Example:

    set menu list tortilla, bean, lettuce, rice, meat, sauce

    debug menu | any list tofu, avocado       // undefined
    debug menu | any list fish, meat          // undefined
    debug menu | any list rice, bean          // true

You can also test against a single value:

    debug menu | any 'bean'                   // 'bean'
    debug menu | any 'gelato'                 // undefined

#### Quirks

You can use iterators on either side. Iterators on the needle side are collected, so don't try to
needle with an infinite generator.

When the haystack is an object, its keys are examined, not values.

Using an object (or any scalar value) as the needle creates a test for _that object/value_.
as if you had placed the object/value in an array by itself.

No optimizations are made that require assumptions about the sortability of either haystack or
needles, so this function is slower at the expense of being robust. If none of the needles are
found in the haystack, every needle has been compared with every item in the haystack. It might
be faster in some cases to sort and search, but this function does not do that.



### | AND - _confirms a collection_ - pipers - ^and

Examines every item in a collection, returning the collection itself if all of them are _truthy_.

    ... | and  
    

For example:

    set t: 'horse', 9, true
    set f: 0, 'goat', 99

    debug t | and
    debug f | and

    > [ 'horse', 9, true ]
    > 0

Rules for source data:

- __undefined__: returns `undefined`.

- __value__: returns the value.

- __array__: returns _the array_ itself if _every_ element in the array is _truthy_,
otherwise returns the first _falsy_ element.

- __traits__: returns _the object_ itself if _every_ value in the object is _truthy_,
otherwise returns the first _falsy_ value.

- __iterable__: yields `true` if _every_ value returned by the iterator is _truthy_,
otherwise yields the first _falsy_ value. This _vill_ lock up on infinite _truthy_
generators as it will try forever to find a _falsy_ value.

If given an iterator, __and__ returns an iterator that will yield one value (or lock up).



### | ANY - _query a collection for any of one-or-more values_ - pipers - ^any

Checks a collection to see if it has _any_ of multiple values. If so, returns the value it found first.

    ... | any [expr]
    

The expression to the left is the _haystack_, it is where we are looking. The expression on the right is the
(set of) _needles_ that we are looking for.

The __any__ piper examines the haystack to see if it has _any_ of the needles. If so, it returns the first
one it finds. Otherwise, it returns __undefined__.

Example:

    set menu list tortilla, bean, lettuce, rice, meat, sauce

    debug menu | any list tofu, avocado       // undefined
    debug menu | any list fish, meat          // 'meat'
    debug menu | any list rice, bean          // 'rice'

You can also test against a single value:

    debug menu | any 'bean'                   // 'bean'
    debug menu | any 'gelato'                 // undefined

#### Quirks

You can use iterators on either side. Iterators on the haystack side are collected, so don't try to
query with an infinite generator.

When the haystack is an object, its keys are examined, not values.

Using an object (or any scalar value) as the needle creates a test for _that object_
as if you had placed the object/value in an array by itself.

No optimizations are made that require assumptions about the sortability of either haystack or
needles, so this function is slower at the expense of being robust. If none of the needles are
found in the haystack, every needle has been compared with every item in the haystack. It might
be faster in some cases to sort and search, but this function does not do that.



### | AUDIT - _observe the elements of a collection_ - pipers - ^audit

Audit is used to pass all values in a collection into an expression, code block or method.
Audit doesn’t alter the collection, it just observes all values and keys/indices going by.

    ... | audit [expr]
    ... | audit (AS var1, var2...) [indent code block]
    ... | audit USING [function reference]
    

For example:

    set b to chain fruit
      |observe debug('Pre-sort')
      |audit debug('Fruit #${key} is ${it}')
      |by desc
      |observe debug('Post-sort')
      |audit debug('Fruit #${key} is ${it}')
    debug b

    > Pre-sort
    > Fruit #0 is Apple
    > Fruit #1 is Banana
    > Fruit #2 is Citron
    > Post-sort
    > Fruit #0 is Citron
    > Fruit #1 is Banana
    > Fruit #2 is Apple
    > [ 'Citron', 'Banana', 'Apple’ ]

This example shows how one could add instrumentation to a process in a light-weight fashion.

Neither **observe** nor **audit** alter the chained data. _A side effect of this is that
**observe** can’t be used with non-reproducible iterators as it’s impossible to statically observe
an iterator without draining it._



### | BY - _collection sorting_ - pipers - ^by

To create a sorted array, use **by**. The newly resulting array will be sorted by the specified
inline expression or code block.

    ... | by [expr]
    ... | by (AS var1, var2...) [indent code block]
    ... | by USING [function reference]
    ... | by  
    
    (optional) asc  
    
    (optional) desc  
    

**By** does not sort in-place; it always returns a new array.
If **by** is used on an iterable, it will **collect** all values before sorting.
If **by** is used on a true object, it will be converted to an array.

#### by inline

When you use the inline expression form, the provided expression is used to extract a
value from a single record, which is then compared against other records to determine the sort order.

The **it** pronoun and its **dot** scoping prefix are available within the expression.

Print a list of friends sorted alphabetically by name:

    set result to friends | by .name

    debug ( result | thru .name ).join(', ')

    > Ann, Doug, Ellie, Harry, Jenna, John, Marshal, Sara

You may sort by more than one value by adding another **by** clause. Multiple sort clauses
are handled the way a database would: if a clause provides no guidance (the values are
identical), subsequent clauses are each checked in turn.

Print sorted by length of name, then age:

    set result to friends | by .name.length by .age

    debug (result | thru '${.name}: ${.age}').join('\n')

    > Ann: 23
    > Doug: 18
    > John: 19
    > Sara: 23
    > Ellie: 22
    > Jenna: 28
    > Harry: 31
    > Marshal: 21

#### Sorting of objects by the property/key

In _inline_ expression sorting only, if your source collection is an object, you can use the __key__
pronoun as sort criteria, in addition to the __it__ pronoun and related accessors. The key is the
property name. In the example below, `rejections` is one of the keys in the folder object,
while `34` is its value.

    set folder:
      rejections 34
      acceptances 23
      pending 4
    debug folder | by key

    > [ 4, 23, 34 ]

As you can see, the key is discarded in the final array output.  If you need
different behaviour, you'll need to partition the task into simpler components, e.g. by using
__enlist__, __by__ and __thru__ in turn to preserve the data in the form you need.

Using __key__ as sort criteria on something that isn't an object will throw a runtime exception.

#### by block

When using **by** with a block of code, the pronouns change. You are given both records
under consideration just as you would using the Javascript `.sort` method. And
perhaps unsurprisingly to those who have ever seen a programming textbook, the pronouns
are the letters **a** and **b**.

Print a sorted list of ages by province:

    set result to friends | by
      return a.province <=> b.province or a.age <=> b.age

    debug (result | thru '${.name}: ${.province}, ${.age}').join('
')

    > Doug: ON, 18
    > John: ON, 19
    > Marshal: ON, 21
    > Sara: ON, 23
    > Jenna: ON, 28
    > Ellie: QC, 22
    > Ann: QC, 23
    > Harry: QC, 31

Note the __key__ pronoun is not available with block expression sorting.

#### by using

__By using__ provides the ability to use a named function for your
sorting facilitator.

    set ProvinceAge to task given a, b
     return a.province <=> b.province or a.age <=> b.age

    set result to friends | by using ProvinceAge

    // same result as previous example

#### desc (descending) _modifier_

    .. [expr] | by [expr] (desc)

Used with the **by** comprehension just like **asc** but the sort order will be highest to lowest.

    debug list Bob, Carol, Ted, Alice | by it desc

    > [ 'Ted', 'Carol', 'Bob', 'Alice' ]



### | COLLECT - _collect values from an iterator into a static list_ - pipers - ^collect

If the expression on the left is an iterator, converts it to an array/list by draining
the iterator; otherwise do nothing. If the iterator never ends, your system will
lock up until you run out of memory. (You could use a **limit** comprehension to keep that from happening.)

    ... | collect  
    

Example:

    set Odds to process
      local i to 1
      while i
        yield i
        set i + 2

    debug Odds() | collect //           !!!! this locks up the computer

    debug Odds() | limit 10 | collect

    > [ 1, 3, 5, 7, 9, 11, 13, 15, 17, 19 ]

The opposite of **collect** is **iterate**. The difference between **enlist** and
**collect** is that collect will only transform an iterator, while **enlist** will
transform all values.



### | CONCAT - _assemble two collections into a single list_ - pipers - ^concat

A list operator, not a string operator, **concat** returns a list that consists of a copy of
set 1 followed by a copy of set 2. Set 1 is not modified, a new array or iterable is returned.

    ... | concat [expr]
    

Undefined values are ignored, and scalar values are promoted to single element arrays.

As follows:

    [1, 2] concat [3, 4] -> [1, 2, 3, 4]
    [1, 2] concat [[3, 4], [5, 6]] -> [1, 2, [3, 4], [5, 6]]
    [1, 2] concat 3 -> [1, 2, 3]
    [1, 2] concat {c:3, d:4} -> [1, 2, {c:3, d:4}]
    [1, 2] concat undef -> [1, 2]

    1 concat [3, 4] -> [1, 3, 4]
    1 concat 3 -> [1, 3]
    1 concat {c:3, d:4} -> [1, {c:3, d:4}]
    1 concat undef -> [1]

    {a:1, b:2} concat [3, 4] -> [{a:1, b:2}, 3, 4]
    {a:1, b:2} concat 3 -> [{a:1, b:2}, 3]
    {a:1, b:2} concat {c:3, d:4} -> [{a:1, b:2}, {c:3, d:4}]
    {a:1, b:2} concat undef -> [{a:1, b:2}]

    undef concat undef -> undef
    undef concat 3 -> [3]
    undef concat {c:3, d:4} -> [{c:3, d:4}]
    undef concat [3, 4] -> [3, 4]

If the first set is an iterable, **concat** will return an iterable, otherwise
it returns a list (draining the second list if it is an iterable).



### | COUNT - _collection item count_ - pipers - ^count

Returns the number of elements in the collection.

    ... | count  
    

If the collection is an iterator, it will be drained.

    debug undefined count
    debug 1 count
    debug fruit count

    > 0
    > 1
    > 3



### | DELETE - _multi-attribute removal_ - pipers - ^delete

Returns a copy of the collection with the specified attributes removed.
Elements in an array are removed in order, so sequence does matter.

    ... | delete [expr]
    

Examples

    set obj to: a 1, b 2, c 3, d 4, e 5, f 6
    set obj delete 'a'
    debug obj

    > { b: 2, c: 3, d: 4, e: 5, f: 6 }

    set obj delete: 'c', 'f'
    debug obj

    > { b: 2, d: 4, e: 5 }

    set obj delete: b 'B is for Bonobo.', d 'D is for Dixie cups.'
    debug obj

    > { e: 5 }

When **delete** operates on an array, it returns a copy of the array with the elements specified removed.

    debug (list 1,2,3,4,5) delete 1

    > 1, 3, 4, 5

    debug (list 1,2,3,4,5) delete (list 1,2)

    > 1, 3, 5

Elements are removed in first->last order, so sequence does matter.

    debug (list 1,2,3,4,5) delete (list 2,1)

    > 1, 4, 5

__delete__ is also a first-class keyword used for undefining variables entirely. Please review
the related entry in the keywords documentation.



### | ENKEY - _convert a list/iterator into a set of traits_ - pipers - ^enkey

__Enkey__ transforms a list into an object (set of traits) by turning each value in the list
into a trait with the value of `true`.

    ... | enkey  
    ... | enkey [expr]
    

This is similar to __entrait__ except that instead of assuming the source list has
embedded key/value pairs, the source list is used for the keys only, and the value is
either `true` or some other expression that is calculated.

Examples:

    set a list a, b, c
    set a | enkey
    debug a

    > { 'a': true, 'b': true, 'c': true }

    set a list a, b, c
    set a | enkey '${it.toUpperCase()}${key}'
    > { 'a': 'A0', 'b': 'B1', 'c': 'C2' }

Dry, but very useful.



### | ENLIST - _convert a collection to an array_ - pipers - ^enlist

Converts various collection types into a list/array such that it should be possible to
reverse the transformation later without losing data.

    ... | enlist  
    

Here are the basic rules:

    undefined -> undefined
    value -> [ value ]
    list -> self
    object -> [ [ key, value ], [ key, value ], ... ]
    iterable -> [ value, value, value, ... ]

Note specifically that **enlist** will turn a set of key, value pairs into an array
of [key,value] arrays thus no data is lost in the conversion.

    set data to:
      cats 7
      dogs 12

    debug data
    debug data | enlist

    > { cats: 7, dogs: 12 }
    > [ [ 'cats', 7 ], [ 'dogs', 12 ] ]

Such a result can be turned back into traits with **entrait**.

The difference between **enlist** and **collect** is that **enlist** will always transform its
source data into an array/list, while **collect** will only transform an iterator.



### | ENTRAIT - _convert collections to object/traits_ - pipers - ^entrait

Converts various collection types into traits/fields using rules that allow the
transformation to be reversed later without losing data.

    ... | entrait  
    

Here are the rules:

    undefined -> undefined
    value -> { value: true }
    list -> { [0][0]: [0][1], [1][0]: [1][1], ... }
    object -> self
    iterable -> { first[0]: first[1], next[0]: next[1]... }

**Entrait** is designed primarily to losslessly restore the results of enlist and iterators
that produce arrays of key/value pairs.

    debug 'Coyote' | entrait

    > { Coyote: true }

    set data to:
      list cats, 7
      list dogs, 12

    debug data
    debug data | entrait

    > [ [ 'cats', 7 ], [ 'dogs', 12 ] ]
    > { cats: 7, dogs: 12 }

**Entrait** ignores input data that does not conform to expectations; if given a list that
does not contain two-element key/value lists, it will not create traits.

You could, however, use **enkey** to transform this data:

     debug fruit | enkey

     > { Apple: true, Banana: true, Citron: true, Durian: true }



### | EXPECTS - _rule enforcer for traits_ - pipers - ^expects

Expects compares an object againt a set of rules and returns a list of rules
that are **violated**. Thus, a successful test of expectations is an empty array.
An array with one or more elements is returned that contains a list of the
rules that were broken.

    ... | expects [expr]
    

In first-class usage as a keyword, __expects__ is used to check whether a function’s named parameters
(or any arbitrary object) has certain traits, and optionally if those traits are of a particular type.
For more on this, review the keywords documentation.

For use as a piper, __expects__ compares an object againt a set of rules and returns a list of rules
that are **violated**. Thus, a successful test of expectations is an empty array. An array with one
or more elements contains a list of the rules that were broken.

#### expects rules

Rules for **expects** take the form of a set of traits. Their names correspond to names of traits
that must be found in the expression under test. The value, if not _true_, is interpreted as an
object type; the trait must be of that type (or a child of that type).

Any violations to the rules are returned in a list; that list being made up of plain objects
with the following fields:

    .trait - name expected
    .expects - type expected
    .found - type found

Here are some examples, first as a keyword

    AddStudent task expects $name string, $age number
      @students.push copy $
    ..
    @AddStudent name 'Sally', age '12'

    > Error: SAI: parameter exception in AddStudent
    > age should be number, but it's string

Performing this same test in an ad-hoc fashion using the **expects** operator would look like this:

    set newStudent to: name 'Sally', age '12'
    set studentRules to: name 'string', age 'number'
    debug newStudent expects studentRules

    > [ { trait: 'age', expects: 'number', found: 'string' } ]

N.B. the **expects** clause rule format is identical to that of a single-line **traits** definition, with the additional requirement of a **$** before every trait name.



### | FIRST - _first value_ - pipers - ^first

Returns the first value in a list or iterator. With an expression, block or function, returns the
first value where the expression is true.

    ... | first  
    ... | first [expr]
    ... | first (AS var1, var2...) [indent code block]
    ... | first USING [function reference]
    

Example:

    debug fruit | first
    > Apple

__First__ doesn't just crop the array, it unwraps whatever value is there, as
if you'd coded `array.[0]`. __First__ unwraps iterators, returning a single
value (or undefined). If you need the iteration chain preserved, use __limit__ (in combination with
__has__ if necessary).

#### First with a comparison

If you provide a comparison expression, block, or function, __first__ will evaluate the comparison
against each element in the collection, and return the first value that the comparison returns true.
If you use __first__ on an object, the order of comparisons is undefined.

Example:

    set fruit list Apple, Banana, Cherry, Durian

    debug fruit | first .length is 6      // Banana



### | HAS - _collection filtering_ - pipers - ^has

Use __has__ to test for exclusion each object in a collection.

    ... | has [expr]
    ... | has (AS var1, var2...) [indent code block]
    ... | has USING [function reference]
    

The pipe operator **has** indicates an expression based filter using the **it**
_pronoun_ to represent the item currently under examination. Since **it** is in play,
the associated **dot** scoping prefix is also active within the expression for
easy access to item fields.

#### has inline

    debug friends | has (.province = 'QC') and (.cat or .dog)

    > [ { name: 'Ann', age: 23, cat: true, province: 'QC' } ]

Any valid expression can be used in a **has** piper, and one can refer
to values outside the expression as well.

    set rentAge to: ON 25, QC 21
    debug friends | has .age >= rentAge[.province]

    > [ { name: 'Ellie', age: 22, province: 'QC' },
    >   { name: 'Ann', age: 23, cat: true, province: 'QC' },
    >   { name: 'Harry', age: 31, province: 'QC' },
    >   { name: 'Jenna', age: 28, dog: true, province: 'ON' } ]

#### has block

**Has** can reference a block of code directly, which makes
the **it** pronoun available within that block, or be given the name of a
task (or a task definition).

You must **return** a _truthy_ (keep) or
_falsy_ (discard) value from the block so the filter can take the according
action. If you don’t return a value, all rows will be discarded.

    friends | has
      return .age >= rentAge[.province]

#### has using

You can integrate function calls by adding the
**using** keyword immediately following the __has__ keyword itself.

    set CanRent to task given row
      return row.age >= rentAge[row.province]

    debug friends | has using CanRent



### | HIGHEST - _search collection for highest value_ - pipers - ^highest

Returns the collection element with the highest value of the expression.

    ... | highest [expr]
    ... | highest USING [function reference]
    ... | highest  
    ... | highest (AS var1, var2...) [indent code block]
    

Prepares the pronouns **it** and **key** for use by the expression.

    debug friends highest .age

    > { name: 'Harry', age: 31, province: 'QC' }

__Highest__ is like a combination of __by__ and __first__, except that instead of
sorting, it just runs through the collection and finds the correct answer. (This is faster.)



### | INTO - _collection integration_ - pipers - ^into

Javascript features the array method `Array.prototype.reduce` which performs the reduction function,
and SAI extends its applicability with the **into** compherension keyword.

    ... | into [value] [expr]
    ... | into [value] (AS var1, var2...) [indent code block]
    ... | into [value] USING [function reference]
    

In addition to the item (**it**) and the item key (**key**) available in other comprehensions,
an sum variable (**sum**) is used when evaluating **into** expressions. The value
of the expression becomes the value of **sum** the next time the expression is evaluated.

#### into inline

The **sum** variable is initialized by the value following **into** — in the example below
`0` — and then each row in the collection is visited and we add `.age` to it. This adds up
the ages of every friend.

    debug friends | into 0 sum + .age

    > 185

#### into block

The **into** comprehension, like other comprehensions, has an inline version with an
expression, and a long-form which takes a block of code.

A more complex example.  See if you can suss out how it works.

    debug friends | into blank
      set sum[.province] to (self default 0) + 1

    > { ON: 5, QC: 3 }

Here’s a hint: **self** is a _pronoun_ used only in **set** statements. It is initialized to
the previous value of the variable being changed by **set**.

Another hint: **default** is an operator that evaluates to its right hand value if
and only if the left hand value is _undefined_.

Last hint: **blank** initializes an object with no traits; it is the SAI equivalent
of Javascript’s `{}`.  The comparable word for arrays/lists with no elements is **empty**;
in Javascript you’d write `[]`.

#### into using

The addition of **using** lets you call an external function.  The function must
always `return` the value you wish to be used as the sum so the value can be preserved
across function calls.  (The block version of **into** takes care of this for you.)

    set ageTotal to task given accumulator, row
      return accumulator + row.age

    debug friends | into 0 using ageTotal

    > 185



### | ITERATE - _transforms a static collection into an iterated form_ - pipers - ^iterate

Ensures the collection is an iterable. If it already is an iterable, return it unchanged.
If it has a **Iterate** method, call it and return that iterable. If it's an Array,
produce an iterator over its values. If it's an object, produce an iterator that returns
for each trait a 2-element array with trait name, trait value.
If it's just a plain value, produce an iterator that returns that value.

    ... | iterate  
    

Using **iterate** in this way allows you to guarantee that processing is being done using an
iteration rather than a static collection. To go the other way, use **collect**, which turns an
iterable into an Array.

The following example follows the one in the keyword **Iterator**:

    ply inventory | by it.1
      debug it

    > [ apple: 2, banana: 1 ]  // undesired result

    ply inventory | iterate | by it.1
      debug it

    > [ 'banana', 1 ]          // correct result
    > [ 'apple', 2 ]

The example wants to print out a sorted list of inventory items.

The undesired result comes about because **by** needs access to the entire collection
in order to sort it, so must process imperatively. Even if you give **by** an object
with a **Iterate** method, which all objects and arrays have whether you put it there
yourself or not, that method will not naturally be called.

So, **by** converts the `inventory` object into an array so it can be sorted. The
array conversion produces an array of all enumerable traits; in the case of `Tally`
objects, the only enumerable trait is `bag`, and so an array that looks like `[ bag ]`
is sorted (to no effect) and then printed one element at a time.

When you force a call to **Iterator** with the **iterate** comprehension, you get the
correct result you're expecting, as you thereby force **by** to use your custom iterator
rather than a naive array conversion.

The upshot is that right now SAI favours imperative collections rather than lazy
iterators except when specifically using the **iterate** statement or comprehension.



### | KEYS - _return a list of trait names_ - pipers - ^keys

Returns the keys (or indices) of a collection’s elements.

    ... | keys  
    

__Keys__ returns a list comprising the keys/indices of the collection it is given.

If you give it an object/traits, you get the trait names.

    debug friends | first | keys

    > [ 'name', 'age', 'cat', 'province']

If you give it an array, it returns an array of indexes.

    debug fruit | keys

    > [ 0, 1, 2 ]

Given an iterator, it returns an iterator that counts from 0 to the number of results in the source.
Of course, you lose the values, but you didn't want them, did you.



### | LAST - _last value_ - pipers - ^last

Returns the last value in a list or iterator. With an expression, block or function, returns the
first value where the expression is true.

    ... | last  
    ... | last [expr]
    ... | last (AS var1, var2...) [indent code block]
    ... | last USING [function reference]
    

If the collection is an iterator, it will be exhausted. If it's an infinite iterator, the machine will stop
responding.

    debug fruit | last

    > Citron

__Last__ doesn't just crop the array, it unwraps whatever value is there, as
if you'd coded `array.[array.length-1]`.  __Last__ unwraps iterators, returning a single
value (or undefined). If you need the iteration chain preserved, use __limit__ (in combination with
__has__ if necessary).

#### Last with a comparison

If you provide a comparison expression, block, or function, __last__ will evaluate the comparison
against each element in the collection, and return the falst value that the comparison returns true.
If you use __last__ on an object, the order of comparisons is undefined.

Example:

    set fruit list Apple, Banana, Cherry, Durian

    debug fruit | last .length is 6      // Durian

#### Quirks

Arrays: __last__ starts looking at the tail of the array.

Objects: since the keys of an object are presented with no defined order, __last__ and __first__
examine elements in an object the same way.

Iterators: __last__ starts at the beginning, compares every item, and returns the last one found
when the iterator ends. If the iterator doesn't end, __last__ will hang. If no match is found,
__last__ does not yield a value, it simply terminates. If your comparison is expensive, you may
wish to __collect__ the iterator (turning it into an array) before applying __last__; this exchanges
memory use for processing speed.



### | LIMIT - _select certain array/iterator elements_ - pipers - ^limit

Returns a subset of elements from an array, string, or iterator.

    ... | limit [expr1 (, expr2, ...)]
    

If one parameter is provided, returns the first **qty** elements in the
collection if **qty** is positive. If negative, returns the last (absolute) **qty**
number of elements.

If an **index** is supplied, return **qty** elements starting at **index**.
Negative index values do interesting things, there's a chart below.

  .. [collection] | limit [qty]
  .. [collection] | limit [index], [qty]

  .. [string] | limit [character count]
  .. [string] | limit [start character], [character count]

You always get a list back, even if just one element will be returned.

    limit -y            last y rows
    limit 0             empty
    limit undefined     everything
    limit +y            first y rows

    limit -x, -y        everything except last y rows starting x from end of list
    limit -x, 0         empty
    limit -x, undef     last x rows
    limit -x, +y        y rows starting x from end of list

    limit 0, -y         everything except last y rows
    limit 0, 0          empty
    limit 0, undef      everything
    limit 0, +y         first y rows

    limit +x, -y        everything except last y rows starting at x
    limit +x, 0         empty
    limit +x, undef     everything starting at x
    limit +x, +y        y rows starting at x

If the collection is an iterable, it will only be fetched as many times as needed.
(If you ask for elements offset from the end, the iterator will be exhausted because an
iterator’s length can only be ascertained by exhausting it.)

If the iterator is infinite and you ask for an offset from the end, you'll run out of
memory as it will cache an infinite number of intermediate results order to comply
with your impossible request. (Infinite iterators have no end.)

#### limit with strings

You can use __limit__ to extract substrings. For this purpose characters are
treated as elements in the array.

    debug 'abcdef' | limit 2,2      // cd
    debug 'abcdef' | limit -3       // def
    debug 'abcdef' | limit -3,2     // de
    debug 'abcdef' | limit 1,-1     // bcde

And so on. One "gotcha" is operator precedence; pipers have the lowest
priority, so you will need to wrap them in parenthesis if using __limit__ in any
logical expression.

    Do this:
      if '#!' is (line | limit 0,2)   // correct code

    Not this:
      if '#!' is line | limit 0,2     // bad code
      if ('#!' is line) | limit 0,2   // also bad

Another "gotcha" is the way  __limit__ intentionally changes behaviour when its
arguments change from negative to positive. If you pass calculated values that
could become negative, __limit__ may return things you don't expect.



### | LOWEST - _search collection for lowest value_ - pipers - ^lowest

Returns the collection element with the lowest value of the expression.

    ... | lowest [expr]
    ... | lowest USING [function reference]
    ... | lowest  
    ... | lowest (AS var1, var2...) [indent code block]
    

Prepares the pronouns **it** and **key** for use by the expression.

    debug friends lowest .age

    > { name: 'Doug', age: 18, province: 'ON' }

__Lowest__ is like a combination of __by__ and __first__, except that instead of
sorting, it just runs through the collection and finds the correct answer. (This is faster.)



### | OBSERVE - _inspect a value without changing it_ - pipers - ^observe

Evaluates the right expression using the left expression value as the **it** pronoun;
however, **observe** always returns the original left hand expression, no matter the result
of the right expression.

    ... | observe [expr]
    ... | observe (AS var1, var2...) [indent code block]
    ... | observe USING [function reference]
    

For example:

    set b to 'Fred'
    set c to b observe debug '${it.length} letters.'
    debug c

    > 4 letters.
    > Fred

See the example for __audit__ for a more interesting reason why __observe__ is nice.

A warning: while you can **observe** an iterable expression, all you will see is a function.
Take care not to invoke it, lest you drain it inadvertently.
Heisenberg's uncertainty principle is very much at play with iterables:
you can either have it or know what's in it.

N.B. **observe** very useful in **chain** expressions.



### | OR - _explores a collection_ - pipers - ^or

Examines every item in a collection, returning the first one found _truthy_.

    ... | or  
    

For example:

    set t: undefined, 9, true
    set f: 0, null, false

    debug t | or
    debug f | or

    > 9
    > undefined

Rules for source data:

- __undefined__: returns `undefined`.

- __value__, returns the value.

- __array__, returns the first _truthy_ element found in the array, otherwise `undefined`.
The array is searched starting at index 0.

- __traits__: returns that value if _any_ value in the collection is _truthy_,
otherwise `undefined`. The collection is searched in no defined order.

- __iterable__: yields the _first_ truthy value returned by the iterator.
Yields `undefined` if the iterator ends before one is found.
Will lock up if an infinite iterator never yields a truthy value.

If given an iterator, __or__ returns an iterator that will yield one value (or lock up).



### | REDUCE - _collection reduction_ - pipers - ^reduce

__Reduce__ transforms a collection of items into a single item by giving a function two items and
asking it to produce a single item in response. This is repeated over the entire collection.

    ... | reduce [expr]
    ... | reduce USING [function reference]
    ... | reduce (AS var1, var2...) [indent code block]
    

Let's say you have an array with four numbers:

    set b to: 4, 7, 12, 34

    set result to b | reduce as a, b
      debug "a:${a} b:${b}"
      return a+b

    debug "result: ${result}"

The __reduce__ operator is given a block of code that accepts two items, a and b. It prints them
both out, then returns their sum. This prints:

    > a:4 b:7
    > a:11 b:12
    > a:23 b:34
    > result: 57

Since there were four items in the list, the code block was called three times. The first time
with the first two elements in the array. Subsequently, it was called with the previous result
and the next element in the array.

If the collection given to __reduce__ only has one element, that element is returned without
calling your code; since there was only one item, there is no way to combine it with another.
If you give __reduce__ an empty collection, the result is __undefined__.

__Reduce__ is a classic operator for transforming sets of data, and there are far better
explanations of how it works and what it is good for to be found elsewhere. The rest of this
documentation will just show examples.

*Note that if you are looking for a reduce-like operator where you provide a starting value for
the accumulator and are given each item in the collection in turn, then look at __into__ which
does exactly what you want. __Reduce__ is kind of a different animal.*

#### reduce inline

The inline form of reduce gets two pronouns, __sum__ and __it__.

    set result to b | reduce sum+it

#### reduce block

A __reduce__ code block has the two available pronounts __sum__ and __it__, which you can
rename with an __as__ clause, as shown in the above example.

#### reduce using

The __using__ clause is available with __reduce__. You're passed two parameters, the sum
and the next element to consider.

    set Adder to task given a, b
      return a+b

    set result to b | reduce using Adder



### | REPLICATE - _create a list by copying an element_ - pipers - ^replicate

Replicate sucks, it should be removed.

    ... | replicate [expr]
    

What it does:

    set b 'a' | replicate 3
    debug b

    > [ 'a', 'a', 'a' ]

I dunno, this doesn't feel meaty.



### | SELECT - _multi-attribute query_ - pipers - ^select

Returns a new collection that is the subset of the original collection
identified by the provided list of keys/indices.

    ... | select [expr]
    

Example:

    debug fruits | select: 2, 0;

    > [ 'Citron', 'Apple' ]

    debug friends | limit 3 | thru <- it | select list name, age

    > [ { name: 'Sara', age: 32 },
        { name: 'John', age: 19 },
        { name: 'Ellie', age: 23 } ]



### | SET - _replacement operator_ - pipers - ^set

A chainable comprehension operator that allows direct reference and replacement of the incoming dataset within an expression or code block, using the  **it**  pronoun.

    ... | set [expr]
    ... | set (AS var1, var2...) [indent code block]
    ... | set USING [function reference]
    

**Set** can be used with an expression:

    debug 4 | set 5*it+2 | set it/7

    > 3.142857142857143

**Set** can use an indented code block:

    debug friends by .age | set
      set .length to 3

    > [ { name: 'Doug', age: 18, province: 'ON' },
    >    { name: 'John', age: 19, cat: true, dog: true, province: 'ON' },
    >    { name: 'Marshal', age: 21, dog: true, province: 'ON' } ]

_If you don’t specifically **return** a value or object from within an **set** code block, the original value will be used (as in the example above). In other words, there is an implicit `return it` at the end of every **set** block._

**Set** supports the **using** clause, in which case the function specified receives the original value as its first parameter, and the return value is passed forward. The two debug statements below are equivalent

    set ExtractFirst to task
      return $[0]

    debug friends #cat | set Extract(it)
    debug friends #cat | set using Extract

    > { name: 'Sara', age: 23, cat: true, province: 'ON' }
    > { name: 'Sara', age: 23, cat: true, province: 'ON' }

You must specifically return a value in the function called by **set using**.



### | THRU - _transform a collection with a map-like operation_ - pipers - ^thru

Pass each element of a collection “thru’ an expression, code block, or previously defined
function. The result of the expression becomes the new value in a copy of the collection.

    ... | thru [expr]
    ... | thru (AS var1, var2...) [indent code block]
    ... | thru USING [function reference]
    

__Thru__ returns the same type of collection it is given. An array yields an array, an object
an object, and an iterator another iterator. (There are other pipers for transforming
between types of collection.)

Conversion to uppercase:

    debug fruit | thru .toUpperCase()

    > [ 'APPLE', 'BANANA', 'CITRON' ]

A more complex formatting that could be an expression but I needed a block example:

    debug friends | has .cat | thru
      return '${key+1}) ${.name}, age ${.age}, lives in ${.province}'

    > [ '1) Sara, age 23, lives in ON',
         '2) Jon, age 19, lives in QC',
         '3) Ann, age 23, lives in QC' ]

Passing values **thru** a function:

    set rot13 to task
      set out to ''
      count $length
        set char to $charCodeAt(key)
        switch char >> 5
          case 2,3
            with char andb 31
              if it > 26
                nop
              elsif it > 13
                set char - 13
              elsif it > 0
                set char + 13
        set out + ~String.fromCharCode(char)
      return out

    debug fruit | thru using rot13

    > [ 'Nccyr', 'Onanan', 'Pvgeba' ]

The function called by the **using** variant receives three parameters for every iteration:

 - item value
 - item key
 - reference to the entire collection



### | TOTAL - _reduction with addition/concatenation_ - pipers - ^total

__Total__ is very like __into__ however the emphasis is on addition.

    ... | total [expr]
    ... | total USING [function reference]
    ... | total  
    

__Total__ is a reduction operator that is specialized for adding numbers, or concatenating strings.

    debug list a,b,c; | total

    > abc

Here, total has iterated through the three elements of the `list a, b, c` and added them together.
Because they are strings, the result is a concatenated version of the array of strings.

    debug array 5, 10, 15, 20 | total

    > 50

Similarly, total has added up the numbers.

You can pass an expression (or function) to __total__ just like with __into__, in which case the
addition is whatever value the expression returns on each iteration.

    debug fruit | total .length

    > 16

The length of the three strings in the fruit array.

#### total using

The addition of **using** lets you call an external function.  The function must
always `return` the value you wish to be added. You get two parameters, the value, and
the key.

    set ageTotal to task given row, id
      return row.age

    debug friends | total using ageTotal

    > 185



### | UPDATE - _multi-attribute update_ - pipers - ^update

Creates a new collection that is a composite of a copy of the left collection with all
of the values in the right collection overlaid. If discrete keys are available in the
left or right collection, they will be used intelligently.
Undefined values on the right will not be assigned.

    ... | update [expr]
    

Examples

    Sample data

    set list1 to list Apple, Banana, Citron
    set list2 to list Kiwi, undefined, Mango
    set traits1 to: a 1, b 2, c 3
    set traits2 to: '1' 4, d 5

    debug list1 | update list2
    > [ 'Kiwi', 'Banana', 'Mango' ]

    debug list1 | update traits1
    > [ 'Apple', 4, 'Citron', d: 5 ]

    debug traits1 | update traits2
    > { '1': 4, a: 1, b: 2, c: 3, d: 5 }

    debug traits1 | update list2
    { '0': 'Kiwi', '2': 'Mango', a: 1, b: 2, c: 3 }

__Update__ also works with iterators and treats them like arrays.
If the left side collection is **undefined**, it will be initialized **blank**.

__Update__ is handy when used in __set__ statements.

  set [variable] | update [collection]

Update a collection variable with a set of keys/values. When used like this in a __set__ statement,
the original data is updated in-place (maybe). Works on lists and traits, and accepts lists, traits
and iterators for update data.

Updating a list:

    debug fruit

    > [ 'Apple', 'Banana', 'Citron' ]

    set fruit update: '1' 'Pear', '3' 'Guava'
    debug fruit

    > [ 'Apple', 'Pear', 'Citron', 'Guava' ]

    set fruit update list Grape, undefined, Melon
    debug fruit

    > [ 'Grape', 'Pear', 'Melon', 'Guava' ]

Updating traits:

    set friend to friends.1
    debug friend

    > { name: 'John', age: 19, cat: true, dog: true, province: 'ON' }

    set friend update: name 'Jon', province 'QC'
    debug friend

    > { name: 'Jon', age: 19, cat: true, dog: true, province: 'QC' }

    set friends update: undefined, friend
    debug friends

    [ { name: 'Sara', age: 23, cat: true, province: 'ON' },
      { name: 'Jon', age: 19, cat: true, dog: true, province: 'QC' },
      { name: 'Ellie', age: 22, province: 'QC' },
      { name: 'Marshal', age: 21, dog: true, province: 'ON' },
      { name: 'Doug', age: 18, province: 'ON' },
      { name: 'Ann', age: 23, cat: true, province: 'QC' },
      { name: 'Harry', age: 31, province: 'QC' },
      { name: 'Jenna', age: 28, dog: true, province: 'ON' } ]

If the left-side collection is undefined, **update** initializes it to **blank** before merging.



### | VALUES - _return a list of collection values_ - pipers - ^values

Returns the values of a collection's elements.

    ... | values  
    

__Values__ returns a list of all of the values in a collection.

If you give it an object/traits, you get the the values.

    debug friends | first | values
    > [ 'Sara', 23, true, 'ON' ]

Given an array, you get a copy of the array.

    debug fruit | values
    > [ 'Apple', 'Banana', 'Citron' ]

With an iterator, it returns that iterator; that is, it does nothing.