exthos

<!DOCTYPE html><html class="default" lang="en"><head><meta charSet="utf-8"/><meta http-equiv="x-ua-compatible" content="IE=edge"/><title>exthos</title><meta name="description" content="Documentation for exthos"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="stylesheet" href="assets/style.css"/><link rel="stylesheet" href="assets/highlight.css"/><script async src="assets/search.js" id="search-script"></script></head><body><script>document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os"</script><header class="tsd-page-toolbar"> <div class="tsd-toolbar-contents container"> <div class="table-cell" id="tsd-search" data-base="."> <div class="field"><label for="tsd-search-field" class="tsd-widget search no-caption"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><path d="M15.7824 13.833L12.6666 10.7177C12.5259 10.5771 12.3353 10.499 12.1353 10.499H11.6259C12.4884 9.39596 13.001 8.00859 13.001 6.49937C13.001 2.90909 10.0914 0 6.50048 0C2.90959 0 0 2.90909 0 6.49937C0 10.0896 2.90959 12.9987 6.50048 12.9987C8.00996 12.9987 9.39756 12.4863 10.5008 11.6239V12.1332C10.5008 12.3332 10.5789 12.5238 10.7195 12.6644L13.8354 15.7797C14.1292 16.0734 14.6042 16.0734 14.8948 15.7797L15.7793 14.8954C16.0731 14.6017 16.0731 14.1267 15.7824 13.833ZM6.50048 10.499C4.29094 10.499 2.50018 8.71165 2.50018 6.49937C2.50018 4.29021 4.28781 2.49976 6.50048 2.49976C8.71001 2.49976 10.5008 4.28708 10.5008 6.49937C10.5008 8.70852 8.71314 10.499 6.50048 10.499Z" fill="var(--color-text)"></path></svg></label><input type="text" id="tsd-search-field" aria-label="Search"/></div> <ul class="results"> <li class="state loading">Preparing search index...</li> <li class="state failure">The search index is not available</li></ul><a href="index.html" class="title">exthos</a></div> <div class="table-cell" id="tsd-widgets"><a href="#" class="tsd-widget menu no-caption" data-toggle="menu" aria-label="Menu"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><rect x="1" y="3" width="14" height="2" fill="var(--color-text)"></rect><rect x="1" y="7" width="14" height="2" fill="var(--color-text)"></rect><rect x="1" y="11" width="14" height="2" fill="var(--color-text)"></rect></svg></a></div></div></header> <div class="container container-main"> <div class="col-8 col-content"> <div class="tsd-page-title"> <h2>exthos</h2></div> <div class="tsd-panel tsd-typography"> <a href="#exthos" id="exthos" style="color: inherit; text-decoration: none;"> <h1>exthos</h1> </a> <p><img src="https://raw.githubusercontent.com/exthosdev/exthos/main/docs/media/logo_exthos.gif" alt="logo"></p> <blockquote> <p>stream processing in NodeJS using the power of Golang</p> </blockquote> <a href="#project-status" id="project-status" style="color: inherit; text-decoration: none;"> <h2>Project status</h2> </a> <p>exthos is <code>incubating</code>, i.e. while the feature list implementation is a WIP, the project <strong>can be used in production with existing features</strong></p> <table> <thead> <tr> <th>API</th> <th>Status</th> <th>Comments</th> </tr> </thead> <tbody><tr> <td>Low level APIs</td> <td>Stable</td> <td>Not all components are implemented yet</td> </tr> <tr> <td>High level APIs</td> <td>Early days</td> <td>The API is not stable and far for being complete. Use with caution</td> </tr> </tbody></table> <a href="#website" id="website" style="color: inherit; text-decoration: none;"> <h2>Website</h2> </a> <p><a href="https://exthosdev.github.io/exthos/">exthos</a></p> <p>Click to <a href="#in-practice">Jump staight to installation and usage</a></p> <a href="#table-of-contents" id="table-of-contents" style="color: inherit; text-decoration: none;"> <h2>Table of Contents</h2> </a> <ul> <li><a href="#exthos">exthos</a><ul> <li><a href="#project-status">Project status</a></li> <li><a href="#website">Website</a></li> <li><a href="#table-of-contents">Table of Contents</a></li> </ul> </li> <li><a href="#theory">Theory</a><ul> <li><a href="#motivation">Motivation</a></li> <li><a href="#how-exthos-works">How exthos works</a></li> <li><a href="#how-to-use-exthos">How to use exthos</a><ul> <li><a href="#overview-low-level-apis">Overview: Low level APIs</a></li> <li><a href="#overview-high-level-apis">Overview: High level APIs</a></li> </ul> </li> <li><a href="#components">Components</a><ul> <li><a href="#inputs">Inputs</a></li> <li><a href="#outputs">Outputs</a></li> <li><a href="#processors">Processors</a></li> <li><a href="#messages">Messages</a></li> </ul> </li> </ul> </li> <li><a href="#in-practice">In practice</a><ul> <li><a href="#installation">Installation</a></li> <li><a href="#usage">Usage</a></li> <li><a href="#using-high-level-apis-fromviato">Using high level APIs: from/via/to</a></li> <li><a href="#othernon-functional-features">Other/non-functional features</a><ul> <li><a href="#configuration">Configuration</a></li> <li><a href="#labels">Labels</a></li> <li><a href="#error-handling">Error handling</a></li> <li><a href="#logging">Logging</a></li> <li><a href="#events">Events</a></li> <li><a href="#metrics">Metrics</a></li> <li><a href="#tracing-and-telemetry">Tracing and Telemetry</a></li> </ul> </li> </ul> </li> <li><a href="#compatability--support">Compatability & Support</a></li> <li><a href="#acknowledgement">Acknowledgement</a></li> </ul> <hr> <a href="#theory" id="theory" style="color: inherit; text-decoration: none;"> <h1>Theory</h1> </a> <a href="#motivation" id="motivation" style="color: inherit; text-decoration: none;"> <h2>Motivation</h2> </a> <p><strong><code>exthos</code></strong> brings together <code>javascript</code> for its ubiquity and ease of use and <code>golang</code> for its high performance nature to solve <em><code>stream processing problem</code></em>.</p> <p>javascript is the most common programming language (source: stackoverflow). The popularity seems to stem from the fact that is it both light weight and easy to integrate with other frameworks/languages. However, it is just not made for heavy compute, and stream processing often requires that. On the other hand, golang's speed, simplicity and memory efficiency pegs it as a great candidate for compute heavy workloads.</p> <p>exthos attempts to bring the two together to provide a streaming processing engine with intuitive APIs - heavily inspired by the Apache Camel project.</p> <p>javascript with typescript provides a solid foundation for building a Domain Specific Language (DSL). exthos attempts to bring streaming/integration DSL to work with otherwise a configuration based stream engine called benthos.</p> <a href="#how-exthos-works" id="how-exthos-works" style="color: inherit; text-decoration: none;"> <h2>How exthos works</h2> </a> <p><code>exthos</code> has two modes of operations:</p> <ul> <li>local <code>isLocal=true</code>: In this mode, the engine takes on the responsibility of managning the engineProcess itself</li> <li>remote <code>isLocal=false</code>: In this mode, the engine communicates with a remote engineProcess over HTTP</li> </ul> <blockquote> <p>The default mode is <code>isLocal=true</code></p> </blockquote> <p>The remote mode can be used for deploying into remote benthos processes, and in the future will also be used to define the <code>exthos deployment framework</code></p> <a href="#how-to-use-exthos" id="how-to-use-exthos" style="color: inherit; text-decoration: none;"> <h2>How to use exthos</h2> </a> <p>exthos has two set of APIs:</p> <ul> <li>the low level APIs and</li> <li>the high level APIs</li> </ul> <p><strong>Users are encouraged to use the HL APIs</strong>, although an overview of the LL APIs helps to create a good foundation.</p> <a href="#overview-low-level-apis" id="overview-low-level-apis" style="color: inherit; text-decoration: none;"> <h3>Overview: Low level APIs</h3> </a> <p>At the core of exthos is the <code>engine</code> that is responsible for running the <code>engineProcess</code>. The <code>engineProcess</code> is esentially a child process that runs benthos (in streams mode). <code>streams</code> represent inputs connected to outputs, optionally via a pipeline containing processors and are created using: <code>new Stream(...)</code>. Stream objects are added to the engine instance at will using: <code>engine.add(stream)</code>. The streams can be updated or removed from the engine at will as well using: <code>engine.update(), engine.remove()</code>.</p> <a href="#overview-high-level-apis" id="overview-high-level-apis" style="color: inherit; text-decoration: none;"> <h3>Overview: High level APIs</h3> </a> <p>The high level APIs abstract a lot of complexity away from the user and enabling the users to focus on the integration/streaming problem. As mentioned before, HLAPIs are heavity inspired by Apache Camel's DSL, however it is important to note that there is no overlap or commonality between exthos and Camel.</p> <p>While using HLAPIs, the users create what we call a <code>route</code>. Internally, a route is akin to a LLAPI stream. A simple route would consist of an input and an output specified in a fluent API manner using <code>from</code> and <code>to</code> constructs. eg: <code>let route01 = from(...).to(...)</code></p> <p>Once a route is created, one can start it. Internally, exthos starts the engine if not running and adds the route to the engine. e.g. <code>route01.start()</code></p> <p>The route can subsequently be stopped using: <code>route01.stop()</code></p> <a href="#components" id="components" style="color: inherit; text-decoration: none;"> <h2>Components</h2> </a> <a href="#inputs" id="inputs" style="color: inherit; text-decoration: none;"> <h3>Inputs</h3> </a> <p><code>exthos</code> supports a number of inputs as sources of data. Take a look <a href="./src/types/inputs.ts">here</a>.</p> <p>Some specials inputs are:</p> <ul> <li><code>inport</code>: enables sending data from JS-land to goland-land</li> <li><code>direct</code>: acts as as alias and can be used to receive data from another <code>direct</code> output</li> </ul> <a href="#outputs" id="outputs" style="color: inherit; text-decoration: none;"> <h3>Outputs</h3> </a> <p><code>exthos</code> supports a number of outputs as destinations of data. Take a look <a href="./src/types/outputs.ts">here</a>.</p> <p>Some specials outputs are:</p> <ul> <li><code>outport</code>: enables receiving data from goland-land to JS-land</li> <li><code>direct</code>: acts as as alias and can be used to send data to another <code>direct</code> input</li> </ul> <a href="#processors" id="processors" style="color: inherit; text-decoration: none;"> <h3>Processors</h3> </a> <p><code>exthos</code> supports a number of processors. Procesors are used to act on a stream of data to perform operations such as lookup, filter, enrichment etc. Take a look <a href="./src/types/processors.ts">here</a>.</p> <p>Some specials processors are:</p> <ul> <li><code>javascript</code>: allows you to write javascript to manipulate <code>content</code> and <code>meta</code> of the messages</li> </ul> <a href="#messages" id="messages" style="color: inherit; text-decoration: none;"> <h3>Messages</h3> </a> <p><code>messages</code> are the events that are produced by sources, processed by processors and consumed by destinations. messages are always part of a <code>batch</code>. If unspcified, a batch contains a single message only. However, you can create batches of multiple messages and operate on them. Each message contains three parts:</p> <ul> <li><code>meta</code>: contains the metadata for the message as key value pairs</li> <li><code>content</code>: contains the actual data you may be intereted it. This data can be anything e.g. raw bytes, structured JSON, text etc.</li> <li><code>error</code>: contains description of the error associated with a message. This is important to note, <strong>because messages are not dropped on error</strong>. i.e. if a processor fails on a message, the error property is tagged with the error description and the message is moved to the next processor.</li> </ul> <hr> <a href="#in-practice" id="in-practice" style="color: inherit; text-decoration: none;"> <h1>In practice</h1> </a> <a href="#installation" id="installation" style="color: inherit; text-decoration: none;"> <h2>Installation</h2> </a> <p><code>npm i exthos</code></p> <a href="#usage" id="usage" style="color: inherit; text-decoration: none;"> <h2>Usage</h2> </a> <p>Now that you hopefully have a fair idea of what Benthos is and is not, it's time to get your hands dirty.</p> <p>A good starting point is to read the notes below and then head to the <code>examples</code> folders.</p> <a href="#using-high-level-apis-fromviato" id="using-high-level-apis-fromviato" style="color: inherit; text-decoration: none;"> <h2>Using high level APIs: from/via/to</h2> </a> <p>The following examples illustrate the basic usage. Most examples below make use of <code>bloblang</code>; please refer to <a href="https://www.benthos.dev/docs/guides/bloblang/about">link</a> for details on how to use it.</p> <pre><code class="language-js"><span class="hl-0">// import the required constructs</span><br/><span class="hl-1">import</span><span class="hl-2"> { </span><span class="hl-3">from</span><span class="hl-2">, </span><span class="hl-3">to</span><span class="hl-2"> } </span><span class="hl-1">from</span><span class="hl-2"> </span><span class="hl-4">"exthos"</span><br/><br/><span class="hl-0">// example: create a route, start it and stop it after 5 seconds</span><br/><span class="hl-5">from</span><span class="hl-2">({ </span><span class="hl-3">generate:</span><span class="hl-2"> { </span><span class="hl-3">mapping:</span><span class="hl-2"> </span><span class="hl-4">'root = count("gen")'</span><span class="hl-2"> } }).</span><span class="hl-5">to</span><span class="hl-2">({ </span><span class="hl-3">stdout:</span><span class="hl-2"> {} }).</span><span class="hl-5">start</span><span class="hl-2">().</span><span class="hl-5">stopAfter</span><span class="hl-2">(</span><span class="hl-6">5000</span><span class="hl-2">)</span><br/><br/><span class="hl-0">// example: create a route with a terminating source and start it.</span><br/><span class="hl-0">// generate input with count = 2 will geenrate 2 msgs and </span><br/><span class="hl-5">from</span><span class="hl-2">({ </span><span class="hl-3">generate:</span><span class="hl-2"> { </span><span class="hl-3">mapping:</span><span class="hl-2"> </span><span class="hl-4">'root = count("gen")'</span><span class="hl-2">, </span><span class="hl-3">count:</span><span class="hl-2"> </span><span class="hl-6">2</span><span class="hl-2"> } }).</span><span class="hl-5">to</span><span class="hl-2">({ </span><span class="hl-3">stdout:</span><span class="hl-2"> {} }).</span><span class="hl-5">start</span><span class="hl-2">()</span><br/><span class="hl-0">// no need to specify stop explicitly here as the route will stop after 2 messages are processed</span> </code></pre> <p>For more examples refer to: <a href="%5B/examples/hlapis/README.md%5D(https://github.com/exthosdev/exthos/tree/main/examples/hlapis)">examples dir</a></p> <a href="#othernon-functional-features" id="othernon-functional-features" style="color: inherit; text-decoration: none;"> <h2>Other/non-functional features</h2> </a> <a href="#configuration" id="configuration" style="color: inherit; text-decoration: none;"> <h3>Configuration</h3> </a> <p>Like any application, exthos relies of some smart defaults but at the same time empowers the users to provide their own configuration via a <code>config file</code> or <code>environment variables</code>.</p> <p>The order or precedence is as follows:</p> <ol> <li>default configuration as defined under <code>config/config.default.ts</code></li> <li>overwritten by any matching keys in <code>exthos.config.json</code> placed in the CWD (Current Working Directory: usually where you project root is)</li> <li>overwritten by any matching keys in environment variables prefixed with <code>EXTHOS_</code> e.g.<ol> <li><code>EXTHOS_engineExtraConfig_benthosDir=/tmp/test</code></li> <li><code>EXTHOS_engineConfig_logger_level=TRACE</code></li> </ol> </li> <li>overwritten by parameters sent to constructor via <code>new Engine({<engineConfig>}, {<extraEngineConfig>})</code></li> </ol> <p>e.g. to use your pre-installed version of benthos create a file named <code>exthos.config.json</code> on CWD with the following content:</p> <pre><code class="language-json"><span class="hl-2">{</span><br/><span class="hl-2"> </span><span class="hl-7">"engineExtraConfig"</span><span class="hl-2">: {</span><br/><span class="hl-2"> </span><span class="hl-7">"benthosDir"</span><span class="hl-2">: </span><span class="hl-4">""</span><span class="hl-2">,</span><br/><span class="hl-2"> </span><span class="hl-7">"benthosFileName"</span><span class="hl-2">: </span><span class="hl-4">"benthos"</span><br/><span class="hl-2"> }</span><br/><span class="hl-2">}</span> </code></pre> <a href="#labels" id="labels" style="color: inherit; text-decoration: none;"> <h3>Labels</h3> </a> <blockquote> <p>this section covers <code>llapis</code> only</p> </blockquote> <p>Labels can be optionally assigned to all components. They aid in debugging and tracing. The labels will be sanitised by the stream instace to follow the following rules:</p> <ul> <li>should match the regular expression /^[a-z0-9_]+$/</li> <li>must not start with an underscore</li> </ul> <a href="#error-handling" id="error-handling" style="color: inherit; text-decoration: none;"> <h3>Error handling</h3> </a> <blockquote> <p>this section covers <code>hlapis</code> only</p> </blockquote> <p><code>exthos</code> makes best attempt at swallowing any errors generated</p> <p>There are two source of errors we must handle:</p> <ol> <li>errors in JS land<ol> <li>For JS land, errors can be handled using <code>try(){}catch(){}finally(){}</code> or <code><Promise>.then().catch().finally()</code></li> <li>Checkout the examples <a href="https://github.com/exthosdev/exthos/blob/main/examples/hlapis/errorHandling.js">here</a></li> </ol> </li> <li>errors in Golang land<ol> <li>For Golang land, stream and message related errors must be handled using the component configurations, which are then handed over to benthos. As a rule of thumb, when an error occurs on a message, the error property of the message is marked with the error description and the message moves to the next component of the stream. To understand error handling please go through <a href="https://www.benthos.dev/docs/configuration/error_handling/">this link</a></li> </ol> </li> </ol> <a href="#logging" id="logging" style="color: inherit; text-decoration: none;"> <h3>Logging</h3> </a> <p><code>exthos</code> makes use of the debug package to log output to stdout.</p> <p>You can configure the debug namespace using one of the following:</p> <ol> <li><code>DEBUG=...*</code> environment variable</li> <li>When initializing the engine<ol> <li>llapi: <code>new Engine({}, {debugNamespace: "...*"})</code></li> <li>hlapi: <code>engine.setEngineConfigOptions({}, {debugNamespace: "...*"})</code></li> </ol> </li> </ol> <p>The default namespace is nil i.e. = ""</p> <p>The following namespaces are available within <code>exthos</code>:</p> <table> <thead> <tr> <th>namespace</th> <th>description</th> </tr> </thead> <tbody><tr> <td>exthos*</td> <td>all exthos logs</td> </tr> <tr> <td>exthos:engine:*</td> <td>engine logs of all <code>log types</code>; engine logs originate in the Javascript/Node land</td> </tr> <tr> <td>exthos:engineProcess:*</td> <td>engineProcess logs of all <code>log levels</code>; engineProcess logs originate from the golang runtime</td> </tr> <tr> <td>exthos:engine:debugLog</td> <td>engine logs of debugLog type, providing info debug and info level information</td> </tr> <tr> <td>exthos:engine:traceLog</td> <td>engine logs of traceLog type, providing very detailed trace level information such as flow of code execution</td> </tr> <tr> <td>exthos:eventLog*</td> <td>all eventLog type, providing events generated as a log; each logline is a JSON containing the <code>eventObj</code></td> </tr> <tr> <td>exthos:eventLog:<code><eventName></code></td> <td>eventLog type with an <code><eventName></code> eventName, refer to <a href="#events">Events</a> for complete list of <code><eventName>s</code></td> </tr> <tr> <td>exthos:engineProcess:trace</td> <td>engineProcess logs of trace level</td> </tr> <tr> <td>exthos:engineProcess:debug</td> <td>engineProcess logs of debug level</td> </tr> <tr> <td>exthos:engineProcess:info</td> <td>engineProcess logs of info level</td> </tr> <tr> <td>exthos:engineProcess:warn</td> <td>engineProcess logs of warn level</td> </tr> <tr> <td>exthos:engineProcess:error</td> <td>engineProcess logs of error level</td> </tr> <tr> <td>exthos:engineProcess:fatal</td> <td>engineProcess logs of fatal level</td> </tr> </tbody></table> <a href="#events" id="events" style="color: inherit; text-decoration: none;"> <h3>Events</h3> </a> <p><code>exthos</code> emits a lot of events. These can be used to trigger custom actions (aka handlers) in JS-land e.g. sending an alert notification on an <code>error</code> type category, or stopping the engine on a <code>fatal</code> type category, etc.</p> <ul> <li>You could provide your own event handling functions (actions/handlers) using any of the following. For even more options on how to handle events refer to <a href="https://github.com/hij1nx/EventEmitter2#readme">EventEmitter2</a> documentation<ul> <li><code>engine.on("<eventName>", (eventObj) => {...})</code></li> <li><code>engine.onAny((eventName, eventObj) => {...})</code> - listens to all events geenrated by the engine and allows you to handle them at one place</li> </ul> </li> <li>Or, you could use the builtin default event handler using: <code>engine.useDefaultEventHandler(addnCustomHandler)</code><ul> <li>the default event handler prints all events on the console/stdout,</li> <li>stops the stream on receiving an "engineProcess.stream.error" event 5 times</li> <li>you could add custom eventHandler on top of already available using <code>addnCustomHandler</code> e.g.</li> </ul> </li> </ul> <pre><code class="language-js"><span class="hl-3">engine</span><span class="hl-2">.</span><span class="hl-5">useDefaultEventHandler</span><span class="hl-2">({</span><br/><span class="hl-2"> </span><span class="hl-4">"engine.fatal"</span><span class="hl-3">:</span><span class="hl-2"> (</span><span class="hl-3">eventObj</span><span class="hl-2">) </span><span class="hl-8">=></span><span class="hl-2"> {</span><br/><span class="hl-2"> </span><span class="hl-3">console</span><span class="hl-2">.</span><span class="hl-5">log</span><span class="hl-2">(</span><span class="hl-4">"</span><span class="hl-9">\n</span><span class="hl-4">Test EXITED with CODE=1"</span><span class="hl-2">, </span><span class="hl-10">JSON</span><span class="hl-2">.</span><span class="hl-5">stringify</span><span class="hl-2">(</span><span class="hl-3">eventObj</span><span class="hl-2">));</span><br/><span class="hl-2"> </span><span class="hl-3">process</span><span class="hl-2">.</span><span class="hl-5">exit</span><span class="hl-2">(</span><span class="hl-6">1</span><span class="hl-2">);</span><br/><span class="hl-2"> },</span><br/><span class="hl-2">});</span> </code></pre> <blockquote> <p>As mention under <a href="#logging">Logging</a>, all events are also logged under <code>exthos:eventLog</code> namespace</p> </blockquote> <p>All possible events are shown in the table below:</p> <table> <thead> <tr> <th>Event name (eventName)</th> <th>Event category</th> <th>Event Object properties (eventObj)</th> <th>Description</th> </tr> </thead> <tbody><tr> <td><code>engine.active</code></td> <td>general</td> <td><code>{msg: string, time: string}</code></td> <td>signifies that the engine is running</td> </tr> <tr> <td><code>engine.inactive</code></td> <td>general</td> <td><code>{msg: string, time: string}</code></td> <td>signifies that the engine is NOT running</td> </tr> <tr> <td><code>engine.warn</code></td> <td>warn</td> <td><code>{msg: string, time: string, error: Error}</code></td> <td>signifies a potential abnormality that was ignored/recovered</td> </tr> <tr> <td><code>engine.error</code></td> <td>error</td> <td><code>{msg: string, time: string, error: Error}</code></td> <td>signifies a failure to an operation, not the entire engine</td> </tr> <tr> <td><code>engine.fatal</code></td> <td>fatal</td> <td><code>{msg: string, time: string, error: Error}</code></td> <td>signifies a failure to the entire engine, leading to shutdown of the engine</td> </tr> <tr> <td><code>engine.stream.add</code></td> <td>general</td> <td><code>{msg: string, time: string, stream: Stream}</code></td> <td>signifies that a stream has been added to the engine</td> </tr> <tr> <td><code>engine.stream.update</code></td> <td>general</td> <td><code>{msg: string, time: string, stream: Stream}</code></td> <td>signifies that a stream has been updated to the engine</td> </tr> <tr> <td><code>engine.stream.remove</code></td> <td>general</td> <td><code>{msg: string, time: string, stream: Stream}</code></td> <td>signifies that a stream has been removed from the engine</td> </tr> <tr> <td><code>engine.stream.error</code></td> <td>error</td> <td><code>{msg: string, time: string, stream: Stream, error: Error}</code></td> <td>signifies an error when the engine tried to add/update/remove the stream from the engine</td> </tr> <tr> <td><code>engineProcess.stream.fatal</code></td> <td>fatal</td> <td><code>{msg: string, time: string, stream: Stream, error: Error}</code></td> <td>jury is out on when engineProcess emits this</td> </tr> <tr> <td><code>engineProcess.stream.error</code></td> <td>error</td> <td><code>{msg: string, time: string, stream: Stream, error: Error}</code></td> <td>jury is out on when engineProcess emits this</td> </tr> <tr> <td><code>engineProcess.stream.warn</code></td> <td>error</td> <td><code>{msg: string, time: string, stream: Stream, error: Error}</code></td> <td>jury is out on when engineProcess emits this</td> </tr> <tr> <td><code>engineProcess.stream.info</code></td> <td>error</td> <td><code>{msg: string, time: string, stream: Stream, error: Error}</code></td> <td>jury is out on when engineProcess emits this</td> </tr> <tr> <td><code>engineProcess.stream.debug</code></td> <td>error</td> <td><code>{msg: string, time: string, stream: Stream, error: Error}</code></td> <td>jury is out on when engineProcess emits this</td> </tr> <tr> <td><code>engineProcess.stream.trace</code></td> <td>error</td> <td><code>{msg: string, time: string, stream: Stream, error: Error}</code></td> <td>jury is out on when engineProcess emits this</td> </tr> </tbody></table> <p>Each event emits the <code>eventObj</code> that takes the form of <code>{ stream?: Stream, msg: string }</code> i.e. it always contains the <code>msg</code> string value and may contain the <code>stream</code> object if the event was generated in the context of a stream.</p> <a href="#metrics" id="metrics" style="color: inherit; text-decoration: none;"> <h3>Metrics</h3> </a> <p>Coming soon.</p> <a href="#tracing-and-telemetry" id="tracing-and-telemetry" style="color: inherit; text-decoration: none;"> <h3>Tracing and Telemetry</h3> </a> <p>Coming soon.</p> <a href="#compatability-amp-support" id="compatability-amp-support" style="color: inherit; text-decoration: none;"> <h1>Compatability & Support</h1> </a> <ul> <li>only linux based OS will work</li> <li>exthos to benthos compatability mapping follows:</li> </ul> <table> <thead> <tr> <th>exthos</th> <th>benthos</th> </tr> </thead> <tbody><tr> <td>v0.0.1 - v0.2.1</td> <td>v4.5.1</td> </tr> </tbody></table> <a href="#acknowledgement" id="acknowledgement" style="color: inherit; text-decoration: none;"> <h1>Acknowledgement</h1> </a> <p>While we thank all the projects that exthos depends on, we would like to show special gratitude towards the following projects:</p> <ul> <li><a href="https://github.com/benthosdev/benthos">benthos</a></li> <li><a href="https://github.com/nickdesaulniers/node-nanomsg">nanomsg</a></li> <li><a href="https://github.com/hij1nx/EventEmitter2">eventemitter2</a></li> <li><a href="https://github.com/debug-js/debug">debug</a></li> <li><a href="https://github.com/DirtyHairy/async-mutex">async-mutex</a></li> </ul> </div></div> <div class="col-4 col-menu menu-sticky-wrap menu-highlight"> <div class="tsd-navigation settings"> <details class="tsd-index-accordion"><summary class="tsd-accordion-summary"> <h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none"><path d="M4.93896 8.531L12 15.591L19.061 8.531L16.939 6.409L12 11.349L7.06098 6.409L4.93896 8.531Z" fill="var(--color-text)"></path></svg> Settings</h3></summary> <div class="tsd-accordion-details"> <div class="tsd-filter-visibility"> <h4 class="uppercase">Member Visibility</h4><form> <ul id="tsd-filter-options"> <li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-protected" name="protected"/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Protected</span></label></li> <li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-private" name="private"/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Private</span></label></li> <li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-inherited" name="inherited" checked/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Inherited</span></label></li> <li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-external" name="external"/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>External</span></label></li></ul></form></div> <div class="tsd-theme-toggle"> <h4 class="uppercase">Theme</h4><select id="theme"><option value="os">OS</option><option value="light">Light</option><option value="dark">Dark</option></select></div></div></details></div> <nav class="tsd-navigation primary"> <details class="tsd-index-accordion" open><summary class="tsd-accordion-summary"> <h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none"><path d="M4.93896 8.531L12 15.591L19.061 8.531L16.939 6.409L12 11.349L7.06098 6.409L4.93896 8.531Z" fill="var(--color-text)"></path></svg> Modules</h3></summary> <div class="tsd-accordion-details"> <ul> <li class="current selected"><a href="modules.html">exthos</a> <ul></ul></li></ul></div></details></nav> <nav class="tsd-navigation secondary menu-sticky"> <ul> <li class="tsd-kind-class"><a href="classes/Engine.html" class="tsd-index-link"><svg class="tsd-kind-icon" width="24" height="24" viewBox="0 0 24 24"><rect fill="var(--color-icon-background)" stroke="var(--color-ts-class)" stroke-width="1.5" x="1" y="1" width="22" height="22" rx="6" id="icon-128-path"></rect><path d="M11.898 16.1201C11.098 16.1201 10.466 15.8961 10.002 15.4481C9.53803 15.0001 9.30603 14.3841 9.30603 13.6001V9.64012C9.30603 8.85612 9.53803 8.24012 10.002 7.79212C10.466 7.34412 11.098 7.12012 11.898 7.12012C12.682 7.12012 13.306 7.34812 13.77 7.80412C14.234 8.25212 14.466 8.86412 14.466 9.64012H13.386C13.386 9.14412 13.254 8.76412 12.99 8.50012C12.734 8.22812 12.37 8.09212 11.898 8.09212C11.426 8.09212 11.054 8.22412 10.782 8.48812C10.518 8.75212 10.386 9.13212 10.386 9.62812V13.6001C10.386 14.0961 10.518 14.4801 10.782 14.7521C11.054 15.0161 11.426 15.1481 11.898 15.1481C12.37 15.1481 12.734 15.0161 12.99 14.7521C13.254 14.4801 13.386 14.0961 13.386 13.6001H14.466C14.466 14.3761 14.234 14.9921 13.77 15.4481C13.306 15.8961 12.682 16.1201 11.898 16.1201Z" fill="var(--color-text)" id="icon-128-text"></path></svg>Engine</a></li> <li class="tsd-kind-class"><a href="classes/Stream.html" class="tsd-index-link"><svg class="tsd-kind-icon" width="24" height="24" viewBox="0 0 24 24"><use href="#icon-128-path"></use><use href="#icon-128-text"></use></svg>Stream</a></li> <li class="tsd-kind-variable"><a href="variables/default.html" class="tsd-index-link"><svg class="tsd-kind-icon" width="24" height="24" viewBox="0 0 24 24"><rect fill="var(--color-icon-background)" stroke="var(--color-ts-variable)" stroke-width="1.5" x="1" y="1" width="22" height="22" rx="6" id="icon-32-path"></rect><path d="M11.106 16L8.85 7.24H9.966L11.454 13.192C11.558 13.608 11.646 13.996 11.718 14.356C11.79 14.708 11.842 14.976 11.874 15.16C11.906 14.976 11.954 14.708 12.018 14.356C12.09 13.996 12.178 13.608 12.282 13.192L13.758 7.24H14.85L12.582 16H11.106Z" fill="var(--color-text)" id="icon-32-text"></path></svg>default</a></li> <li class="tsd-kind-variable"><a href="variables/engine-1.html" class="tsd-index-link"><svg class="tsd-kind-icon" width="24" height="24" viewBox="0 0 24 24"><use href="#icon-32-path"></use><use href="#icon-32-text"></use></svg>engine</a></li> <li class="tsd-kind-function"><a href="functions/from.html" class="tsd-index-link"><svg class="tsd-kind-icon" width="24" height="24" viewBox="0 0 24 24"><rect fill="var(--color-icon-background)" stroke="var(--color-ts-function)" stroke-width="1.5" x="1" y="1" width="22" height="22" rx="6" id="icon-64-path"></rect><path d="M9.39 16V7.24H14.55V8.224H10.446V11.128H14.238V12.112H10.47V16H9.39Z" fill="var(--color-text)" id="icon-64-text"></path></svg>from</a></li></ul></nav></div></div> <div class="container tsd-generator"> <p>Generated using <a href="https://typedoc.org/" target="_blank">TypeDoc</a></p></div> <div class="overlay"></div><script src="assets/main.js"></script></body></html>