chevrotain
Version:
Chevrotain is a high performance fault Tolerant Javascript parsing DSL for building recursive decent parsers
374 lines (362 loc) • 18.5 kB
HTML
<html class="default no-js">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>Chevrotain</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="assets/css/main.css">
<script src="assets/js/modernizr.js"></script>
</head>
<body>
<header>
<div class="tsd-page-toolbar">
<div class="container">
<div class="table-wrap">
<div class="table-cell" id="tsd-search" data-index="assets/js/search.js" data-base=".">
<div class="field">
<label for="tsd-search-field" class="tsd-widget search no-caption">Search</label>
<input id="tsd-search-field" type="text" />
</div>
<ul class="results">
<li class="state loading">Preparing search index...</li>
<li class="state failure">The search index is not available</li>
</ul>
<a href="index.html" class="title">Chevrotain</a>
</div>
<div class="table-cell" id="tsd-widgets">
<div id="tsd-filter">
<a href="#" class="tsd-widget options no-caption" data-toggle="options">Options</a>
<div class="tsd-filter-group">
<div class="tsd-select" id="tsd-filter-visibility">
<span class="tsd-select-label">All</span>
<ul class="tsd-select-list">
<li data-value="public">Public</li>
<li data-value="protected">Public/Protected</li>
<li data-value="private" class="selected">All</li>
</ul>
</div>
<input type="checkbox" id="tsd-filter-inherited" checked />
<label class="tsd-widget" for="tsd-filter-inherited">Inherited</label>
<input type="checkbox" id="tsd-filter-only-exported" />
<label class="tsd-widget" for="tsd-filter-only-exported">Only exported</label>
</div>
</div>
<a href="#" class="tsd-widget menu no-caption" data-toggle="menu">Menu</a>
</div>
</div>
</div>
</div>
<div class="tsd-page-title">
<div class="container">
<ul class="tsd-breadcrumb">
<li>
<a href="globals.html">Globals</a>
</li>
</ul>
<h1> Chevrotain</h1>
</div>
</div>
</header>
<div class="container container-main">
<div class="row">
<div class="col-8 col-content">
<div class="tsd-panel tsd-typography">
<p><a href="https://travis-ci.org/SAP/chevrotain"><img src="https://travis-ci.org/SAP/chevrotain.svg?branch=master" alt="Build Status"></a>
<a href="https://coveralls.io/r/SAP/chevrotain?branch=master"><img src="https://coveralls.io/repos/SAP/chevrotain/badge.svg?branch=master" alt="Coverage Status"></a>
<a href="https://npmjs.org/package/chevrotain"><img src="https://nodei.co/npm/chevrotain.png?mini=true" alt="NPM"></a></p>
<h1 id="chevrotain">Chevrotain</h1>
<p>Chevrotain is a high performance fault Tolerant Javascript parsing DSL for building recursive decent parsers.</p>
<p>Chevrotain is <strong>NOT</strong> a parser generator. it solves the same kind of problems as a parser generator, just without
the code generation phase.</p>
<h2 id="features">Features</h2>
<ul>
<li><p><strong>Lexer engine</strong> based on RexExps.</p>
<ul>
<li>Supports Token location tracking.</li>
<li>Supports Token skipping (whitespace/comments/...)</li>
<li>Allows prioritising shorter matches (Keywords vs Identifiers).</li>
<li><strong>No code generation</strong> The Lexer does not require any code generation phase. </li>
</ul>
</li>
<li><p><strong>Parsing DSL</strong> for creating the parsing rules.</p>
<ul>
<li><strong>No code generation</strong> - the DSL is just javascript not a new external language, what is written is what will be run, this speeds up development,
makes debugging trivial and provides great flexibility for inserting custom actions into the grammar.</li>
<li>Strong <strong>Error Recovery</strong> capabilities based on Antlr3's algorithms.</li>
<li>Automatic lookahead calculation for LL(1) grammars.</li>
<li>In addition custom lookahead logic can be provided explicitly.</li>
<li>Backtracking support. </li>
</ul>
</li>
<li><p><strong>High performance</strong> see: <a href="http://chevrotain.github.io/performance/">performance comparison</a> </p>
</li>
<li><p><strong>Grammar Introspection</strong>, the grammar's structure is known and <strong>exposed</strong> this can be used to implement features such as automatically generated syntax diagrams or Syntactic error recovery.</p>
</li>
<li><p>Well tested with <strong>~100% code coverage</strong> </p>
</li>
</ul>
<h2 id="installation">Installation</h2>
<ul>
<li><strong>npm</strong>: <code>npm install chevrotain</code></li>
<li><strong>Bower</strong> <code>bower install chevrotain</code></li>
<li>or download directly from <a href="https://github.com/SAP/chevrotain/releases/latest">github releases</a>:<ul>
<li>the 'chevrotain-binaries-...' files contain the compiled javascript code.</li>
</ul>
</li>
</ul>
<h2 id="usage-example-json-parser-">Usage example JSON Parser:</h2>
<ul>
<li>The following example uses several features of ES6 (fat arrow/classes).
These are not mandatory for using Chevrotain, they just make the example clearer.
The example is also provided in <a href="https://github.com/Chevrotain/examples_nodejs">ES5 syntax</a></li>
</ul>
<h4 id="step-1-define-your-tokens-">step 1: define your Tokens:</h4>
<pre><code class="lang-JavaScript">
var Token = require("chevrotain").Token
class Keyword extends Token { static PATTERN = NA }
class True extends Keyword { static PATTERN = /true/ }
class False extends Keyword { static PATTERN = /false/ }
class Null extends Keyword { static PATTERN = /null/ }
class LCurly extends Token { static PATTERN = /{/ }
class RCurly extends Token { static PATTERN = /}/ }
class LSquare extends Token { static PATTERN = /\[/ }
class RSquare extends Token { static PATTERN = /]/ }
class Comma extends Token { static PATTERN = /,/ }
class Colon extends Token { static PATTERN = /:/ }
class StringLiteral extends Token { static PATTERN = /"(:?[^\\"]+|\\(:?[bfnrtv"\\/]|u[0-9a-fA-F]{4}))*"/}
class NumberLiteral extends Token { static PATTERN = /-?(0|[1-9]\d*)(\.\d+)?([eE][+-]?\d+)?/ }
class WhiteSpace extends Token {
static PATTERN = /\s+/
static GROUP = SKIPPED
}
</code></pre>
<h4 id="step-2-create-a-lexer-from-the-token-definitions-">step 2: create a lexer from the Token definitions:</h4>
<pre><code class="lang-JavaScript">
var Lexer = require("chevrotain").Lexer
var JsonLexer = new chevrotain.Lexer([WhiteSpace, NumberLiteral, StringLiteral,
RCurly, LCurly, LSquare, RSquare, Comma, Colon, True, False, Null])
</code></pre>
<h4 id="step-3-define-the-parsing-rules-">step 3: define the parsing rules:</h4>
<pre><code class="lang-JavaScript">
var Parser = require("chevrotain").Parser
class JsonParser extends Parser {
constructor(input) {
Parser.performSelfAnalysis(this)
}
public object = this.RULE("object", () => {
this.CONSUME(LCurly)
this.OPTION(() => {
this.SUBRULE(this.objectItem)
this.MANY(() => {
this.CONSUME(Comma)
this.SUBRULE2(this.objectItem)
})
})
this.CONSUME(RCurly)
})
public objectItem = this.RULE("objectItem", () => {
this.CONSUME(StringLiteral)
this.CONSUME(Colon)
this.SUBRULE(this.value)
})
public array = this.RULE("array", () => {
this.CONSUME(LSquare)
this.OPTION(() => {
this.SUBRULE(this.value)
this.MANY(() => {
this.CONSUME(Comma)
this.SUBRULE2(this.value)
})
})
this.CONSUME(RSquare)
})
public value = this.RULE("value", () => {
this.OR([
{ALT: () => {this.CONSUME(StringLiteral)}},
{ALT: () => {this.CONSUME(NumberLiteral)}},
{ALT: () => {this.SUBRULE(this.object)}},
{ALT: () => {this.SUBRULE(this.array)}},
{ALT: () => {this.CONSUME(True)}},
{ALT: () => {this.CONSUME(False)}},
{ALT: () => {this.CONSUME(Null)}}
], "a value")
})
}
</code></pre>
<h4 id="step-4-add-custom-actions-to-the-grammar-defined-in-step-3">step 4: add custom actions to the grammar defined in step 3</h4>
<ul>
<li>this shows the modification for just two grammar rules.</li>
</ul>
<pre><code class="lang-JavaScript">
public object = this.RULE("object", () => {
var items = []
this.CONSUME(LCurly)
this.OPTION(() => {
items.push(this.SUBRULE(this.objectItem)) // .push to collect the objectItems
this.MANY(() => {
this.CONSUME(Comma)
items.push(this.SUBRULE2(this.objectItem)) // .push to collect the objectItems
})
})
this.CONSUME(RCurly)
// merge all the objectItems
var obj = {}
items.forEach((item) => {
obj[item.itemName] = item.itemValue
})
return obj
})
public objectItem = this.RULE("objectItem", () => {
var nameToken = this.CONSUME(StringLiteral)
this.CONSUME(Colon)
var value = this.SUBRULE(this.value) // assumes SUBRULE(this.value) returns the JS value (null/number/string/...)
var itemNameString = nameToken.image // nameToken.image to get the literalString the lexer consumed
var itemName = itemNameString.substr(1, itemNameString.length - 2) // chop off the string quotes
return {itemName:itemName, itemValue:value}
})
...
</code></pre>
<h4 id="step-5-wrap-it-all-together">step 5: wrap it all together</h4>
<pre><code class="lang-JavaScript">
function lexAndParse(text) {
var lexResult = JsonLexer.tokenize(text)
var parser = new JsonParser(lexResult.tokens)
return parser.object()
}
</code></pre>
<h2 id="getting-started">Getting Started</h2>
<p>The best way to start is by looking at some runable (and debugable) examples:</p>
<ul>
<li><a href="https://github.com/Chevrotain/examples_nodejs/blob/master/jsonParser.js">Json Parser</a></li>
<li><a href="https://github.com/Chevrotain/examples_nodejs/blob/master/calculator.js">Simple Calculator</a></li>
<li><a href="https://github.com/Chevrotain/examples_ecma5_typescript/blob/master/src/ecmascript5_parser.ts">ECMAScript5 Parser</a></li>
<li><a href="https://github.com/Chevrotain/examples_lexer">Lexer advanced features</a></li>
<li><a href="https://github.com/Chevrotain">and more</a></li>
</ul>
<h2 id="documentation">Documentation</h2>
<ul>
<li><p><a href="http://chevrotain.github.io/documentation">Latest released version's HTML docs</a></p>
<ul>
<li>Also packaged in both the github and npm releases.</li>
</ul>
</li>
<li><p>Annotated source code (dev version):</p>
<ul>
<li><a href="https://github.com/SAP/chevrotain/blob/master/src/scan/tokens_public.ts">tokens_public.ts</a></li>
<li><a href="https://github.com/SAP/chevrotain/blob/master/src/scan/lexer_public.ts">lexer_public.ts</a></li>
<li><a href="https://github.com/SAP/chevrotain/blob/master/src/parse/parser_public.ts">parser_public.ts</a></li>
<li><a href="https://github.com/SAP/chevrotain/blob/master/src/parse/grammar/gast.ts">gast_public.ts</a></li>
</ul>
</li>
<li><p>The aggregated Typescript definitions :</p>
<ul>
<li><a href="https://github.com/SAP/chevrotain/blob/master/release/chevrotain.d.ts">chevrotain.d.ts</a> (dev version)</li>
<li>Also packaged in both the github and npm releases.</li>
</ul>
</li>
</ul>
<h2 id="dependencies">Dependencies</h2>
<p>Only a single dependency to <a href="https://lodash.com/">lodash</a>.</p>
<h2 id="compatibility">Compatibility</h2>
<p>The Generated artifact(chevrotain.js) should run on any modern Javascript ES5.1 runtime. </p>
<ul>
<li>The CI build runs the tests under Node.js.</li>
<li>Additionally local testing is done on latest versions of Chrome/Firefox/IE.</li>
<li>The dependency to lodash is imported via <a href="https://github.com/umdjs/umd">UMD</a>,
in order to make chevrotain.js portable to multiple environments.</li>
</ul>
</div>
</div>
<div class="col-4 col-menu menu-sticky-wrap menu-highlight">
<nav class="tsd-navigation primary">
<ul>
<li class="globals ">
<a href="globals.html"><em>Globals</em></a>
</li>
<li class=" tsd-kind-module">
<a href="modules/chevrotain.html">chevrotain</a>
</li>
<li class=" tsd-kind-module tsd-parent-kind-module">
<a href="modules/chevrotain.exceptions.html">chevrotain.exceptions</a>
</li>
<li class=" tsd-kind-module tsd-parent-kind-module">
<a href="modules/chevrotain.gast.html">chevrotain.gast</a>
</li>
</ul>
</nav>
<nav class="tsd-navigation secondary menu-sticky">
<ul class="before-current">
</ul>
</nav>
</div>
</div>
</div>
<footer class="with-border-bottom">
<div class="container">
<h2>Legend</h2>
<div class="tsd-legend-group">
<ul class="tsd-legend">
<li class="tsd-kind-module"><span class="tsd-kind-icon">Module</span></li>
<li class="tsd-kind-object-literal"><span class="tsd-kind-icon">Object literal</span></li>
<li class="tsd-kind-variable"><span class="tsd-kind-icon">Variable</span></li>
<li class="tsd-kind-function"><span class="tsd-kind-icon">Function</span></li>
<li class="tsd-kind-function tsd-has-type-parameter"><span class="tsd-kind-icon">Function with type parameter</span></li>
<li class="tsd-kind-index-signature"><span class="tsd-kind-icon">Index signature</span></li>
<li class="tsd-kind-type-alias"><span class="tsd-kind-icon">Type alias</span></li>
</ul>
<ul class="tsd-legend">
<li class="tsd-kind-enum"><span class="tsd-kind-icon">Enumeration</span></li>
<li class="tsd-kind-enum-member"><span class="tsd-kind-icon">Enumeration member</span></li>
<li class="tsd-kind-property tsd-parent-kind-enum"><span class="tsd-kind-icon">Property</span></li>
<li class="tsd-kind-method tsd-parent-kind-enum"><span class="tsd-kind-icon">Method</span></li>
</ul>
<ul class="tsd-legend">
<li class="tsd-kind-interface"><span class="tsd-kind-icon">Interface</span></li>
<li class="tsd-kind-interface tsd-has-type-parameter"><span class="tsd-kind-icon">Interface with type parameter</span></li>
<li class="tsd-kind-constructor tsd-parent-kind-interface"><span class="tsd-kind-icon">Constructor</span></li>
<li class="tsd-kind-property tsd-parent-kind-interface"><span class="tsd-kind-icon">Property</span></li>
<li class="tsd-kind-method tsd-parent-kind-interface"><span class="tsd-kind-icon">Method</span></li>
<li class="tsd-kind-index-signature tsd-parent-kind-interface"><span class="tsd-kind-icon">Index signature</span></li>
</ul>
<ul class="tsd-legend">
<li class="tsd-kind-class"><span class="tsd-kind-icon">Class</span></li>
<li class="tsd-kind-class tsd-has-type-parameter"><span class="tsd-kind-icon">Class with type parameter</span></li>
<li class="tsd-kind-constructor tsd-parent-kind-class"><span class="tsd-kind-icon">Constructor</span></li>
<li class="tsd-kind-property tsd-parent-kind-class"><span class="tsd-kind-icon">Property</span></li>
<li class="tsd-kind-method tsd-parent-kind-class"><span class="tsd-kind-icon">Method</span></li>
<li class="tsd-kind-accessor tsd-parent-kind-class"><span class="tsd-kind-icon">Accessor</span></li>
<li class="tsd-kind-index-signature tsd-parent-kind-class"><span class="tsd-kind-icon">Index signature</span></li>
</ul>
<ul class="tsd-legend">
<li class="tsd-kind-constructor tsd-parent-kind-class tsd-is-inherited"><span class="tsd-kind-icon">Inherited constructor</span></li>
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-inherited"><span class="tsd-kind-icon">Inherited property</span></li>
<li class="tsd-kind-method tsd-parent-kind-class tsd-is-inherited"><span class="tsd-kind-icon">Inherited method</span></li>
<li class="tsd-kind-accessor tsd-parent-kind-class tsd-is-inherited"><span class="tsd-kind-icon">Inherited accessor</span></li>
</ul>
<ul class="tsd-legend">
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-protected"><span class="tsd-kind-icon">Protected property</span></li>
<li class="tsd-kind-method tsd-parent-kind-class tsd-is-protected"><span class="tsd-kind-icon">Protected method</span></li>
<li class="tsd-kind-accessor tsd-parent-kind-class tsd-is-protected"><span class="tsd-kind-icon">Protected accessor</span></li>
</ul>
<ul class="tsd-legend">
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-private"><span class="tsd-kind-icon">Private property</span></li>
<li class="tsd-kind-method tsd-parent-kind-class tsd-is-private"><span class="tsd-kind-icon">Private method</span></li>
<li class="tsd-kind-accessor tsd-parent-kind-class tsd-is-private"><span class="tsd-kind-icon">Private accessor</span></li>
</ul>
<ul class="tsd-legend">
<li class="tsd-kind-property tsd-parent-kind-class tsd-is-static"><span class="tsd-kind-icon">Static property</span></li>
<li class="tsd-kind-call-signature tsd-parent-kind-class tsd-is-static"><span class="tsd-kind-icon">Static method</span></li>
</ul>
</div>
</div>
</footer>
<div class="container tsd-generator">
<p>Generated using <a href="http://typedoc.io" target="_blank">TypeDoc</a></p>
</div>
<div class="overlay"></div>
<script src="assets/js/main.js"></script>
<script>if (location.protocol == 'file:') document.write('<script src="assets/js/search.js"><' + '/script>');</script>
</body>
</html>