jmdict-streaming-parser/readme.md

Streaming parser for JMdict and related files.

# jmdict-streaming-parser

Streaming parser for JMdict and related files.

## API

```typescript
import { createGunzip } from 'zlib'
import { createReadStream } from 'fs'
import { JmdictTransform } from 'jmdict'
import { pipeline } from 'stream'

// Stream style
pipeline(
  createReadStream("JMdict.gz"),
  createGunzip(),
  new JmdictTransform()
).on('data', data => { console.log(data) })
```

### JmdictTransform

```typescript
class JmdictTransform extends Duplex
```

A duplex stream that reads XML data and writes plain objects subject to the rules in [§ Object structure](#object-structure).

### transform = new JmdictTransform(opts?: DuplexOptions)

Each object streamed from the transform can have one of the 3 following types. The data itself is stored in the property `data` while the type name is stored in the property `type`.

#### type === entity

An object containing keys `name` and `value` representing entities detected.

#### type === mdate

The modification date of the file, if detected. String type.

#### type === entities

The value of `transform.entities` when `mdate` is encountered.

#### type === node

##### Object structure

Each result object is transformed from the source XML.

* Text nodes are transformed into a string value keyed by `$text`.
  * If the parent XML element only has a text node as its child, the resulting object is collapsed into just a string with the text.
    * This exploits the fact that JMdict does not contain mixed text nodes and XML elements.
  * Text nodes whose sole content is a newline are ignored.
* XML elements are transformed into an object and appended into an array value in its corresponding parent object where the key is the name of the XML element.
  * Attributes of the element are merged into the object.
* Children of the root node are streamed as output.
* Entities are represented by the entity name.

This deliberate generalization is to allow for possible parsing of files similar to the JMdict.

### transform.entities

Maps entity names to entity values.