join2
Version:
combine stream chunks pairwise
117 lines (94 loc) • 3.18 kB
Markdown
Sometimes you want to work on a text stream one line at a time.
[split2](https://www.npmjs.com/package/split2) lets you do just that, but it's
got an annoying limitation: either it will
1. strip out all newlines for you, in which case you need to add them back in
yourself, or
2. you can [keep the newlines](https://github.com/dominictarr/split#keep-matched-splitter),
but they'll be in separate chunks all on their own, and you're no longer
working line-by-line anymore.
The problem with the first approach is that you lose information. Did the
original stream end with a newline or not? Did it use Unix or Windows-style
newlines, or perhaps [some other line boundary supported by Unicode](
http://www.unicode.org/reports/tr18/#Line_Boundaries)? If you can't abide that,
you've got to use option #2.
This is where join2 comes in. It's a simple transform that combines every pair
of chunks into a single chunk. So you setup split2 with a newline-capturing
regex, then pipe it straight into join2, and out comes your original stream,
chunked line by line.
Example
-------
```javascript
const split2 = require('split2')
const join2 = require('join2')
tap.test('Add line numbers to every line', t => {
const addLineNos = new Transform({
objectMode: true,
transform: (chunk, encoding, callback) => {
if (this.lineNo === undefined) this.lineNo = 0
this.lineNo++
callback(null, this.lineNo + ' ' + chunk)
}
})
const inputChunks = [
'so much',
' depends\nupon',
'\n\n',
'a red wheel\n',
'bar',
'row\n'
]
const expectedOutputChunks = [
'1 so much depends\n',
'2 upon\n',
'3 \n',
'4 a red wheel\n',
'5 barrow\n'
]
t.plan(expectedOutputChunks.length)
const dest = new Writable({
objectMode: true,
write: (chunk, encoding, callback) => {
const expected = expectedOutputChunks.shift()
t.equal(chunk, expected)
callback()
}
})
const src = split2(/(\n)/) /* Notice the capturing parens; see
* https://github.com/dominictarr/split#keep-matched-splitter */
src.pipe(join2())
.pipe(addLineNos)
.pipe(dest)
inputChunks.forEach(chunk => src.write(chunk))
src.end()
})
```
Usage
-----
**join2(options)**
- `options` *Object* Options object passed to Stream.Transform. **Default:** `{}`
- Returns: *stream.Transform*
Creates a Transform stream that concatenates every two input chunks into a
single output chunk. Returns said transform stream.
If the input stream ends on an odd-numbered chunk, that last chunk is output
as-is.
```javascript
tap.test('Odd chunk count', t => {
t.plan(1)
const src = join2()
const dest = new Writable({
write (chunk, encoding, callback) {
t.equal(chunk.toString(), 'foobar')
}
})
src.pipe(dest)
src.end('foobar')
})
```
Contributing
------------
You're welcome to contribute to this project following the
[C4 process](https://rfc.zeromq.org/spec:42/C4/).
All patches must follow [standard style](https://standardjs.com/) and have 100%
test coverage. You can make sure of this by adding
./.pre-commit
to your git pre-commit hook.