UNPKG

join2

Version:

combine stream chunks pairwise

117 lines (94 loc) 3.18 kB
Sometimes you want to work on a text stream one line at a time. [split2](https://www.npmjs.com/package/split2) lets you do just that, but it's got an annoying limitation: either it will 1. strip out all newlines for you, in which case you need to add them back in yourself, or 2. you can [keep the newlines](https://github.com/dominictarr/split#keep-matched-splitter), but they'll be in separate chunks all on their own, and you're no longer working line-by-line anymore. The problem with the first approach is that you lose information. Did the original stream end with a newline or not? Did it use Unix or Windows-style newlines, or perhaps [some other line boundary supported by Unicode]( http://www.unicode.org/reports/tr18/#Line_Boundaries)? If you can't abide that, you've got to use option #2. This is where join2 comes in. It's a simple transform that combines every pair of chunks into a single chunk. So you setup split2 with a newline-capturing regex, then pipe it straight into join2, and out comes your original stream, chunked line by line. Example ------- ```javascript const split2 = require('split2') const join2 = require('join2') tap.test('Add line numbers to every line', t => { const addLineNos = new Transform({ objectMode: true, transform: (chunk, encoding, callback) => { if (this.lineNo === undefined) this.lineNo = 0 this.lineNo++ callback(null, this.lineNo + ' ' + chunk) } }) const inputChunks = [ 'so much', ' depends\nupon', '\n\n', 'a red wheel\n', 'bar', 'row\n' ] const expectedOutputChunks = [ '1 so much depends\n', '2 upon\n', '3 \n', '4 a red wheel\n', '5 barrow\n' ] t.plan(expectedOutputChunks.length) const dest = new Writable({ objectMode: true, write: (chunk, encoding, callback) => { const expected = expectedOutputChunks.shift() t.equal(chunk, expected) callback() } }) const src = split2(/(\n)/) /* Notice the capturing parens; see * https://github.com/dominictarr/split#keep-matched-splitter */ src.pipe(join2()) .pipe(addLineNos) .pipe(dest) inputChunks.forEach(chunk => src.write(chunk)) src.end() }) ``` Usage ----- **join2(options)** - `options` *Object* Options object passed to Stream.Transform. **Default:** `{}` - Returns: *stream.Transform* Creates a Transform stream that concatenates every two input chunks into a single output chunk. Returns said transform stream. If the input stream ends on an odd-numbered chunk, that last chunk is output as-is. ```javascript tap.test('Odd chunk count', t => { t.plan(1) const src = join2() const dest = new Writable({ write (chunk, encoding, callback) { t.equal(chunk.toString(), 'foobar') } }) src.pipe(dest) src.end('foobar') }) ``` Contributing ------------ You're welcome to contribute to this project following the [C4 process](https://rfc.zeromq.org/spec:42/C4/). All patches must follow [standard style](https://standardjs.com/) and have 100% test coverage. You can make sure of this by adding ./.pre-commit to your git pre-commit hook.