learnyounode

<!doctype html>  <html lang="en"> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <title></title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="created-with" content="https://github.com/rvagg/gfm2html"> <style type="text/css"> /* most of normalize.css */ article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section,summary{display:block;}[hidden],template{display:none;}html{font-family:sans-serif;/*1*/-ms-text-size-adjust:100%;/*2*/-webkit-text-size-adjust:100%;/*2*/}body{margin:0;}a{background:transparent;}a:focus{outline:thindotted;}a:active,a:hover{outline:0;}h1{font-size:2em;margin:0.67em0;}abbr[title]{border-bottom:1pxdotted;}b,strong{font-weight:bold;}dfn{font-style:italic;}hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0;}mark{background:#ff0;color:#000;}code,kbd,pre,samp{font-family:monospace,serif;font-size:1em;}pre{white-space:pre-wrap;}q{quotes:"\201C""\201D""\2018""\2019";}small{font-size:80%;}sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline;}sup{top:-0.5em;}sub{bottom:-0.25em;}img{border:0;}svg:not(:root){overflow:hidden;}table{border-collapse:collapse;border-spacing:0;} html { font: 14px 'Helvetica Neue', Helvetica, arial, freesans, clean, sans-serif; } .container { line-height: 1.6; color: #333; background: #eee; border-radius: 3px; padding: 3px; width: 790px; margin: 10px auto; } .body-content { background-color: #fff; border: 1px solid #CACACA; padding: 30px; } .body-content > *:first-child { margin-top: 0 !important; } a, a:visited { color: #4183c4; text-decoration: none; } a:hover { text-decoration: underline; } p, blockquote, ul, ol, dl, table, pre { margin: 15px 0; } .markdown-body h1 , .markdown-body h2 , .markdown-body h3 , .markdown-body h4 , .markdown-body h5 , .markdown-body h6 { margin: 20px 0 10px; padding: 0; font-weight: bold; } h1 { font-size: 2.5em; color: #000; border-bottom: 1px solid #ddd; } h2 { font-size: 2em; border-bottom: 1px solid #eee; color: #000; } img { max-width: 100%; } hr { background: transparent url("/img/hr.png") repeat-x 0 0; border: 0 none; color: #ccc; height: 4px; padding: 0; } table { border-collapse: collapse; border-spacing: 0; } tr:nth-child(2n) { background-color: #f8f8f8; } .markdown-body tr { border-top: 1px solid #ccc; background-color: #fff; } td, th { border: 1px solid #ccc; padding: 6px 13px; } th { font-weight: bold; } blockquote { border-left: 4px solid #ddd; padding: 0 15px; color: #777; } blockquote > :last-child, blockquote > :first-child { margin-bottom: 0px; } pre, code { font-size: 13px; font-family: 'UbuntuMono', monospace; white-space: nowrap; margin: 0 2px; padding: 0px 5px; border: 1px solid #eaeaea; background-color: #f8f8f8; border-radius: 3px; } pre > code { white-space: pre; } pre { overflow-x: auto; white-space: pre; padding: 10px; line-height: 150%; background-color: #f8f8f8; border-color: #ccc; } pre code, pre tt { margin: 0; padding: 0; border: 0; background-color: transparent; border: none; } .highlight .c , .highlight .cm , .highlight .cp , .highlight .c1 { color:#999988; font-style:italic; } .highlight .err { color:#a61717; background-color:#e3d2d2 } .highlight .o , .highlight .gs , .highlight .kc , .highlight .kd , .highlight .kn , .highlight .kp , .highlight .kr { font-weight:bold } .highlight .cs { color:#999999; font-weight:bold; font-style:italic } .highlight .gd { color:#000000; background-color:#ffdddd } .highlight .gd .x { color:#000000; background-color:#ffaaaa } .highlight .ge { font-style:italic } .highlight .gr , .highlight .gt { color:#aa0000 } .highlight .gh , .highlight .bp { color:#999999 } .highlight .gi { color:#000000; background-color:#ddffdd } .highlight .gi .x { color:#000000; background-color:#aaffaa } .highlight .go { color:#888888 } .highlight .gp , .highlight .nn { color:#555555 } .highlight .gu { color:#800080; font-weight:bold } .highlight .kt { color:#445588; font-weight:bold } .highlight .m , .highlight .mf , .highlight .mh , .highlight .mi , .highlight .mo , .highlight .il { color:#009999 } .highlight .s , .highlight .sb , .highlight .sc , .highlight .sd , .highlight .s2 , .highlight .se , .highlight .sh , .highlight .si , .highlight .sx , .highlight .s1 { color:#d14 } .highlight .n { color:#333333 } .highlight .na , .highlight .no , .highlight .nv , .highlight .vc , .highlight .vg , .highlight .vi , .highlight .nb { color:#0086B3 } .highlight .nc { color:#445588; font-weight:bold } .highlight .ni { color:#800080 } .highlight .ne , .highlight .nf { color:#990000; font-weight:bold } .highlight .nt { color:#000080 } .highlight .ow { font-weight:bold } .highlight .w { color:#bbbbbb } .highlight .sr { color:#009926 } .highlight .ss { color:#990073 } .highlight .gc { color:#999; background-color:#EAF2F5 }</style> </head> <body> <div class="container"> <div class="body-content"><h1 id="bl-bufferlist-">bl (BufferList)</h1> <a href="http://travis-ci.org/rvagg/bl"><img src="https://secure.travis-ci.org/rvagg/bl.png" alt="Build Status"></a> A Node.js Buffer list collector, reader and streamer thingy. bl is a storage object for collections of Node Buffers, exposing them with the main Buffer readable API. Also works as a duplex stream so you can collect buffers from a stream that emits them and emit buffers to a stream that consumes them! The original buffers are kept intact and copies are only done as necessary. Any reads that require the use of a single original buffer will return a slice of that buffer only (which references the same memory as the original buffer). Reads that span buffers perform concatenation as required and return the results transparently. <div class="highlight"><pre>const BufferList = require('bl') var bl = new BufferList() bl.append(new Buffer('abcd')) bl.append(new Buffer('efg')) bl.append('hi') // bl will also accept & convert Strings bl.append(new Buffer('j')) bl.append(new Buffer([ 0x3, 0x4 ])) console.log(bl.length) // 12 console.log(bl.slice(0, 10).toString('ascii')) // 'abcdefghij' console.log(bl.slice(3, 10).toString('ascii')) // 'defghij' console.log(bl.slice(3, 6).toString('ascii')) // 'def' console.log(bl.slice(3, 8).toString('ascii')) // 'defgh' console.log(bl.slice(5, 10).toString('ascii')) // 'fghij' // or just use toString! console.log(bl.toString()) // 'abcdefghij\u0003\u0004' console.log(bl.toString('ascii', 3, 8)) // 'defgh' console.log(bl.toString('ascii', 5, 10)) // 'fghij' // other standard Buffer readables console.log(bl.readUInt16BE(10)) // 0x0304 console.log(bl.readUInt16LE(10)) // 0x0403 </pre></div> Give it a callback in the constructor and use it just like <a href="https://github.com/maxogden/node-concat-stream">concat-stream</a>: <div class="highlight"><pre>const bl = require('bl') , fs = require('fs') fs.createReadStream('README.md') .pipe(bl(function (err, data) { // note 'new' isn't strictly required // `data` is a complete Buffer object containing the full data console.log(data.toString()) }) </pre></div> Note that when you use the callback method like this, the resulting <code>data</code> parameter is a concatenation of all <code>Buffer</code> objects in the list. If you want to avoid the overhead of this concatenation (in cases of extreme performance consciousness), then avoid the callback method and just listen to <code>'end'</code> instead, like a standard Stream. Or to fetch a URL using <a href="https://github.com/substack/hyperquest">hyperquest</a> (should work with <a href="http://github.com/mikeal/request">request</a> and even plain Node http too!): <div class="highlight"><pre>const hyperquest = require('hyperquest') , bl = require('bl') , url = 'https://raw.github.com/rvagg/bl/master/README.md' hyperquest(url).pipe(bl(function (err, data) { console.log(data.toString()) })) </pre></div> Or, use it as a readable stream to recompose a list of Buffers to an output source: <div class="highlight"><pre>const BufferList = require('bl') , fs = require('fs') var bl = new BufferList() bl.append(new Buffer('abcd')) bl.append(new Buffer('efg')) bl.append(new Buffer('hi')) bl.append(new Buffer('j')) bl.pipe(fs.createWriteStream('gibberish.txt')) </pre></div> <h2 id="api">API</h2> <ul> <li><a href="#ctor"><code>new BufferList([ callback ])</code></a></li> <li><a href="#length"><code>bl.length</code></a></li> <li><a href="#append"><code>bl.append(buffer)</code></a></li> <li><a href="#get"><code>bl.get(index)</code></a></li> <li><a href="#slice"><code>bl.slice([ start[, end ] ])</code></a></li> <li><a href="#copy"><code>bl.copy(dest, [ destStart, [ srcStart [, srcEnd ] ] ])</code></a></li> <li><a href="#duplicate"><code>bl.duplicate()</code></a></li> <li><a href="#consume"><code>bl.consume(bytes)</code></a></li> <li><a href="#toString"><code>bl.toString([encoding, [ start, [ end ]]])</code></a></li> <li><a href="#readXX"><code>bl.readDoubleBE()</code>, <code>bl.readDoubleLE()</code>, <code>bl.readFloatBE()</code>, <code>bl.readFloatLE()</code>, <code>bl.readInt32BE()</code>, <code>bl.readInt32LE()</code>, <code>bl.readUInt32BE()</code>, <code>bl.readUInt32LE()</code>, <code>bl.readInt16BE()</code>, <code>bl.readInt16LE()</code>, <code>bl.readUInt16BE()</code>, <code>bl.readUInt16LE()</code>, <code>bl.readInt8()</code>, <code>bl.readUInt8()</code></a></li> <li><a href="#streams">Streams</a></li> </ul> <hr> <a name="ctor"></a> <h3 id="new-bufferlist-callback-buffer-buffer-array-">new BufferList([ callback | buffer | buffer array ])</h3> The constructor takes an optional callback, if supplied, the callback will be called with an error argument followed by a reference to the bl instance, when <code>bl.end()</code> is called (i.e. from a piped stream). This is a convenient method of collecting the entire contents of a stream, particularly when the stream is chunky, such as a network stream. Normally, no arguments are required for the constructor, but you can initialise the list by passing in a single <code>Buffer</code> object or an array of <code>Buffer</code> object. <code>new</code> is not strictly required, if you don't instantiate a new object, it will be done automatically for you so you can create a new instance simply with: <div class="highlight"><pre>var bl = require('bl') var myinstance = bl() // equivilant to: var BufferList = require('bl') var myinstance = new BufferList() </pre></div> <hr> <a name="length"></a> <h3 id="bl-length">bl.length</h3> Get the length of the list in bytes. This is the sum of the lengths of all of the buffers contained in the list, minus any initial offset for a semi-consumed buffer at the beginning. Should accurately represent the total number of bytes that can be read from the list. <hr> <a name="append"></a> <h3 id="bl-append-buffer-">bl.append(buffer)</h3> <code>append(buffer)</code> adds an additional buffer to the internal list. <hr> <a name="get"></a> <h3 id="bl-get-index-">bl.get(index)</h3> <code>get()</code> will return the byte at the specified index. <hr> <a name="slice"></a> <h3 id="bl-slice-start-end-">bl.slice([ start, [ end ] ])</h3> <code>slice()</code> returns a new <code>Buffer</code> object containing the bytes within the range specified. Both <code>start</code> and <code>end</code> are optional and will default to the beginning and end of the list respectively. If the requested range spans a single internal buffer then a slice of that buffer will be returned which shares the original memory range of that Buffer. If the range spans multiple buffers then copy operations will likely occur to give you a uniform Buffer. <hr> <a name="copy"></a> <h3 id="bl-copy-dest-deststart-srcstart-srcend-">bl.copy(dest, [ destStart, [ srcStart [, srcEnd ] ] ])</h3> <code>copy()</code> copies the content of the list in the <code>dest</code> buffer, starting from <code>destStart</code> and containing the bytes within the range specified with <code>srcStart</code> to <code>srcEnd</code>. <code>destStart</code>, <code>start</code> and <code>end</code> are optional and will default to the beginning of the <code>dest</code> buffer, and the beginning and end of the list respectively. <hr> <a name="duplicate"></a> <h3 id="bl-duplicate-">bl.duplicate()</h3> <code>duplicate()</code> performs a shallow-copy of the list. The internal Buffers remains the same, so if you change the underlying Buffers, the change will be reflected in both the original and the duplicate. This method is needed if you want to call <code>consume()</code> or <code>pipe()</code> and still keep the original list.Example: <div class="highlight"><pre>var bl = new BufferList() bl.append('hello') bl.append(' world') bl.append('\n') bl.duplicate().pipe(process.stdout, { end: false }) console.log(bl.toString()) </pre></div> <hr> <a name="consume"></a> <h3 id="bl-consume-bytes-">bl.consume(bytes)</h3> <code>consume()</code> will shift bytes off the start of the list. The number of bytes consumed don't need to line up with the sizes of the internal Buffers—initial offsets will be calculated accordingly in order to give you a consistent view of the data. <hr> <a name="toString"></a> <h3 id="bl-tostring-encoding-start-end-">bl.toString([encoding, [ start, [ end ]]])</h3> <code>toString()</code> will return a string representation of the buffer. The optional <code>start</code> and <code>end</code> arguments are passed on to <code>slice()</code>, while the <code>encoding</code> is passed on to <code>toString()</code> of the resulting Buffer. See the <a href="http://nodejs.org/docs/latest/api/buffer.html#buffer_buf_tostring_encoding_start_end">Buffer#toString()</a> documentation for more information. <hr> <a name="readXX"></a> <h3 id="-code-bl-readdoublebe-code-code-bl-readdoublele-code-code-bl-readfloatbe-code-code-bl-readfloatle-code-code-bl-readint32be-code-code-bl-readint32le-code-code-bl-readuint32be-code-code-bl-readuint32le-code-code-bl-readint16be-code-code-bl-readint16le-code-code-bl-readuint16be-code-code-bl-readuint16le-code-code-bl-readint8-code-code-bl-readuint8-code-"><code>bl.readDoubleBE()</code>, <code>bl.readDoubleLE()</code>, <code>bl.readFloatBE()</code>, <code>bl.readFloatLE()</code>, <code>bl.readInt32BE()</code>, <code>bl.readInt32LE()</code>, <code>bl.readUInt32BE()</code>, <code>bl.readUInt32LE()</code>, <code>bl.readInt16BE()</code>, <code>bl.readInt16LE()</code>, <code>bl.readUInt16BE()</code>, <code>bl.readUInt16LE()</code>, <code>bl.readInt8()</code>, <code>bl.readUInt8()</code></h3> All of the standard byte-reading methods of the <code>Buffer</code> interface are implemented and will operate across internal Buffer boundaries transparently. See the <code><a href="http://nodejs.org/docs/latest/api/buffer.html">Buffer</a></code> documentation for how these work. <hr> <a name="streams"></a> <h3 id="streams">Streams</h3> bl is a Node <a href="http://nodejs.org/docs/latest/api/stream.html#stream_class_stream_duplex">Duplex Stream</a>, so it can be read from and written to like a standard Node stream. You can also <code>pipe()</code> to and from a bl instance. <hr> <h2 id="contributors">Contributors</h2> bl is brought to you by the following hackers: <ul> <li><a href="https://github.com/rvagg">Rod Vagg</a></li> <li><a href="https://github.com/mcollina">Matteo Collina</a></li> </ul> ======= <h2 id="license">License</h2> bl is Copyright (c) 2013 Rod Vagg <a href="https://twitter.com/rvagg">@rvagg</a> and licenced under the MIT licence. All rights not explicitly granted in the MIT license are reserved. See the included LICENSE file for more details. </div> </div> </body> </html>