epubjs
Version:
Render ePub documents in the browser, across many devices
628 lines (566 loc) • 110 kB
HTML
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>Chapter 6. Data Access</title><link rel="stylesheet" href="core.css" type="text/css"/><meta name="generator" content="DocBook XSL Stylesheets V1.74.0"/></head><body><div class="chapter" title="Chapter 6. Data Access"><div class="titlepage"><div><div><h1 class="title"><a id="chapter_8"/>Chapter 6. Data Access</h1></div></div></div><p>Like any web server, Node <a id="I_indexterm3_d1e9472" class="indexterm"/>needs access to data stores for persistent storage; without
persistence, all you have is a brochure website, which would make using Node
pointless. In this chapter, we’ll run through the basic ways to connect to
common open source database choices and to store and retrieve data.</p><div class="sect1" title="NoSQL and Document Stores"><div class="titlepage"><div><div><h1 class="title"><a id="I_sect13_d1e9478"/>NoSQL and Document Stores</h1></div></div></div><p>The following NoSQL and document stores are increasingly popular for
web-facing applications and are easy to use with Node.</p><div class="sect2" title="CouchDB"><div class="titlepage"><div><div><h2 class="title"><a id="id839227"/>CouchDB</h2></div></div></div><p>CouchDB <a id="da6.1.1" class="indexterm"/>provides <a id="I_indexterm3_d1e9494" class="indexterm"/><a id="no6.1.1" class="indexterm"/><a id="I_indexterm3_d1e9502" class="indexterm"/>MVCC-based<sup>[<a id="id839274" href="#ftn.id839274" class="footnote">15</a>]</sup> document storage in a JavaScript environment. When
documents (records) are added or updated in CouchDB, the entire dataset
is saved to storage and older versions of that data marked obsolete.
Older versions of the record can still be merged into the newest
version, but in every case a whole new version is created and written to
contiguous memory for faster read times. CouchDB is said to be
“eventually consistent.” In a large, scalable deployment, multiple
instances can sometimes serve older, unsynced versions of records to
clients with the expectation that any changes to those records will
eventually be merged into the master.</p><div class="sect3" title="Installation"><div class="titlepage"><div><div><h3 class="title"><a id="id839285"/>Installation</h3></div></div></div><p>Specific CouchDB <a id="I_indexterm3_d1e9517" class="indexterm"/><a id="I_indexterm3_d1e9522" class="indexterm"/>libraries are not required to access the database, but
they are useful for providing a high level of abstraction and making
code easier to work with. A CouchDB server is needed to test any
examples, but it does not require a lot of work to get it
running.</p><div class="sect4" title="Installing CouchDB"><div class="titlepage"><div><div><h4 class="title"><a id="id839313"/>Installing CouchDB</h4></div></div></div><p>The most recent version of CouchDB can be installed from the
<a class="ulink" href="http://couchdb.apache.org/downloads.html">Apache project
page</a>. Installation instructions for a wide array of
platforms can be found on the <a class="ulink" href="http://wiki.apache.org/couchdb/Installation">wiki</a>.</p><p>If you’re running Windows, you will find a number of binary
installers as well as instructions for building from source. As with
many of the NoSQL options, installation is easiest and best
supported on a Linux-based system, but don’t be dissuaded.</p></div><div class="sect4" title="Installing CouchDB’s Node module"><div class="titlepage"><div><div><h4 class="title"><a id="id839336"/>Installing CouchDB’s Node module</h4></div></div></div><p>Additional modules <a id="I_indexterm3_d1e9546" class="indexterm"/><a id="I_indexterm3_d1e9551" class="indexterm"/><a id="I_indexterm3_d1e9556" class="indexterm"/>are not strictly necessary, because CouchDB exposes
all of its services through REST, as described in more detail
later.</p></div></div><div class="sect3" title="Using CouchDB over HTTP"><div class="titlepage"><div><div><h3 class="title"><a id="id839372"/>Using CouchDB over HTTP</h3></div></div></div><p>One of the nice things about CouchDB is that its API is actually
all just <a id="ht6.1.1.2" class="indexterm"/><a id="co6.1.1.2" class="indexterm"/>HTTP. Because Node is great at interacting with HTTP,
this means it is really easy to work with CouchDB. Exploiting this
fact, it is possible to perform database operations <a id="do6.1.1.2" class="indexterm"/>directly without any additional client libraries.</p><p><a class="xref" href="ch06.html#example6-1" title="Example 6-1. Retrieving a list of CouchDB stores via HTTP">Example 6-1</a> shows how to generate a list of
databases in the current CouchDB installation. In this case, there is
no authentication or administrative permission on the CouchDB server—a
decidedly bad idea for a database connected to the Internet, but
suitable for demonstration purposes.</p><div class="example"><a id="example6-1"/><p class="title">Example 6-1. Retrieving a list of CouchDB stores via HTTP</p><div class="example-contents"><pre class="programlisting">var http = require('http');
http.createServer(function (req, res) {
var client = http.createClient(5984, "127.0.0.1");
var request = client.request("GET", "/_all_dbs");
request.end();
request.on("response", function(response) {
var responseBody = "";
response.on("data", function(chunk) {
responseBody += chunk;
});
response.on("end", function() {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write(responseBody);
res.end();
});
});
}).listen(8080);</pre></div></div><p>A client connection is created with the <code class="literal">http</code> library. Nothing distinguishes this
connection from any other <code class="literal">http</code>
<a id="I_indexterm3_d1e9599" class="indexterm"/>connection; because CouchDB is RESTful, no additional
communication protocol is needed. Of special note is the <code class="literal">request.end()</code> line inside the <code class="literal">createServer</code> method. If this line is
omitted, the request will hang.</p><p>As mentioned earlier, all CouchDB methods are exposed in HTTP
calls. Creating and deleting databases, therefore, involves making the
appropriate <a id="I_indexterm3_d1e9613" class="indexterm"/><a id="I_indexterm3_d1e9616" class="indexterm"/>PUT and DELETE statements against the server, as
demonstrated in <a class="xref" href="ch06.html#example6-2" title="Example 6-2. Creating a CouchDB database">Example 6-2</a>.</p><div class="example"><a id="example6-2"/><p class="title">Example 6-2. Creating a CouchDB database</p><div class="example-contents"><pre class="programlisting"> var client = http.createClient(5984, "127.0.0.1")
var request = client.request("PUT", "/dbname");
request.end();
request.on("response", function(response) {
response.on("end", function() {
if ( response.statusCode == 201 ) {
console.log("Database successfully created.");
} else {
console.log("Could not create database.");
}
});
});</pre></div></div><p>Here, <code class="literal">/dbname</code> refers to the
resource being accessed. Combined with a PUT command, CouchDB is
instructed to create a new database called <code class="literal">dbname</code>. An HTTP response code of 201
confirms that the database was created.</p><p>As shown in <a class="xref" href="ch06.html#example6-3" title="Example 6-3. Deleting a CouchDB database">Example 6-3</a>, deleting the resource
is the reverse of a PUT: the DELETE command. An HTTP response code of
200 confirms the request was completed successfully.</p><div class="example"><a id="example6-3"/><p class="title">Example 6-3. Deleting a CouchDB database</p><div class="example-contents"><pre class="programlisting"> var client = http.createClient(5984, "127.0.0.1")
var request = client.request("DELETE", "/dbname");
request.end();
request.on("response", function(response) {
response.on("end", function() {
if ( response.statusCode == 200 ) {
console.log("Deleted database.");
} else {
console.log("Could not delete database.");
}
});
});</pre></div></div><p>These elements aren’t very useful on their own, but they can be
put together to form a very basic (if unfriendly) database manager
using the methods shown in <a class="xref" href="ch06.html#example6-4" title="Example 6-4. A simple CouchDB database creation form">Example 6-4</a>.</p><div class="example"><a id="example6-4"/><p class="title">Example 6-4. A simple CouchDB database creation form</p><div class="example-contents"><pre class="programlisting">var http = require('http');
var qs = require('querystring');
var url = require('url');
var dbHost = "127.0.0.1";
var dbPort = 5984;
deleteDb = function(res, dbpath) {
var client = http.createClient(dbPort, dbHost)
var request = client.request("DELETE", dbpath);
request.end();
request.on("response", function(response) {
response.on("end", function() {
if ( response.statusCode == 200 ) {
showDbs(res, "Deleted database.");
} else {
showDbs(res, "Could not delete database.");
}
});
});
}
createDb = function(res, dbname) {
var client = http.createClient(dbPort, dbHost)
var request = client.request("PUT", "/" + dbname);
request.end();
request.on("response", function(response) {
response.on("end", function() {
if ( response.statusCode == 201 ) {
showDbs(res, dbname + " created.");
} else {
showDbs(res, "Could not create " + dbname);
}
});
});
}
showDbs = function(res, message) {
var client = http.createClient(dbPort, dbHost);
var request = client.request("GET", "/_all_dbs");
request.end();
request.on("response", function(response) {
var responseBody = "";
response.on("data", function(chunk) {
responseBody += chunk;
});
response.on("end", function() {
res.writeHead(200, {'Content-Type': 'text/html'});
res.write("<form method='post'>");
res.write("New Database Name: <input type='text' name='dbname' />");
res.write("<input type='submit' />");
res.write("</form>");
if ( null != message ) res.write("<h1>" + message + "</h1>");
res.write("<h1>Active databases:</h1>");
res.write("<ul>");
var dblist = JSON.parse(responseBody);
for ( i = 0; i < dblist.length; i++ ) {
var dbname = dblist[i];
res.write("<li><a href='/" + dbname + "'>"+dbname+"</a></li>");
}
res.write("</ul>");
res.end();
});
});
};
http.createServer(function (req, res) {
if ( req.method == 'POST' ) {
// Parse the request
var body = '';
req.on('data', function (data) {
body += data;
});
req.on('end', function () {
var POST = qs.parse(body);
var dbname = POST['dbname'];
if ( null != dbname ) {
// Create the DB
createDb(res,dbname);
} else {
showDbs(res, "Bad DB name, cannot create database.");
}
});
} else {
var path = url.parse(req.url).pathname;
if ( path != "/" ) {
deleteDb(res,path);
} else {
showDbs(res);
}
}
}).listen(8080);</pre></div></div></div><div class="sect3" title="Using node-couchdb"><div class="titlepage"><div><div><h3 class="title"><a id="id839588"/>Using node-couchdb</h3></div></div></div><p>Knowing <a id="I_indexterm3_d1e9662" class="indexterm"/><a id="co6.1.1.3" class="indexterm"/>how to work <a id="I_indexterm3_d1e9673" class="indexterm"/><a id="I_indexterm3_d1e9675" class="indexterm"/><a id="I_indexterm3_d1e9677" class="indexterm"/>with CouchDB over HTTP is useful, but this approach is
verbose. Although it has the advantage of not needing external
libraries, most developers opt for higher-level abstraction layers,
regardless of how simple their database’s native driver implementation
is. In this section, we look at the node-couchdb package, which
simplifies the interface between Node <a id="I_indexterm3_d1e9680" class="indexterm"/><a id="I_indexterm3_d1e9685" class="indexterm"/>and CouchDB.</p><p>You can install the drivers for CouchDB using
<code class="literal">npm</code>:</p><a id="I_programlisting3_d1e9697"/><pre class="programlisting">npm install felix-couchdb</pre><div class="sect4" title="Working with databases"><div class="titlepage"><div><div><h4 class="title"><a id="id839685"/>Working with databases</h4></div></div></div><p>The module’s first obvious <a id="I_indexterm3_d1e9704" class="indexterm"/><a id="I_indexterm3_d1e9709" class="indexterm"/><a id="I_indexterm3_d1e9714" class="indexterm"/>benefit is succinct program code, as demonstrated in
<a class="xref" href="ch06.html#example6-5" title="Example 6-5. Creating a table in CouchDB">Example 6-5</a>.</p><div class="example"><a id="example6-5"/><p class="title">Example 6-5. Creating a table in CouchDB</p><div class="example-contents"><pre class="programlisting">var dbHost = "127.0.0.1";
var dbPort = 5984;
var dbName = 'users';
var couchdb = require('felix-couchdb');
var client = couchdb.createClient(dbPort, dbHost);
var db = client.db(dbName);
db.exists(function(err, exists) {
if (!exists) {
db.create();
console.log('Database ' + dbName + ' created.');
} else {
console.log('Database ' + dbName + ' exists.');
}
});</pre></div></div><p>This example checks for a database called <code class="literal">users</code>, creating one if it doesn’t already
exist. Notice the similarities between the <code class="literal">createClient</code> function <a id="I_indexterm3_d1e9733" class="indexterm"/>call here and the one from the <code class="literal">http</code> module demonstrated earlier. This is
no accident; even though the module makes CouchDB’s interfaces
easier to work with, in the end you are using HTTP to transmit
data.</p></div><div class="sect4" title="Creating documents"><div class="titlepage"><div><div><h4 class="title"><a id="id839767"/>Creating documents</h4></div></div></div><p>In <a class="xref" href="ch06.html#example6-6" title="Example 6-6. Creating a document in CouchDB">Example 6-6</a>, we’ll <a id="I_indexterm3_d1e9749" class="indexterm"/><a id="I_indexterm3_d1e9754" class="indexterm"/>save a document into the CouchDB database created in
the previous example.</p><div class="example"><a id="example6-6"/><p class="title">Example 6-6. Creating a document in CouchDB</p><div class="example-contents"><pre class="programlisting">var dbHost = "127.0.0.1";
var dbPort = 5984;
var dbName = 'users';
var couchdb = require('felix-couchdb');
var client = couchdb.createClient(dbPort, dbHost);
var user = {
name: {
first: 'John',
last: 'Doe'
}
}
var db = client.db(dbName);
db.saveDoc('jdoe', user, function(err, doc) {
if( err) {
console.log(JSON.stringify(err));
} else {
console.log('Saved user.');
}
});</pre></div></div><p>This example creates a user named John Doe in the database
with the username <span class="emphasis"><em>jdoe</em></span> as its identity. Notice
the user is created as a JSON object and passed directly into the
client. No more work is needed to parse the information.</p><p>After running this example, the user can be accessed in the
web browser at
<span class="emphasis"><em>http://127.0.0.1:5984/users/jdoe</em></span>.</p></div><div class="sect4" title="Reading documents"><div class="titlepage"><div><div><h4 class="title"><a id="id839825"/>Reading documents</h4></div></div></div><p>Once documents <a id="I_indexterm3_d1e9783" class="indexterm"/><a id="I_indexterm3_d1e9788" class="indexterm"/><a id="I_indexterm3_d1e9793" class="indexterm"/>are stored in CouchDB, they can be retrieved again as
objects, as shown in <a class="xref" href="ch06.html#example6-7" title="Example 6-7. Retrieving a record from CouchDB">Example 6-7</a>.</p><div class="example"><a id="example6-7"/><p class="title">Example 6-7. Retrieving a record from CouchDB</p><div class="example-contents"><pre class="programlisting">var dbHost = "127.0.0.1";
var dbPort = 5984;
var dbName = 'users';
var couchdb = require('felix-couchdb');
var client = couchdb.createClient(dbPort, dbHost);
var db = client.db(dbName);
db.getDoc('jdoe', function(err,doc) {
console.log(doc);
});</pre></div></div><p>The output from this query is:</p><pre class="screen">{ _id: 'jdoe',
_rev: '3-67a7414d073c9ebce3d4af0a0e49691d',
name: { first: 'John', last: 'Doe' }
}</pre><p>There are three steps happening here:</p><div class="orderedlist"><ol class="orderedlist"><li class="listitem"><p>Connect to the database server using <code class="literal">createClient</code>.</p></li><li class="listitem"><p>Select the document store using the client’s <code class="literal">db</code> command.</p></li><li class="listitem"><p>Get the document using the <a id="I_indexterm3_d1e9828" class="indexterm"/><a id="I_indexterm3_d1e9833" class="indexterm"/>database’s <code class="literal">getDoc</code> command.</p></li></ol></div><p>In this case, the record with ID <code class="literal">jdoe</code>—created in the previous example—is
retrieved from the database. If the record did not exist (because it
was deleted or not yet inserted), the callback’s error parameter
would contain data about the error.</p></div><div class="sect4" title="Updating documents"><div class="titlepage"><div><div><h4 class="title"><a id="id839944"/>Updating documents</h4></div></div></div><p>Updating documents <a id="I_indexterm3_d1e9850" class="indexterm"/><a id="I_indexterm3_d1e9855" class="indexterm"/>uses the same <code class="literal">saveDoc</code> command <a id="I_indexterm3_d1e9864" class="indexterm"/><a id="I_indexterm3_d1e9869" class="indexterm"/>as creating documents. If CouchDB detects an existing
record with the same ID, it will overwrite the old one.</p><p><a class="xref" href="ch06.html#example6-8" title="Example 6-8. Updating a record in CouchDB">Example 6-8</a> demonstrates how to update a
document after reading it from the data store.</p><div class="example"><a id="example6-8"/><p class="title">Example 6-8. Updating a record in CouchDB</p><div class="example-contents"><pre class="programlisting">var dbHost = "127.0.0.1";
var dbPort = 5984;
var dbName = 'users';
var couchdb = require('felix-couchdb');
var client = couchdb.createClient(dbPort, dbHost);
var db = client.db(dbName);
db.getDoc('jdoe', function(err,doc) {
doc.name.first = 'Johnny';
doc.email = 'jdoe@johndoe.com';
db.saveDoc('jdoe', doc );
db.getDoc('jdoe', function(err,revisedUser) {
console.log(revisedUser);
});
});</pre></div></div><p>The output from this operation is:</p><pre class="screen">{ _id: 'jdoe',
_rev: '7-1fb9a3bb6db27cbbbf1c74b2d601ccaa',
name: { first: 'Johnny', last: 'Doe' },
email: 'jdoe@johndoe.com'
}</pre><p>This example reads information about the
<span class="emphasis"><em>jdoe</em></span> user from the data store, gives it an
email address and a new first name, and saves it back into
CouchDB.</p><p>Notice that <code class="literal">saveDoc</code> and
<code class="literal">getDoc</code> follow the initial read,
instead of putting <code class="literal">getDoc</code> inside
<code class="literal">saveDoc</code>’s callback. The CouchDB
drivers queue commands and execute them sequentially, so this
example will not result in a race condition where the document read
completes before the updates are saved.</p></div><div class="sect4" title="Deleting documents"><div class="titlepage"><div><div><h4 class="title"><a id="id840052"/>Deleting documents</h4></div></div></div><p>To delete a document from CouchDB, <a id="I_indexterm3_d1e9909" class="indexterm"/><a id="I_indexterm3_d1e9914" class="indexterm"/><a id="I_indexterm3_d1e9919" class="indexterm"/>you need to supply both an ID and a revision number.
Fortunately, this is easy after a read, as shown in <a class="xref" href="ch06.html#example6-9" title="Example 6-9. Deleting from CouchDB">Example 6-9</a>.</p><div class="example"><a id="example6-9"/><p class="title">Example 6-9. Deleting from CouchDB</p><div class="example-contents"><pre class="programlisting">var dbHost = "127.0.0.1";
var dbPort = 5984;
var dbName = 'users';
var couchdb = require('felix-couchdb');
var client = couchdb.createClient(dbPort, dbHost);
var db = client.db(dbName);
db.getDoc('jdoe', function(err,doc) {
db.removeDoc(doc._id, doc._rev);
});</pre></div></div><p>After connecting to the CouchDB datastore, a <code class="literal">getDoc</code> command is issued here to get the
internal ID (the <code class="literal">_id</code> field) and
revision number (<code class="literal">_rev</code> field) for
that document. Once this information has been obtained, <a id="I_indexterm3_d1e9943" class="indexterm"/><a id="I_indexterm3_d1e9948" class="indexterm"/>a <code class="literal">removeDoc</code> command
is issued, which sends a <code class="literal">DELETE</code>
request to the <a id="I_indexterm3_d1e9959" class="indexterm"/><a id="I_indexterm3_d1e9961" class="indexterm"/><a id="I_indexterm3_d1e9963" class="indexterm"/>database.</p></div></div></div><div class="sect2" title="Redis"><div class="titlepage"><div><div><h2 class="title"><a id="id840180"/>Redis</h2></div></div></div><p>Redis is a <a id="da6.1.2" class="indexterm"/><a id="do6.1.2" class="indexterm"/><a id="I_indexterm3_d1e9981" class="indexterm"/><a id="no6.1.2" class="indexterm"/>memory-centric key-value store with persistence that will
feel very familiar if you have experience with key-value caches such as
Memcache. Redis is used when performance and scaling are important; in
many cases, developers choose to use it as a cache for data retrieved
from a relational database such as MySQL, although it is capable of much
more.</p><p>Beyond its key-value storage capabilities, Redis provides
network-accessible shared memory, is a nonblocking event bus, and
exposes subscription and publishing capabilities.</p><div class="sect3" title="Installation"><div class="titlepage"><div><div><h3 class="title"><a id="id840242"/>Installation</h3></div></div></div><p>As with many of the <a id="I_indexterm3_d1e10002" class="indexterm"/><a id="I_indexterm3_d1e10007" class="indexterm"/>rest of the database engines, using Redis requires
installing the database application as well as the Node drivers to
communicate with it.</p><div class="sect4" title="Installing Redis"><div class="titlepage"><div><div><h4 class="title"><a id="id840269"/>Installing Redis</h4></div></div></div><p>Redis is available in <a class="ulink" href="http://redis.io/download">source form</a>. There isn’t
anything to do in the way of configuration; just download and
compile per the instructions on the website.</p><p>If you are using Windows, you are on your own at the time of
this writing because Redis is not supported on Windows. Fortunately,
there is a passionate community behind Redis development, and
several ports have been made available for both Cygwin and native
compilation. The port at <a class="ulink" href="https://github.com/dmajkic/redis">https://github.com/dmajkic/redis</a> compiles to a native
Windows binary using MinGW.</p></div><div class="sect4" title="Installing Redis’s Node module"><div class="titlepage"><div><div><h4 class="title"><a id="id840291"/>Installing Redis’s Node module</h4></div></div></div><p>The <code class="literal">redis</code> module is available <a id="I_indexterm3_d1e10033" class="indexterm"/><a id="I_indexterm3_d1e10038" class="indexterm"/>from <a class="ulink" href="https://github.com/mranney/node_redis">GitHub</a>, but can
be installed using <code class="literal">npm</code>:</p><a id="I_programlisting3_d1e10050"/><pre class="programlisting">npm install redis</pre><p>Optionally, you may install the mimimalist <code class="literal">hiredis</code> library along with Node’s
<code class="literal">redis</code> module.</p></div></div><div class="sect3" title="Basic usage"><div class="titlepage"><div><div><h3 class="title"><a id="id840351"/>Basic usage</h3></div></div></div><p><a class="xref" href="ch06.html#example6-10" title="Example 6-10. A basic get and set operation against Redis">Example 6-10</a> demonstrates a <a id="I_indexterm3_d1e10066" class="indexterm"/>basic set and get operation against Redis by
Node.</p><div class="example"><a id="example6-10"/><p class="title">Example 6-10. A basic get and set operation against Redis</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.on("error", function (err) {
console.log("Error " + err);
});
console.log("Setting key1");
client.set("key1", "My string!", redis.print);
console.log("Getting key1");
client.get("key1", function (err, reply) {
console.log("Results for key1:");
console.log(reply);
client.end();
});</pre></div></div><p>This example begins by creating a connection to the Redis
database and setting a callback to handle errors. If you are not
running an instance of the Redis server, you will receive an error
like this:</p><a id="I_programlisting3_d1e10079"/><pre class="programlisting">Error Error: Redis connection to 127.0.0.1:6379 failed - ECONNREFUSED,
Connection refused</pre><div class="tip" title="Tip"><h3 class="title">Tip</h3><p>Note the lack of callbacks in this example. If you need to
perform database reads immediately after writing, it is safer to use
a callback, to ensure your code is executed in the correct
sequence.</p></div><p>After the connection is opened, the client sets basic data for a
string key and hash key, and then reads those values back from the
store. Library calls have the same names as basic Redis <a id="I_indexterm3_d1e10086" class="indexterm"/><a id="I_indexterm3_d1e10091" class="indexterm"/><a id="I_indexterm3_d1e10096" class="indexterm"/><a id="I_indexterm3_d1e10101" class="indexterm"/><a id="I_indexterm3_d1e10104" class="indexterm"/><a id="I_indexterm3_d1e10107" class="indexterm"/>commands (<code class="literal">set</code>,
<code class="literal">hset</code>, <code class="literal">get</code>). Redis treats data
coming through the <code class="literal">set</code> command as strings, and
allows for values up to 512 MB in size.</p></div><div class="sect3" title="Hashes"><div class="titlepage"><div><div><h3 class="title"><a id="id840474"/>Hashes</h3></div></div></div><p>Hashes are <a id="ha6.1.2.3" class="indexterm"/><a id="re6.1.2.3" class="indexterm"/>objects that contain multiple keys. <a class="xref" href="ch06.html#example6-11" title="Example 6-11. Setting hash values one key at a time">Example 6-11</a> sets a single key at a time.</p><div class="example"><a id="example6-11"/><p class="title">Example 6-11. Setting hash values one key at a time</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.on("error", function (err) {
console.log("Error " + err);
});
console.log("Setting user hash");
client.hset("user", "username", "johndoe");
client.hset("user", "firstname", "john");
client.hset("user", "lastname", "doe");
client.hkeys("user", function(err,replies) {
console.log("Results for user:");
console.log(replies.length + " replies:");
replies.forEach(function (reply, i) {
console.log(i + ": " + reply );
});
client.end();
});</pre></div></div><p><a class="xref" href="ch06.html#example6-12" title="Example 6-12. Setting multiple hash values simultaneously">Example 6-12</a> shows how to set multiple
<a id="I_indexterm3_d1e10150" class="indexterm"/>keys at the same time.</p><div class="example"><a id="example6-12"/><p class="title">Example 6-12. Setting multiple hash values simultaneously</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.on("error", function (err) {
console.log("Error " + err);
});
console.log("Setting user hash");
client.hmset("user", "username", "johndoe", "firstname", "john", "lastname", "doe");
client.hkeys("user", function(err,replies) {
console.log("Results for user:");
console.log(replies.length + " replies:");
replies.forEach(function (reply, i) {
console.log(i + ": " + reply );
});
client.end();
});</pre></div></div><p>We could accomplish the same thing by providing a more
developer-friendly <a id="I_indexterm3_d1e10163" class="indexterm"/>object, rather than breaking it out into a list, as
shown in <a class="xref" href="ch06.html#example6-13" title="Example 6-13. Setting multiple hash values using an object">Example 6-13</a>.</p><div class="example"><a id="example6-13"/><p class="title">Example 6-13. Setting multiple hash values using an object</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.on("error", function (err) {
console.log("Error " + err);
});
var user = {
username: 'johndoe',
firstname: 'John',
lastname: 'Doe',
email: 'john@johndoe.com',
website: 'http://www.johndoe.com'
}
console.log("Setting user hash");
client.hmset("user", user);
client.hkeys("user", function(err,replies) {
console.log("Results for user:");
console.log(replies.length + " replies:");
replies.forEach(function (reply, i) {
console.log(i + ": " + reply );
});
client.end();
});</pre></div></div><p>Instead of manually supplying each field to Redis, you can pass
an entire object <a id="I_indexterm3_d1e10178" class="indexterm"/><a id="I_indexterm3_d1e10183" class="indexterm"/><a id="I_indexterm3_d1e10188" class="indexterm"/><a id="I_indexterm3_d1e10191" class="indexterm"/>into <code class="literal">hmset</code>, which
will parse the fields and send the correct information to <a id="I_indexterm3_d1e10198" class="indexterm"/><a id="I_indexterm3_d1e10200" class="indexterm"/>Redis.</p><div class="warning" title="Warning"><h3 class="title">Warning</h3><p>Be careful to use <code class="literal">hmset</code> and
not <code class="literal">hset</code> when adding multiple
objects. Forgetting that a single object contains multiple values is
a common pitfall.</p></div></div><div class="sect3" title="Lists"><div class="titlepage"><div><div><h3 class="title"><a id="id840660"/>Lists</h3></div></div></div><p>The list type can be <a id="I_indexterm3_d1e10217" class="indexterm"/><a id="I_indexterm3_d1e10222" class="indexterm"/><a id="I_indexterm3_d1e10225" class="indexterm"/><a id="I_indexterm3_d1e10230" class="indexterm"/><a id="I_indexterm3_d1e10235" class="indexterm"/><a id="I_indexterm3_d1e10238" class="indexterm"/>thought of as multiple values inside one key (see <a class="xref" href="ch06.html#example6-14" title="Example 6-14. Using a list in Redis">Example 6-14</a>). Because it’s possible to push content to
the beginning or end of a list, these collections are ideal for
showing ordered events, such as lists of users who have recently
received an honor.</p><div class="example"><a id="example6-14"/><p class="title">Example 6-14. Using a list in Redis</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.on("error", function (err) {
console.log("Error " + err);
});
client.lpush("pendingusers", "user1" );
client.lpush("pendingusers", "user2" );
client.lpush("pendingusers", "user3" );
client.lpush("pendingusers", "user4" );
client.rpop("pendingusers", function(err,username) {
if( !err ) {
console.log("Processing " + username);
}
client.end();
});</pre></div></div><p>The output from this example is:</p><pre class="screen">Processing user1</pre><p>This example demonstrates a first-in-first-out (FIFO)
queue<a id="I_indexterm3_d1e10255" class="indexterm"/><a id="I_indexterm3_d1e10258" class="indexterm"/> using Redis’s list commands. A real-world use for FIFO
is in registration systems: the quantity of incoming registration
requests is too great to handle in real time, so registration data is
hived off to a queue for processing outside the main application.
Registrations will be processed in the order they were received, but
the primary application is not slowed down by handling the actual
record creation and introductory tasks such as welcome emails.</p></div><div class="sect3" title="Sets"><div class="titlepage"><div><div><h3 class="title"><a id="id840770"/>Sets</h3></div></div></div><p>Sets are used in situations where it is desirable to have
<a id="I_indexterm3_d1e10267" class="indexterm"/><a id="I_indexterm3_d1e10272" class="indexterm"/><a id="I_indexterm3_d1e10277" class="indexterm"/><a id="I_indexterm3_d1e10280" class="indexterm"/>lists of nonrepeated items, as in <a class="xref" href="ch06.html#example6-15" title="Example 6-15. Using Redis’s set commands">Example 6-15</a>.</p><div class="example"><a id="example6-15"/><p class="title">Example 6-15. Using Redis’s set commands</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.on("error", function (err) {
console.log("Error " + err);
});
client.sadd( "myteam", "Neil" );
client.sadd( "myteam", "Peter" );
client.sadd( "myteam", "Brian" );
client.sadd( "myteam", "Scott" );
client.sadd( "myteam", "Brian" );
client.smembers( "myteam", function(err, members) {
console.log( members );
client.end();
});</pre></div></div><p>The output is:</p><pre class="screen">[ 'Brian', 'Scott', 'Neil', 'Peter' ]</pre><p>Even though “Brian” was given to the list twice, he was added
only once. In a real-world situation, it would be entirely possible to
have two team members named Brian; this highlights the importance of
ensuring that your values are unique when they need to be. Otherwise,
the set can cause unintended behavior when you expect more elements
than are actually present due to the removal of repeated items.</p></div><div class="sect3" title="Sorted sets"><div class="titlepage"><div><div><h3 class="title"><a id="id840842"/>Sorted sets</h3></div></div></div><p>Like regular sets, <a id="I_indexterm3_d1e10302" class="indexterm"/><a id="I_indexterm3_d1e10305" class="indexterm"/><a id="I_indexterm3_d1e10310" class="indexterm"/><a id="I_indexterm3_d1e10315" class="indexterm"/><a id="I_indexterm3_d1e10320" class="indexterm"/><a id="I_indexterm3_d1e10323" class="indexterm"/><a id="I_indexterm3_d1e10326" class="indexterm"/>sorted sets do not allow duplicate members. Sorted sets
add the concept of <span class="emphasis"><em>weighting</em></span>, enabling
score-based operations on data such as leaderboards, top scores, and
content tables.</p><p>The producers of the American weight-loss reality show
<span class="emphasis"><em>The Biggest Loser</em></span> are real-world fans of sorted
sets. In the 11th season of the series, the contestants were split
into three groups based upon their age. On air, they had to perform a
crude sorting operation by checking a number printed on everyone’s
shirts and then line up in ascending order under the hot sun. If one
of the contestants had brought her Node- and Redis-equipped laptop to the competition, she
might have made a small program to do the work for them, such as the
one in <a class="xref" href="ch06.html#example6-16" title="Example 6-16. Ranking a sorted list using Redis">Example 6-16</a>.</p><div class="example"><a id="example6-16"/><p class="title">Example 6-16. Ranking a sorted list using Redis</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.on("error", function (err) {
console.log("Error " + err);
});
client.zadd( "contestants", 60, "Deborah" );
client.zadd( "contestants", 65, "John" );
client.zadd( "contestants", 26, "Patrick" );
client.zadd( "contestants", 62, "Mike" );
client.zadd( "contestants", 24, "Courtney" );
client.zadd( "contestants", 39, "Jennifer" );
client.zadd( "contestants", 26, "Jessica" );
client.zadd( "contestants", 46, "Joe" );
client.zadd( "contestants", 63, "Bonnie" );
client.zadd( "contestants", 27, "Vinny" );
client.zadd( "contestants", 27, "Ramon" );
client.zadd( "contestants", 51, "Becky" );
client.zadd( "contestants", 41, "Sunny" );
client.zadd( "contestants", 47, "Antone" );
client.zadd( "contestants", 40, "John" );
client.zcard( "contestants", function( err, length ) {
if( !err ) {
var contestantCount = length;
var membersPerTeam = Math.ceil( contestantCount / 3 );
client.zrange( "contestants", membersPerTeam * 0, membersPerTeam * 1 - 1,
function(err, values) {
console.log('Young team: ' + values);
});
client.zrange( "contestants", membersPerTeam * 1, membersPerTeam * 2 - 1,
function(err, values) {
console.log('Middle team: ' + values);
});
client.zrange( "contestants", membersPerTeam * 2, contestantCount,
function(err, values) {
console.log('Elder team: ' + values);
client.end();
});
}
});</pre></div></div><p>The output is:</p><pre class="screen">Young team: Courtney,Jessica,Patrick,Ramon,Vinny
Middle team: Jennifer,John,Sunny,Joe,Antone
Elder team: Becky,Deborah,Mike,Bonnie</pre><p>Adding members to a sorted set follows a pattern similar to the
one for adding members to a normal set, with the addition of a rank.
This allows for interesting slicing and dicing, as in this example.
Knowing that each team consists of similarly aged individuals, getting
three teams from a sorted list is a matter of pulling three equal
groups straight out of the set. The number of contestants (14) is not
perfectly divisible by 3, so the final group has only 4
members.</p></div><div class="sect3" title="Subscriptions"><div class="titlepage"><div><div><h3 class="title"><a id="id840966"/>Subscriptions</h3></div></div></div><p>Redis supports the <a id="I_indexterm3_d1e10359" class="indexterm"/><a id="I_indexterm3_d1e10364" class="indexterm"/><a id="I_indexterm3_d1e10369" class="indexterm"/><a id="I_indexterm3_d1e10374" class="indexterm"/>publish-subscribe (or pub-sub) messaging pattern,
allowing senders (publishers) to issue messages into channels for use
by receivers (subscribers) whom they know nothing about (see <a class="xref" href="ch06.html#example6-17" title="Example 6-17. Subscribing and publishing with Redis">Example 6-17</a>). Subscribers register their areas of
interests (channels), and Redis pushes all relevant messages to them.
Publishers do not need to be registered to specific channels, nor do
subscribers need to be listening when messages are sent. Redis takes
care of the brokering, which allows for a great deal of flexibility,
as neither the publisher nor the subscriber needs to be aware of the
other.</p><div class="example"><a id="example6-17"/><p class="title">Example 6-17. Subscribing and publishing with Redis</p><div class="example-contents"><pre class="programlisting">var redis = require("redis"),
talkativeClient = redis.createClient(),
pensiveClient = redis.createClient();
pensiveClient.on("subscribe", function (channel, count) {
talkativeClient.publish( channel, "Welcome to " + channel );
talkativeClient.publish( channel, "You subscribed to " + count + " channels!" );
});
pensiveClient.on("unsubscribe", function(channel, count) {
if (count === 0) {
talkativeClient.end();
pensiveClient.end();
}
});
pensiveClient.on("message", function (channel, message) {
console.log(channel + ': ' + message);
});
pensiveClient.on("ready", function() {
pensiveClient.subscribe("quiet channel", "peaceful channel", "noisy channel" );
setTimeout(function() {
pensiveClient.unsubscribe("quiet channel", "peaceful channel", "noisy channel" );
}, 1000);
});</pre></div></div><p>The output is:</p><pre class="screen">quiet channel: Welcome to quiet channel
quiet channel: You subscribed to 1 channels!
peaceful channel: Welcome to peaceful channel
peaceful channel: You subscribed to 2 channels!
noisy channel: Welcome to noisy channel
noisy channel: You subscribed to 3 channels!</pre><p>This example tells the story of two clients. One is quiet and
thoughtful, while the other broadcasts inane details about its
surroundings to anyone who will listen. The pensive client subscribes
to three channels: quiet, peaceful, and noisy. The talkative client
responds to each subscription by welcoming the newcomer to the channel
and counting the number of active subscriptions.</p><p>About one second after subscribing, the pensive client
unsubscribes from all three channels. When the unsubscribe command
detects no more active subscriptions, both clients end their
connection to Redis, and the program execution stops.</p></div><div class="sect3" title="Securing Redis"><div class="titlepage"><div><div><h3 class="title"><a id="id841056"/>Securing Redis</h3></div></div></div><p>Redis supports <a id="I_indexterm3_d1e10401" class="indexterm"/><a id="I_indexterm3_d1e10404" class="indexterm"/><a id="I_indexterm3_d1e10407" class="indexterm"/>password authentication. To add a password, edit Redis’s
configuration file and include a line for <code class="literal">requirepass</code>, as shown in <a class="xref" href="ch06.html#example6-18" title="Example 6-18. Snippet from Redis password configuration">Example 6-18</a>.</p><div class="example"><a id="example6-18"/><p class="title">Example 6-18. Snippet from Redis password configuration</p><div class="example-contents"><pre class="programlisting">################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
# commands. This might be useful in environments in which you do not trust
# others with access to the host running redis-server.
#
# This should stay commented out for backward compatibility and because most
# people do not need auth (e.g., they run their own servers).
#
requirepass hidengoseke</pre></div></div><p>Once Redis is restarted, it will perform commands only for
clients who authenticate using “hidengoseke” as their password (<a class="xref" href="ch06.html#example6-19" title="Example 6-19. Authenticating Redis">Example 6-19</a>).</p><div class="example"><a id="example6-19"/><p class="title">Example 6-19. Authenticating Redis</p><div class="example-contents"><pre class="programlisting">var redis = require('redis'),
client = redis.createClient();
client.auth("hidengoseke");</pre></div></div><p>The <code class="function">auth</code> command <a id="I_indexterm3_d1e10437" class="indexterm"/><a id="I_indexterm3_d1e10442" class="indexterm"/>must occur before any other queries are issued. The
client will store the password and use it on reconnection
attempts.</p><p>Notice the lack of usernames and multiple passwords. Redis does
not include user management functionality, because of the overhead it
would incur. Instead, system administrators are expected to secure
their servers using other means, such as port-blocking Redis from the
outside world so that only internal, trusted users may access
it.</p><p>Some “dangerous” commands can be renamed or removed entirely.
For example, you may never need to use the<a id="I_indexterm3_d1e10450" class="indexterm"/> <code class="literal">CONFIG</code> command. In that case, you
can update the configuration file to either change its name to
something obscure, or you can fully disable it to protect against
unwanted access; both options are <a id="I_indexterm3_d1e10457" class="indexterm"/><a id="I_indexterm3_d1e10459" class="indexterm"/><a id="I_indexterm3_d1e10461" class="indexterm"/>shown in <a class="xref" href="ch06.html#example6-20" title="Example 6-20. Renaming Redis commands">Example 6-20</a>.</p><div class="example"><a id="example6-20"/><p class="title">Example 6-20. Renaming Redis commands</p><div class="example-contents"><pre class="programlisting"># Change CONFIG command to something obscure
rename-command CONFIG 923jfiosflkja98rufadskjgfwefu89awtsga09nbhsdalkjf3p49
# Clear CONFIG command, so no one can use it
rename-command CONFIG ""</pre></div></div></div></div><div class="sect2" title="MongoDB"><div class="titlepage"><div><div><h2 class="title"><a id="id841231"/>MongoDB</h2></div></div></div><p>Because Mongo <a id="da6.1.3" class="indexterm"/><a id="no6.1.3" class="indexterm"/><a id="I_indexterm3_d1e10486" class="indexterm"/>supplies a JavaScript environment with <a id="I_indexterm3_d1e10492" class="indexterm"/>BSON object storage (a binary adaption of JSON), reading
and writing data from Node is extremely efficient. Mongo stores incoming
records in memory, so it is ideal in high-write situations. Each new
version adds improved clustering, replication, and sharding.</p><p>Because incoming records are stored in memory, inserting data into
Mongo is nonblocking, making it ideal for logging operations and
telemetry data. Mongo supports JavaScript functions inside queries,
making it very powerful in read situations, including MapReduce
queries.</p><p>Using MongoDB’s document-based storage allows you to store child
records inside parent records. For example, a blog article and all of
its associated comments can be stored inside a single record, allowing
for incredibly fast retrieval.</p><div class="sect3" title="MongoDB native driver"><div class="titlepage"><div><div><h3 class="title"><a id="id841292"/>MongoDB native driver</h3></div></div></div><p>The <a class="ulink" href="https://github.com/christkv/node-mongodb-native">native MongoDB
driver</a> by <a id="I_indexterm3_d1e10508" class="indexterm"/><a id="I_indexterm3_d1e10511" class="indexterm"/>Christian Kvaleim provides nonblocking access to
MongoDB. Previous versions of the module included a C/C++ BSON
parser/serializer, which has been deprecated due to improvements in
the JavaScript parser/serializer.</p><p>The native MongoDB driver is a good choice when you need precise
control over your MongoDB connection.</p><div class="sect4" title="Installation"><div class="titlepage"><div><div><h4 class="title"><a id="id841325"/>Installation</h4></div></div></div><p>To install the <a id="I_indexterm3_d1e10524" class="indexterm"/>driver, run the following command:</p><a id="I_programlisting3_d1e10530"/><pre class="programlisting">npm install mongodb</pre><div class="warning" title="Warning"><h3 class="title">Warning</h3><p>“mongodb” is not to be confused with “mongo,” discussed
later in this chapter.</p></div></div><div class="sect4" title="Data types"><div class="titlepage"><div><div><h4 class="title"><a id="id841351"/>Data types</h4></div></div></div><p>Node’s MongoDB driver supports the data types listed in <a class="xref" href="ch06.html#table6-1" title="Table 6-1. Data types supported for MongoDB">Table 6-1</a>.</p><div class="table"><a id="table6-1"/><p class="title">Table 6-1. Data types supported for MongoDB</p><div class="table-contents"><table summary="Data types supported for MongoDB" style="border-collapse: collapse;border-top: 0.5pt solid ; border-bottom: 0.5pt solid ; border-left: 0.5pt solid ; border-right: 0.5pt solid ; "><colgroup><col/><col/><col/></colgroup><thead><tr><th style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">Type</th><th style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">Description</th><th style="border-bottom: 0.5pt solid ; ">Example</th></tr></thead><tbody><tr><td style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">Array</td><td style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">A list of items</td><td style="border-bottom: 0.5pt solid ; "><code class="literal">cardsInHand: [9,4,3]</code></td></tr><tr><td style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">Boolean</td><td style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">A true/false condition</td><td style="border-bottom: 0.5pt solid ; "><code class="literal">hasBeenRead: false</code></td></tr><tr><td style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">Code</td><td style="border-right: 0.5pt solid ; border-bottom: 0.5pt solid ; ">Represents a block of JavaScript code that is
runnable inside the database</td><td style="border-bottom: 0.5pt solid ; "><code class="li