nyaatorrents
Version:
Interact with NyaaTorrents (nyaa.eu, formerly nyaatorrents.info)
467 lines (322 loc) • 20.7 kB
HTML
<html>
<head>
<title>index.js</title>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width, target-densitydpi=160dpi, initial-scale=1.0; maximum-scale=1.0; user-scalable=0;">
<link rel="stylesheet" media="all" href="docco.css" />
</head>
<body>
<div id="container">
<div id="background"></div>
<ul class="sections">
<li id="title">
<div class="annotation">
<h1>index.js</h1>
</div>
</li>
<li id="section-1">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-1">¶</a>
</div>
<p>This is where all the fun happens. Please take a moment to read through this
file to acquaint yourself with the functionality provided and the
implementation thereof.</p>
</div>
<div class="content"><div class='highlight'><pre><span class="keyword">var</span> cheerio = require(<span class="string">"cheerio"</span>),
ent = require(<span class="string">"ent"</span>),
filesize_parser = require(<span class="string">"filesize-parser"</span>),
request = require(<span class="string">"request"</span>),
url = require(<span class="string">"url"</span>);</pre></div></div>
</li>
<li id="section-2">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-2">¶</a>
</div>
<p>Main entry point. This is the client class. It takes a single optional
argument, being the base URL of the NyaaTorrents site you want to interact
with. If left out, it will default to "<a href="http://www.nyaa.eu/">http://www.nyaa.eu/</a>".</p>
</div>
<div class="content"><div class='highlight'><pre><span class="keyword">var</span> NyaaTorrents = module.exports = <span class="function"><span class="keyword">function</span> <span class="title">NyaaTorrents</span><span class="params">(base_url)</span> {</span>
<span class="keyword">if</span> (<span class="keyword">typeof</span> base_url === <span class="string">"undefined"</span>) {
base_url = <span class="string">"http://www.nyaa.eu/"</span>;
}
<span class="keyword">this</span>.base_url = base_url;
};</pre></div></div>
</li>
<li id="section-3">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-3">¶</a>
</div>
<p>Search method. This maps pretty transparently to <a href="http://www.nyaa.eu/?page=search">the search page</a>,
passing through the <code>query</code> hash verbatim as url parameters. If the <code>query</code>
argument is left out, you'll get a list of the latest torrents as you will
have provided no filter arguments. The second argument is a callback that
will be called on completion with <code>err</code> and <code>results</code> arguments. <code>err</code> will
be null in the case of success.</p>
</div>
<div class="content"><div class='highlight'><pre>NyaaTorrents.prototype.search = <span class="function"><span class="keyword">function</span> <span class="title">search</span><span class="params">(query, cb)</span> {</span>
<span class="keyword">var</span> uri = url.parse(<span class="keyword">this</span>.base_url);
<span class="keyword">if</span> (<span class="keyword">typeof</span> query === <span class="string">"function"</span>) {
cb = query;
query = <span class="literal">null</span>;
}
<span class="keyword">if</span> (<span class="keyword">typeof</span> query !== <span class="string">"object"</span> || query === <span class="literal">null</span>) {
query = {};
}
query.page = <span class="string">"torrents"</span>;
uri.query = query;
request(url.format(uri), <span class="keyword">function</span>(err, res, data) {
<span class="keyword">if</span> (err) {
<span class="keyword">return</span> cb(err);
}
<span class="keyword">var</span> $ = cheerio.load(data);</pre></div></div>
</li>
<li id="section-4">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-4">¶</a>
</div>
<p>Our results are found in a table with a predictable structure. Some of
this code might be fragile - expect updates here to improve performance
or stability. PATCHES WELCOME LOL.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">var</span> torrents = Array.prototype.slice.apply($(<span class="string">"table.tlist .tlistrow"</span>)).map(<span class="keyword">function</span>(row) {
<span class="keyword">var</span> obj = {};</pre></div></div>
</li>
<li id="section-5">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-5">¶</a>
</div>
<p>If we can't find the download link or the category image, we just give
up on this row. It shouldn't happen, but it might indicate bad markup
or unhandled stuff.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">var</span> download_link = $(row).find(<span class="string">".tlistdownload a"</span>)[<span class="number">0</span>];
<span class="keyword">if</span> (!download_link) {
<span class="keyword">return</span> <span class="literal">null</span>;
}
<span class="keyword">var</span> category_image = $(row).find(<span class="string">".tlisticon a"</span>)[<span class="number">0</span>];
<span class="keyword">if</span> (!category_image) {
<span class="keyword">return</span> <span class="literal">null</span>;
}
obj.id = parseInt(download_link.attribs.href.trim().replace(<span class="regexp">/^.+?(\d+)$/</span>, <span class="string">"$1"</span>), <span class="number">10</span>);
obj.name = $(row).find(<span class="string">".tlistname"</span>).text().trim();
obj.categories = ent.decode(category_image.attribs.title).trim().split(<span class="regexp">/ >> /g</span>).map(<span class="keyword">function</span>(e) { <span class="keyword">return</span> e.toLowerCase().trim().replace(<span class="regexp">/\s+/g</span>, <span class="string">"-"</span>); });
obj.flags = row.attribs.class.split(<span class="regexp">/ /g</span>).filter(<span class="keyword">function</span>(e) { <span class="keyword">return</span> e !== <span class="string">"tlistrow"</span>; });
obj.size = filesize_parser($(row).find(<span class="string">".tlistsize"</span>).text().trim());
obj.seeds = parseInt($(row).find(<span class="string">".tlistsn"</span>).text().trim(), <span class="number">10</span>);
obj.leeches = parseInt($(row).find(<span class="string">".tlistln"</span>).text().trim(), <span class="number">10</span>);
obj.downloads = parseInt($(row).find(<span class="string">".tlistdn"</span>).text().trim(), <span class="number">10</span>);
obj.comments = parseInt($(row).find(<span class="string">".tlistmn"</span>).text().trim(), <span class="number">10</span>);
<span class="keyword">return</span> obj;
}).filter(<span class="keyword">function</span>(e) {
<span class="keyword">return</span> e !== <span class="literal">null</span>;
});
<span class="keyword">return</span> cb(<span class="literal">null</span>, torrents);
});
};</pre></div></div>
</li>
<li id="section-6">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-6">¶</a>
</div>
<p>This method gets the information about a specific torrent, identified by ID.
First argument is a number or a string containing the torrent ID, second is
a callback to be called on completion with <code>err</code> and <code>result</code> arguments. As
with the previous method, <code>err</code> will be null in the case of success.</p>
</div>
<div class="content"><div class='highlight'><pre>NyaaTorrents.prototype.get = <span class="function"><span class="keyword">function</span> <span class="title">get</span><span class="params">(id, cb)</span> {</span>
<span class="keyword">var</span> uri = url.parse(<span class="keyword">this</span>.base_url);
uri.query = {
page: <span class="string">"torrentinfo"</span>,
tid: id,
};
request(url.format(uri), <span class="keyword">function</span>(err, res, data) {
<span class="keyword">if</span> (err) {
<span class="keyword">return</span> cb(err);
}
<span class="keyword">var</span> $ = cheerio.load(data);
<span class="keyword">var</span> content = $(<span class="string">".content"</span>)[<span class="number">0</span>];</pre></div></div>
</li>
<li id="section-7">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-7">¶</a>
</div>
<p>When there's an error, it's displayed as text in the spot where the page
content would usually go. We pass that through as-is to the user.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">if</span> (content.children.length === <span class="number">1</span> && content.children[<span class="number">0</span>].type === <span class="string">"text"</span>) {
<span class="keyword">return</span> cb(Error(ent.decode(content.children[<span class="number">0</span>].data).trim()));
}
<span class="keyword">var</span> obj = {};</pre></div></div>
</li>
<li id="section-8">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-8">¶</a>
</div>
<p>These are things like "trusted" or "remake". See <a href="http://forums.nyaa.eu/index.php?/topic/1284-nyaatorrents-manual/">the NyaaTorrents manual</a>
for more information.</p>
</div>
<div class="content"><div class='highlight'><pre> obj.flags = content.attribs.class.split(<span class="regexp">/ /g</span>).filter(<span class="keyword">function</span>(e) { <span class="keyword">return</span> e !== <span class="string">"content"</span>; });</pre></div></div>
</li>
<li id="section-9">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-9">¶</a>
</div>
<p>Categories. Super simple stuff. These are lower-cased, hyphen-delimited,
human-readable strings.</p>
</div>
<div class="content"><div class='highlight'><pre> obj.categories = Array.prototype.slice.apply($(content).find(<span class="string">"td.viewinfocategory a"</span>)).map(<span class="keyword">function</span>(e) {
<span class="keyword">return</span> $(e).text().toLowerCase().trim().replace(<span class="regexp">/\s+/g</span>, <span class="string">"-"</span>);
});</pre></div></div>
</li>
<li id="section-10">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-10">¶</a>
</div>
<p>The torrent details are displayed in a wonky table thing. Each field is
displayed as a pair of cells, with the former containing the field name
and the latter displaying the field value. Based on the name, we do some
field-specific transformations on some fields. Others just get passed on
through as text.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">var</span> tds = $(content).find(<span class="string">"table.viewtable tr > td"</span>);
<span class="keyword">for</span> (<span class="keyword">var</span> i=<span class="number">0</span>;i<tds.length;i+=<span class="number">2</span>) {</pre></div></div>
</li>
<li id="section-11">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-11">¶</a>
</div>
<p>This is the field name.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">var</span> k = $(tds[i]).text().replace(<span class="regexp">/:$/</span>, <span class="string">""</span>).replace(<span class="regexp">/\s+/g</span>, <span class="string">"_"</span>).trim().toLowerCase();
<span class="keyword">switch</span> (k) {</pre></div></div>
</li>
<li id="section-12">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-12">¶</a>
</div>
<p>"information" is basically synonymous with "website"</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">case</span> <span class="string">"information"</span>:
<span class="keyword">var</span> link = $(tds[i+<span class="number">1</span>]).find(<span class="string">"a"</span>);
<span class="keyword">if</span> (link.length)
obj.website = link[<span class="number">0</span>].attribs.href;
<span class="keyword">break</span>;</pre></div></div>
</li>
<li id="section-13">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-13">¶</a>
</div>
<p>"file_size" is exactly what it sounds like, and it has to be turned
into a real number.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">case</span> <span class="string">"file_size"</span>:
obj.size = filesize_parser($(tds[i+<span class="number">1</span>]).text().trim());
<span class="keyword">break</span>;</pre></div></div>
</li>
<li id="section-14">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-14">¶</a>
</div>
<p>This is the user that submitted the torrent. We parse it out into the
separate <code>id</code> and <code>name</code> values.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">case</span> <span class="string">"submitter"</span>:
obj.user = {
id: parseInt($(tds[i+<span class="number">1</span>]).find(<span class="string">"a"</span>)[<span class="number">0</span>].attribs.href.replace(<span class="regexp">/^.+?(\d+)$/</span>, <span class="string">"$1"</span>), <span class="number">10</span>),
name: $(tds[i+<span class="number">1</span>]).text(),
};
<span class="keyword">break</span>;</pre></div></div>
</li>
<li id="section-15">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-15">¶</a>
</div>
<p>This might not work on anything except V8. Will have to check that
if this ever works on anything except node.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">case</span> <span class="string">"date"</span>:
obj.date = <span class="keyword">new</span> Date($(tds[i+<span class="number">1</span>]).text());
<span class="keyword">break</span>;</pre></div></div>
</li>
<li id="section-16">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-16">¶</a>
</div>
<p>The "stardom" field just displays the number of people who've set
themselves as "fans" of this torrent. I don't really know what the
deal is here.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">case</span> <span class="string">"stardom"</span>:
obj.fans = parseInt($(tds[i+<span class="number">1</span>]).text().replace(<span class="regexp">/^.+(\d+).+$/</span>, <span class="string">"$1"</span>), <span class="number">10</span>);
<span class="keyword">break</span>;</pre></div></div>
</li>
<li id="section-17">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-17">¶</a>
</div>
<p>All these need to be turned to real numbers instead of strings.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">case</span> <span class="string">"seeders"</span>:
<span class="keyword">case</span> <span class="string">"leechers"</span>:
<span class="keyword">case</span> <span class="string">"downloads"</span>:
obj[k] = parseInt($(tds[i+<span class="number">1</span>]).text(), <span class="number">10</span>);
<span class="keyword">break</span>;</pre></div></div>
</li>
<li id="section-18">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-18">¶</a>
</div>
<p>Anything not otherwise handled is just sent through as text.</p>
</div>
<div class="content"><div class='highlight'><pre> <span class="keyword">default</span>:
obj[k] = $(tds[i+<span class="number">1</span>]).text();
}
}</pre></div></div>
</li>
<li id="section-19">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-19">¶</a>
</div>
<p>This is the torrent ID... You already have it, but this seemed like a
good idea for completeness.</p>
</div>
<div class="content"><div class='highlight'><pre> obj.id = parseInt($(content).find(<span class="string">"div.viewdownloadbutton a"</span>)[<span class="number">0</span>].attribs.href.replace(<span class="regexp">/^.+?(\d+)$/</span>, <span class="string">"$1"</span>), <span class="number">10</span>);</pre></div></div>
</li>
<li id="section-20">
<div class="annotation">
<div class="pilwrap ">
<a class="pilcrow" href="#section-20">¶</a>
</div>
<p>This is a chunk of HTML. You'll probably want to turn it into some other
kind of markup.</p>
</div>
<div class="content"><div class='highlight'><pre> obj.description = $($(content).find(<span class="string">"div.viewdescription"</span>)[<span class="number">0</span>]).html();
<span class="keyword">return</span> cb(<span class="literal">null</span>, obj);
});
};</pre></div></div>
</li>
</ul>
</div>
</body>
</html>