elsewhere
Version:
A node project that aims to replicate the functionality of the Google Social Graph API
395 lines (248 loc) • 13.6 kB
HTML
<!--
_____ __ ___ __
| \| |~-.~-~.-.-~-~.-~-~-~-~.-~-.-.' _| |.-~.~-.
| | | | _ | _| | _ | _| || | |
|_____/|__|__|___._|__| |__|__|__|___._|__| |__||___ |
|_____|
dharmafly.com
-->
<!--[if IE 9]><html class="ie" lang="en"><![endif]-->
<!--[if lt IE 9 ]> <html class="ie ltIE9" lang="en"> <![endif]-->
<!--[if !IE ]><!-->
<html lang="en"> <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="">
<title>Reference : Elsewhere</title>
<script>
var satya = satya || {};
satya.isIPad = navigator.userAgent.match(/iPad/i) != null;
if(document.getElementsByName && !satya.isIPad){
document.getElementsByName('viewport')[0]
.setAttribute('content','width=device-width,initial-scale=0.75,user-scalable=no,maximum-scale=0.75');
}
</script>
<!--[if lt IE 9]><script src="/javascript/html5shiv.js"></script><![endif]-->
<script type="text/javascript">
var WebFontConfig = {
custom: {
families: [
'Open+Sans:400,600:latin',
'Zeyada::latin',
'Ubuntu+Mono::latin'
],
urls: [
'/css/fonts/seagrass.css'
]
}
};
(function() {
var wf = document.createElement('script');
wf.src = ('https:' == document.location.protocol ? 'https' : 'http') +
'://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js';
wf.type = 'text/javascript';
wf.async = 'true';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(wf, s);
})();
</script>
<link href="/css/seagrass.css?v3" rel="stylesheet">
<link rel="shortcut icon" sizes="16x16" type="image/vnd.microsoft.icon" href="/img/seagrass-favicon.ico">
<link rel="apple-touch-icon" href="/img/seagrass-apple-touch-icon.png">
</head>
<body class="subpage loading">
<header>
<h1 class="title">
<a href="/">
Elsewhere
</a>
</h1>
</header>
<nav id="navigation">
<ul>
<li class="show-subnav"><a href="#subnav" class="icon">Go to navigation for this page</a></li>
<li>
<a href="/">
Overview
</a>
</li>
<li>
<a href="/reference/" class="active">
Reference
</a>
</li>
</ul>
</nav>
<section class="content">
<nav id="subnav">
<ul>
<li><a href="#Requiring-Elsewhere">
Requiring Elsewhere
</a></li>
<li><a href="#graph">
graph()
</a></li>
<li><a href="#Global-configuration">
Global configuration
</a></li>
<li><a href="#Custom-cache">
Custom cache
</a></li>
<li><a href="#Custom-logger">
Custom logger
</a></li>
</ul>
</nav>
<section>
<h1><a id="Requiring-Elsewhere" class="permalink" href="#Requiring-Elsewhere">⚓</a>
Requiring Elsewhere
</h1>
<article class="embedded markdown">
<p>Elsewhere is available on <a href='https://npmjs.org/package/elsewhere'>NPM</a>. To install it, run:</p>
<pre><code>npm install elsewhere</code></pre>
<p>Once you have it installed, you can <a href='http://nodejs.org/api/globals.html#globals_require'>require()</a> it and interact with it using the <code>graph()</code> method.</p>
<pre><code>var elsewhere = require('elsewhere');
elsewhere.graph('http://premasagar.com').then(function (err, graph) {
res.end(graph);
});</code></pre>
</article>
</section>
<aside class="icons">
<ul>
<li class="github"><a href="https://github.com/dharmafly/elsewhere" title="Elsewhere on Github">Elsewhere on Github</a></li>
<li class="twitter"><a href="https://twitter.com/dharmafly" title="Dharmafly on Twitter">
Dharmafly on Twitter</a></li>
<li class="code-javascript" title="This is a JavaScript project"><span>This is a JavaScript project</span></li>
</ul>
</aside>
<section>
<h1><a id="graph" class="permalink" href="#graph">⚓</a>
graph()
</h1>
<article class="embedded markdown">
<p>The <code>elsewhere.graph()</code> method accepts the following parameters:</p>
<p><strong>url</strong> This is the URL that you wish to graph. The assumption is that it represents a person (or a company or organisation) and that the page at that URL has <code>rel=me</code> links to other URLs that also represent the person.</p>
<p><strong>options</strong> (optional) This object contains properties used to configure the graph. The properties that can be passed in are: <code>strict</code>, <code>logLevel</code>, <code>crawlLimit</code>, <code>domainLimit</code>, <code>stripDeeperLinks</code>, <code>useCache</code>, <code>cacheTimeLimit</code> and <code>cacheItemLimit</code>. These are the same options that can be set in the global configuration (see below).</p>
<p><strong>callback</strong> (optional) This is a function to be called once the graph is ready. The callback is passed an error (string or null) and the completed graph as a object literal.</p>
<p>The method returns a <a href='http://wiki.commonjs.org/wiki/Promises'>promise</a> object (created by the <a href='https://npmjs.org/package/underscore.deferred'>Underscore.deferred</a> module). This is an alternative to using the <code>callback</code> parameter and provides fine-grained flow control.</p>
<p>Some examples of valid calls to the <code>graph()</code> method:</p>
<pre><code>var options = {strict:true},
callback = function (err, graph) {
if (err) {
console.log(graph);
} else {
console.log(JSON.stringify(graph));
}
};
elsewhere.graph('http://premasagar.com', options, callback);
elsewhere.graph('http://chrisnewtn.com', callback);
elsewhere.graph('http://glennjones.net').then(callback);</code></pre>
<p>If an error happens trying to parse the starting point URL, the returning ‘err’ object will contain a string message of the error. The ‘graph’ object will still be returned. Any after during the parsing of additional URLs will be added to the warnings collection, part of the graph object.</p>
</article>
</section>
<section>
<h1><a id="Global-configuration" class="permalink" href="#Global-configuration">⚓</a>
Global configuration
</h1>
<article class="embedded markdown">
<p>Instead of using a local options object each time you call the <code>graph()</code> method, you can also set global options by setting properties in the <code>elsewhere.options</code> object.</p>
<p>Global options act as default values, which can then be overriden by options passed when calling the <code>graph()</code> method.</p>
<p><strong>strict</strong> (boolean) Whether the crawler allows only reciprocal <code>rel="me"</code> links or not. A reciprocal link is where the page at a URL links to another page in the graph, and a page in the graph links back to the original URL. When <code>true</code>, there will be no false positives, but fewer results. Default: <code>false</code></p>
<p><strong>logLevel</strong> (integer) There are 4 levels of logging in Elsewhere: 4 - log, 3 - info, 2 - warn and 1 - error. The 4 setting gives the most granular logs, which are useful in a debugging scenario. Default: <code>3</code></p>
<p><strong>crawlLimit</strong> (integer) The number of links that Elsewhere will follow without a successful verification before it abandons the chain. Default: <code>3</code></p>
<p><strong>domainLimit</strong> (integer) The number of links crawled within a particular domain before the crawling of subsequent links in the domain is abandoned. Default: <code>3</code></p>
<p><strong>stripDeeperLinks</strong> (boolean) If set to <code>true</code> then Elsewhere will remove links from the graph if they are at a deeper path than other links in the same domain. For example, <code>plus.google.com/{id}</code> is retained, but <code>plus.google.com/{id}/posts</code> is discarded. This is useful, for example, to strip out paginated contacts pages on social networks. Default: <code>true</code></p>
<p><strong>useCache</strong> Whether a request should use the cache during a request. Default: <code>true</code></p>
<p><strong>cacheTimeLimit</strong> The amount of time, in milliseconds, that graphs and pages are kept in the cache before they are discarded. Default: <code>3600000</code></p>
<p><strong>cacheItemLimit</strong> (integer) The maximum number of items that can be kept in the cache before the oldest items are discarded. Use to limit memory. Default: <code>1000</code></p>
<p><strong>httpHeaders</strong> (object) An object the HTTP header properties use when requesting resources from the internet.</p>
<p>For example:</p>
<pre><code>elsewhere.options.strict = false;</code></pre>
<p>If you are running Elsewhere as a server, then you may set the options directly in <code>lib/options.js</code>.</p>
</article>
</section>
<section>
<h1><a id="Custom-cache" class="permalink" href="#Custom-cache">⚓</a>
Custom cache
</h1>
<article class="embedded markdown">
<p>Elsewhere use an in-memory cache to store the HTML of web pages.</p>
<p>The options object contains a property called <code>cacheTimeLimit</code> that can be used to set the cache refresh time. By default, this is 3600000ms (1 hour). The number of items stored in the cache can be limited using the options property <code>cacheItemLimit</code>. By default, the cache is limited to 1000 items.</p>
<p>You can replace the cache with your own, for example, to store the cached date in a database or file system. To add you own custom cache, all you need to do is provide an object with the following interface:</p>
<pre><code>{
get: function (url) {
// add code to get data
return data
},
has: function (url) {
// add code to check your data store
return true or false
},
fetch: function (url, callback) {
// add code to return data
callback(null, data);
},
set: function (url, data) {
// add code to store data
return object
}
}</code></pre>
<p>You must then add this object as the <code>cache</code> property of the options object passed into the <code>graph()</code> method.</p>
</article>
</section>
<section>
<h1><a id="Custom-logger" class="permalink" href="#Custom-logger">⚓</a>
Custom logger
</h1>
<article class="embedded markdown">
<p>Elsewhere uses a simple logging system that writes to Node’s console. You can replace the logger with your own, for example, to store warnings and errors in a database or log file. To add your own custom logger, all you need to do is provide an object with the following interface:</p>
<pre><code>{
info: function (message) { /* code to pass on message */ },
log: function (message) { /* code to pass on message */ },
warn: function (message) { /* code to pass on message */ },
error: function (message) { /* code to pass on message */ }
}</code></pre>
<p>You must then add this object to the <code>logger</code> property of the options object passed into the <code>graph()</code> method.</p>
</article>
</section>
</section>
<footer>
<div>
<p>
<em>by</em> <a href="http://dharmafly.com">Dharmafly</a>
</p>
</div>
</footer>
<script>
satya.narrowScreen = window.screen.width < 480;
satya.isltIE10 = false;
satya.relative_path = '../';
satya.noConflict = 'true' === 'true' || false;
</script>
<!--[if lt IE 10]>
<script>
satya.isltIE10 = true;
</script>
<![endif]-->
<!--[if IE 9]>
<script src="/javascript/IEshims.js"></script>
<![endif]-->
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.8.1/jquery.min.js"></script>
<script>window.jQuery || document.write('<script src="/javascript/jquery-1.8.1.min.js"><\/script>')</script>
<script src="/javascript/main.js"></script>
<script src="/javascript/hijs.js"></script>
<script src="/javascript/demo.js?v2"></script>
<script src="https://github.com/downloads/dharmafly/pablo/pablo.min.js"></script>
<script>
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-34978047-3']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
}());
</script>
</body>
</html>