qminer
Version:
A C++ based data analytics platform for processing large-scale real-time streams containing structured and unstructured data
366 lines (355 loc) • 16.2 kB
HTML
<html>
<head>
<meta name="generator" content="JSDoc 3">
<meta charset="utf-8">
<title>Class: TimeWindowGk</title>
<link rel="stylesheet" href="https://brick.a.ssl.fastly.net/Karla:400,400i,700,700i" type="text/css">
<link rel="stylesheet" href="https://brick.a.ssl.fastly.net/Noto+Serif:400,400i,700,700i" type="text/css">
<link rel="stylesheet" href="https://brick.a.ssl.fastly.net/Inconsolata:500" type="text/css">
<link href="css/baseline.css" rel="stylesheet">
</head>
<body onload="prettyPrint()">
<nav id="jsdoc-navbar" role="navigation" class="jsdoc-navbar">
<div id="jsdoc-navbar-container">
<div id="jsdoc-navbar-content">
<a href="index.html" class="jsdoc-navbar-package-name">QMiner JavaScript API v9.1.2</a>
</div>
</div>
</nav>
<div id="jsdoc-body-container">
<div id="jsdoc-content">
<div id="jsdoc-content-container">
<div id="jsdoc-main" role="main">
<header class="page-header">
<h1><div class="symbol-detail-labels"><span class="label label-kind">class</span> <span class="label label-static">static</span></div><small><a href="module-analytics.html">analytics</a>.<wbr></small><span class="symbol-name">TimeWindowGk</span></h1>
<p class="source-link">Source: <a href="analyticsdoc.js.html#source-line-2778">analyticsdoc.<wbr>js:2778</a></p>
<div class="symbol-classdesc">
<p>Greenwald - Khanna algorithm for quantile estimation on sliding windows. Given a cumulative probability p, the algorithm returns the approximate value of the p-th quantile of all the values in a sliding window.</p>
<p> The algorithm works by keeping a summary of buckets. Each bucket summarizes a range of values. Through the run of the algorithm, new buckets are created and old ones merged. To allow for the computation on a sliding window, each bucket uses
an Exponential Histogram structure to remember how many values it summarizes.</p>
<p> It is summarized in: "Online Algorithm for Approximate Quantile Queries on Sliding Windows" http://dl.acm.org/citation.cfm?id=2954329
</p>
<p> The error is not bounded by the absolute value of the output, but by the error in rank of the output element. For instance, if we have 100 elements and query the median, we could get the 48-th (p=0.48), 50-th (p=0.5), 51-th (p=0.51), etc.
element.</p>
<p> The algorithms error is bounded by two factors. The first is the quantile estimation factor <code>quantileEps</code> occurs because of the summary structure and defines the maximum size of the buckets. The second type of error <code>countEps</code> occurs because of the Exponential Histograms inside the buckets.</p>
<p> The worst-case error is bounded by (quantileEps + 2*countEps + O(countEps^2)), although in practice the error is lower.</p>
<p> This version of the algorithm uses a time-based fixed duration sliding window.</p>
</div>
<dl class="dl-compact">
</dl>
</header>
<section id="summary">
<div class="summary-callout">
<h2 class="summary-callout-heading">Methods</h2>
<div class="summary-content">
<div class="summary-column">
<dl class="dl-summary-callout">
<dt><a href="module-analytics.TimeWindowGk.html#.partialFit">partialFit(timestamp[, value])</a></dt>
<dd>
</dd>
<dt><a href="module-analytics.TimeWindowGk.html#.save">save(fout)</a></dt>
<dd>
</dd>
</dl>
</div>
<div class="summary-column">
<dl class="dl-summary-callout">
<dt><a href="module-analytics.TimeWindowGk.html#getParams">getParams()</a></dt>
<dd>
</dd>
<dt><a href="module-analytics.TimeWindowGk.html#predict">predict(p)</a></dt>
<dd>
</dd>
</dl>
</div>
<div class="summary-column">
</div>
</div>
</div>
</section>
<section>
<h2 id="TimeWindowGk">new <span class="symbol-name">TimeWindowGk</span><span class="signature"><span class="signature-params">([arg])</span></span></h2>
<section>
<h3>
Example
</h3>
<div>
<pre class="prettyprint"><code>// import modules
var qm = require('qminer');
var fs = qm.fs;
var analytics = qm.analytics;
// create the default object
var gk = new analytics.TimeWindowGk({
window: 5,
quantileEps: 0.001,
countEps: 0.0005
});
// create the data used for calculating quantiles
var inputs = [10, 1, 2, 8, 9, 5, 6, 4, 7, 3];
var times = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
// fit the model
for (var i = 0; i < inputs.length; i++) {
gk.partialFit(times[i], inputs[i]);
}
// make the prediction for the 0.1 quantile
var prediction = gk.predict(0.1);
// save the model
gk.save(fs.openWrite('gk.bin')).close();
// open the gk model under a new variable
var gk2 = new analytics.TimeWindowGk(fs.openRead('gk.bin'));</code></pre>
</div>
</section>
<section>
<h3>Parameter</h3>
<table class="jsdoc-details-table">
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Optional</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<p>arg</p>
</td>
<td>
<p>(<a href="module-analytics.html#~TimeWindowGkParam">module:analytics~TimeWindowGkParam</a> or <a href="module-fs.FIn.html">module:fs.FIn</a>)</p>
</td>
<td>
<p>Yes</p>
</td>
<td>
<p>Construction arguments. There are two ways of constructing:
<br>1. Using the <a href="module-analytics.html#~TimeWindowGkParam">module:analytics~TimeWindowGkParam</a> object,
<br>2. using the file input stream <a href="module-fs.FIn.html">module:fs.FIn</a>.</p>
</td>
</tr>
</tbody>
</table>
</section>
<dl class="dl-compact">
</dl>
</section>
<section>
<h2>Methods</h2>
<section>
<h3 id=".partialFit"><div class="symbol-detail-labels"><span class="label label-static">static</span></div><span class="symbol-name">partialFit</span><span class="signature"><span class="signature-params">(timestamp[, value])</span> → <span class="signature-returns"> <a href="module-analytics.TimeWindowGk.html">module:analytics.TimeWindowGk</a></span></span></h3>
<p>Adds a new observation to the window. The window is updated with the provided timestamp, all records which fall outside the new window are forgotten.</p>
<section>
<h4>
Example
</h4>
<div>
<pre class="prettyprint"><code>var qm = require('qminer');
var gk = new qm.analytics.TimeWindowGk({
window: 1000*60*60*24*2 // window is 2 days
});
gk.partialFit(new Date('2017-06-06'), 1.0);
gk.partialFit(new Date('2017-06-07').getTime(), 1.0);
gk.partialFit(new Date('2017-06-08')); // only move the time window, the first value is forgotten</code></pre>
</div>
</section>
<section>
<h4>Parameters</h4>
<table class="jsdoc-details-table">
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Optional</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<p>timestamp</p>
</td>
<td>
<p>(number or Date)</p>
</td>
<td>
<p> </p>
</td>
<td>
<p>time of the observation</p>
</td>
</tr>
<tr>
<td>
<p>value</p>
</td>
<td>
<p>number</p>
</td>
<td>
<p>Yes</p>
</td>
<td>
<p>the observation</p>
</td>
</tr>
</tbody>
</table>
</section>
<dl class="dl-compact">
<dt>Returns</dt>
<dd>
<p><code><a href="module-analytics.TimeWindowGk.html">module:analytics.TimeWindowGk</a></code> reference to self</p>
</dd>
</dl>
<h3 id=".save"><div class="symbol-detail-labels"><span class="label label-static">static</span></div><span class="symbol-name">save</span><span class="signature"><span class="signature-params">(fout)</span> → <span class="signature-returns"> <a href="module-fs.FOut.html">module:fs.FOut</a></span></span></h3>
<p>Saves the objects state into a binary file.</p>
<section>
<h4>
Example
</h4>
<div>
<pre class="prettyprint"><code>var qm = require('qminer');
var fs = qm.fs;
var gk = new qm.analytics.TimeWindowGk({
window: 100 // window is 2 days
});
gk.partialFit(0, 1.0);
gk.partialFit(1, 1.0);
gk.partialFit(2, 1.0);
gk.partialFit(3, 1.0);
gk.partialFit(4, 1.0);
// save the model
gk.save(fs.openWrite('gk.bin')).close();
// open the model under a new variable
var gk = new analytics.TimeWindowGk(fs.openRead('gk.bin'));</code></pre>
</div>
</section>
<section>
<h4>Parameter</h4>
<table class="jsdoc-details-table">
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Optional</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<p>fout</p>
</td>
<td>
<p><a href="module-fs.FOut.html">module:fs.FOut</a></p>
</td>
<td>
<p> </p>
</td>
<td>
<p>the output stream</p>
</td>
</tr>
</tbody>
</table>
</section>
<dl class="dl-compact">
<dt>Returns</dt>
<dd>
<p><code><a href="module-fs.FOut.html">module:fs.FOut</a></code> the output stream <code>fout</code></p>
</dd>
</dl>
<h3 id="getParams"><span class="symbol-name">getParams</span><span class="signature"><span class="signature-params">()</span> → <span class="signature-returns"> <a href="module-analytics.html#~TimeWindowGkParam">module:analytics~TimeWindowGkParam</a></span></span></h3>
<p>Returns the models' parameters as a JavaScript object (JSON). These parameters are the same as are set through the constructor.</p>
<dl class="dl-compact">
<dt>Returns</dt>
<dd>
<p><code><a href="module-analytics.html#~TimeWindowGkParam">module:analytics~TimeWindowGkParam</a></code> The construction parameters.</p>
<p>var qm = require('qminer'); var fs = qm.fs; var analytics = qm.analytics;</p>
<p>// create the default object var gk = new analytics.TimeWindowGk({ window: 5, quantileEps: 0.001, countEps: 0.0005 });
</p>
<p>// create the data used for calculating quantiles var inputs = [10, 1, 2, 8, 9, 5, 6, 4, 7, 3]; var times = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];</p>
<p>// fit the model for (var i = 0; i < inputs.length; i++) { gk.partialFit(times[i], inputs[i]); } var params = gk.getParams();</p>
<p>console.log(params.window); console.log(params.quantileEps); console.log(params.countEps);
</p>
</dd>
</dl>
<h3 id="predict"><span class="symbol-name">predict</span><span class="signature"><span class="signature-params">(p)</span> → <span class="signature-returns"> number</span></span></h3>
<p>Given an input cumulative probability, returns a quantile associated with that probability (e.g. for input 0.5 it will return the median).</p>
<section>
<h4>
Example
</h4>
<div>
<pre class="prettyprint"><code>var qm = require('qminer');
var gk = new qm.analytics.TimeWindowGk({
window: 100 // window is 2 days
});
gk.partialFit(0, 1.0);
gk.partialFit(1, 1.0);
gk.partialFit(2, 1.0);
gk.partialFit(3, 1.0);
gk.partialFit(4, 1.0);
console.log(gk.predict(0.01)); // prints the first percentile
console.log(gk.predict(0.25)); // prints the first quartile
console.log(gk.predict(0.5)); // prints the median</code></pre>
</div>
</section>
<section>
<h4>Parameter</h4>
<table class="jsdoc-details-table">
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Optional</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<p>p</p>
</td>
<td>
<p>number</p>
</td>
<td>
<p> </p>
</td>
<td>
<p>cumulative probability between 0 and 1 (both inclusive)</p>
</td>
</tr>
</tbody>
</table>
</section>
<dl class="dl-compact">
<dt>Returns</dt>
<dd>
<p><code>number</code> quantile associated with p</p>
</dd>
</dl>
</section>
</section>
</div>
</div>
<nav id="jsdoc-toc-nav" role="navigation"></nav>
</div>
</div>
<footer id="jsdoc-footer" class="jsdoc-footer">
<div id="jsdoc-footer-container">
<p>
</p>
</div>
</footer>
<script src="scripts/jquery.min.js"></script>
<script src="scripts/jquery.cookie.js"></script>
<script src="scripts/tree.jquery.js"></script>
<script src="scripts/prettify.js"></script>
<script src="scripts/jsdoc-toc.js"></script>
<script src="scripts/linenumber.js"></script>
<script src="scripts/scrollanchor.js"></script>
</body>
</html>