feedr
Version:
Use feedr to fetch the data from a remote url, respect its caching, and parse its data. Despite its name, it's not just for feed data but also for all data that you can feed into it (including binary data).
177 lines (113 loc) • 11.5 kB
Markdown
<!-- TITLE/ -->
<h1>Feedr</h1>
<!-- /TITLE -->
<!-- BADGES/ -->
<span class="badge-travisci"><a href="http://travis-ci.com/bevry/feedr" title="Check this project's build status on TravisCI"><img src="https://img.shields.io/travis/com/bevry/feedr/master.svg" alt="Travis CI Build Status" /></a></span>
<span class="badge-npmversion"><a href="https://npmjs.org/package/feedr" title="View this project on NPM"><img src="https://img.shields.io/npm/v/feedr.svg" alt="NPM version" /></a></span>
<span class="badge-npmdownloads"><a href="https://npmjs.org/package/feedr" title="View this project on NPM"><img src="https://img.shields.io/npm/dm/feedr.svg" alt="NPM downloads" /></a></span>
<span class="badge-daviddm"><a href="https://david-dm.org/bevry/feedr" title="View the status of this project's dependencies on DavidDM"><img src="https://img.shields.io/david/bevry/feedr.svg" alt="Dependency Status" /></a></span>
<span class="badge-daviddmdev"><a href="https://david-dm.org/bevry/feedr#info=devDependencies" title="View the status of this project's development dependencies on DavidDM"><img src="https://img.shields.io/david/dev/bevry/feedr.svg" alt="Dev Dependency Status" /></a></span>
<br class="badge-separator" />
<span class="badge-githubsponsors"><a href="https://github.com/sponsors/balupton" title="Donate to this project using GitHub Sponsors"><img src="https://img.shields.io/badge/github-donate-yellow.svg" alt="GitHub Sponsors donate button" /></a></span>
<span class="badge-patreon"><a href="https://patreon.com/bevry" title="Donate to this project using Patreon"><img src="https://img.shields.io/badge/patreon-donate-yellow.svg" alt="Patreon donate button" /></a></span>
<span class="badge-flattr"><a href="https://flattr.com/profile/balupton" title="Donate to this project using Flattr"><img src="https://img.shields.io/badge/flattr-donate-yellow.svg" alt="Flattr donate button" /></a></span>
<span class="badge-liberapay"><a href="https://liberapay.com/bevry" title="Donate to this project using Liberapay"><img src="https://img.shields.io/badge/liberapay-donate-yellow.svg" alt="Liberapay donate button" /></a></span>
<span class="badge-buymeacoffee"><a href="https://buymeacoffee.com/balupton" title="Donate to this project using Buy Me A Coffee"><img src="https://img.shields.io/badge/buy%20me%20a%20coffee-donate-yellow.svg" alt="Buy Me A Coffee donate button" /></a></span>
<span class="badge-opencollective"><a href="https://opencollective.com/bevry" title="Donate to this project using Open Collective"><img src="https://img.shields.io/badge/open%20collective-donate-yellow.svg" alt="Open Collective donate button" /></a></span>
<span class="badge-crypto"><a href="https://bevry.me/crypto" title="Donate to this project using Cryptocurrency"><img src="https://img.shields.io/badge/crypto-donate-yellow.svg" alt="crypto donate button" /></a></span>
<span class="badge-paypal"><a href="https://bevry.me/paypal" title="Donate to this project using Paypal"><img src="https://img.shields.io/badge/paypal-donate-yellow.svg" alt="PayPal donate button" /></a></span>
<span class="badge-wishlist"><a href="https://bevry.me/wishlist" title="Buy an item on our wishlist for us"><img src="https://img.shields.io/badge/wishlist-donate-yellow.svg" alt="Wishlist browse button" /></a></span>
<!-- /BADGES -->
<!-- DESCRIPTION/ -->
Use feedr to fetch the data from a remote url, respect its caching, and parse its data. Despite its name, it's not just for feed data but also for all data that you can feed into it (including binary data).
<!-- /DESCRIPTION -->
<!-- INSTALL/ -->
<h2>Install</h2>
<a href="https://npmjs.com" title="npm is a package manager for javascript"><h3>npm</h3></a>
<ul>
<li>Install: <code>npm install --save feedr</code></li>
<li>Require: <code>require('feedr')</code></li>
</ul>
<h3><a href="https://editions.bevry.me" title="Editions are the best way to produce and consume packages you care about.">Editions</a></h3>
<p>This package is published with the following editions:</p>
<ul><li><code>feedr</code> aliases <code>feedr/source/index.js</code></li>
<li><code>feedr/source/index.js</code> is esnext source code with require for modules</li></ul>
<p>Environments older than Node.js v8 may need <a href="https://babeljs.io/docs/usage/polyfill/" title="A polyfill that emulates missing ECMAScript environment features">Babel's Polyfill</a> or something similar.</p>
<!-- /INSTALL -->
## Usage
``` javascript
// Create a new feedr instance
const feedr = require('feedr').create({/* optional configuration */})
// Prepare our feeds that we want read
const feeds = {
someAtomFeed: 'https://github.com/bevry/feedr/commits/for-testing.atom'
someJsonFile: 'https://raw.githubusercontent.com/bevry/feedr/for-testing/package.json'
}
// Read a single feed
feedr.readFeed(feeds.someAtomFeed, {/* optional configuration */}, function (err, data, headers) {
console.log(err, data, headers)
})
// Read all the feeds together
feedr.readFeeds(feeds, {/* optional configuration */}, function (err, result) {
console.log(err, result.someAtomFeed, result.someJsonFile)
})
```
## Configuration
Feed configuration defaults / global configuration properties are:
- `log` defaults to `null`, log function to use
- `tmpPath` defaults to system tmp path, the tempory path to cache our feedr results to
- `cache` defaults to one day `1000*60*60*24`, available values:
- `Number` prefers to use the cache when it is within the range of the number in milliseconds
- `true` prefers to use the cache when the response headers indicate that the cache is still valid
- `"preferred"` will always use the cache if the cache exists
- `false` will never use the cache
- `xml2jsOptions` defaults to `null`, the options to send to [xml2js](https://github.com/Leonidas-from-XIV/node-xml2js)
- `requestOptions` defaults to `null`, the options to send to [request](https://github.com/mikeal/request)
Feed configuration properties are:
- `url` required, the url to fetch
- `hash` defaults to hash of the url, the hashed url for caching
- `name` defaults to hash, the name of the feed for use in debugging
- `path` defaults to tmp feed path, the path to save the file to
- `parse` defaults to `true`, whether or not we should attempt to parse the response data, supported values are `xml`, `json`, `cson`, `yaml`, `string`, `raw`/`false`. If `true` will try all the available parsers. Can also be a function with the signature `({response, data, feed, feedr}, next(err, data))`
- `check` defaults to `true`, whether or not we should check the response data for custom error messages. Can also be a function with the signature `({response, data, feed, feedr}, next(err))`
- `xml2jsOptions` defaults to global value, the options to send to [xml2js](https://github.com/Leonidas-from-XIV/node-xml2js)
- `requestOptions` defaults to global value, the options to send to [request](https://github.com/mikeal/request)
<!-- HISTORY/ -->
<h2>History</h2>
<a href="https://github.com/bevry/feedr/blob/master/HISTORY.md#files">Discover the release history by heading on over to the <code>HISTORY.md</code> file.</a>
<!-- /HISTORY -->
<!-- CONTRIBUTE/ -->
<h2>Contribute</h2>
<a href="https://github.com/bevry/feedr/blob/master/CONTRIBUTING.md#files">Discover how you can contribute by heading on over to the <code>CONTRIBUTING.md</code> file.</a>
<!-- /CONTRIBUTE -->
<!-- BACKERS/ -->
<h2>Backers</h2>
<h3>Maintainers</h3>
These amazing people are maintaining this project:
<ul><li><a href="http://balupton.com">Benjamin Lupton</a> — <a href="https://github.com/bevry/feedr/commits?author=balupton" title="View the GitHub contributions of Benjamin Lupton on repository bevry/feedr">view contributions</a></li></ul>
<h3>Sponsors</h3>
No sponsors yet! Will you be the first?
<span class="badge-githubsponsors"><a href="https://github.com/sponsors/balupton" title="Donate to this project using GitHub Sponsors"><img src="https://img.shields.io/badge/github-donate-yellow.svg" alt="GitHub Sponsors donate button" /></a></span>
<span class="badge-patreon"><a href="https://patreon.com/bevry" title="Donate to this project using Patreon"><img src="https://img.shields.io/badge/patreon-donate-yellow.svg" alt="Patreon donate button" /></a></span>
<span class="badge-flattr"><a href="https://flattr.com/profile/balupton" title="Donate to this project using Flattr"><img src="https://img.shields.io/badge/flattr-donate-yellow.svg" alt="Flattr donate button" /></a></span>
<span class="badge-liberapay"><a href="https://liberapay.com/bevry" title="Donate to this project using Liberapay"><img src="https://img.shields.io/badge/liberapay-donate-yellow.svg" alt="Liberapay donate button" /></a></span>
<span class="badge-buymeacoffee"><a href="https://buymeacoffee.com/balupton" title="Donate to this project using Buy Me A Coffee"><img src="https://img.shields.io/badge/buy%20me%20a%20coffee-donate-yellow.svg" alt="Buy Me A Coffee donate button" /></a></span>
<span class="badge-opencollective"><a href="https://opencollective.com/bevry" title="Donate to this project using Open Collective"><img src="https://img.shields.io/badge/open%20collective-donate-yellow.svg" alt="Open Collective donate button" /></a></span>
<span class="badge-crypto"><a href="https://bevry.me/crypto" title="Donate to this project using Cryptocurrency"><img src="https://img.shields.io/badge/crypto-donate-yellow.svg" alt="crypto donate button" /></a></span>
<span class="badge-paypal"><a href="https://bevry.me/paypal" title="Donate to this project using Paypal"><img src="https://img.shields.io/badge/paypal-donate-yellow.svg" alt="PayPal donate button" /></a></span>
<span class="badge-wishlist"><a href="https://bevry.me/wishlist" title="Buy an item on our wishlist for us"><img src="https://img.shields.io/badge/wishlist-donate-yellow.svg" alt="Wishlist browse button" /></a></span>
<h3>Contributors</h3>
These amazing people have contributed code to this project:
<ul><li><a href="http://balupton.com">Benjamin Lupton</a> — <a href="https://github.com/bevry/feedr/commits?author=balupton" title="View the GitHub contributions of Benjamin Lupton on repository bevry/feedr">view contributions</a></li>
<li><a href="https://github.com/Zearin">Zearin</a> — <a href="https://github.com/bevry/feedr/commits?author=Zearin" title="View the GitHub contributions of Zearin on repository bevry/feedr">view contributions</a></li>
<li><a href="http://github.com/apps/dependabot">dependabot[bot]</a> — <a href="https://github.com/bevry/feedr/commits?author=dependabot[bot]" title="View the GitHub contributions of dependabot[bot] on repository bevry/feedr">view contributions</a></li>
<li><a href="http://github.com/apps/dependabot-preview">dependabot-preview[bot]</a> — <a href="https://github.com/bevry/feedr/commits?author=dependabot-preview[bot]" title="View the GitHub contributions of dependabot-preview[bot] on repository bevry/feedr">view contributions</a></li></ul>
<a href="https://github.com/bevry/feedr/blob/master/CONTRIBUTING.md#files">Discover how you can contribute by heading on over to the <code>CONTRIBUTING.md</code> file.</a>
<!-- /BACKERS -->
<!-- LICENSE/ -->
<h2>License</h2>
Unless stated otherwise all works are:
<ul><li>Copyright © 2012+ <a href="http://bevry.me">Bevry Pty Ltd</a></li></ul>
and licensed under:
<ul><li><a href="http://spdx.org/licenses/MIT.html">MIT License</a></li></ul>
<!-- /LICENSE -->