glob-fs
Version:
file globbing for node.js. speedy and powerful alternative to node-glob.
565 lines (392 loc) • 14.8 kB
Markdown
# glob-fs [](http://badge.fury.io/js/glob-fs)
> file globbing for node.js. speedy and powerful alternative to node-glob.
## Usage
```js
var glob = require('glob-fs')({ gitignore: true });
var files = glob.readdirSync('**/*.js');
```
[Run actual examples](./examples/):
Jump to docs sections:
* [glob.readdir](#async)
* [glob.readPromise](#promise)
* [glob.readStream](#stream)
* [glob.readdirSync](#sync)
## Table of contents
<!-- toc -->
* [Install](#install)
* [Usage](#usage)
* [API](#api)
* [Middleware](#middleware)
- [Middleware examples](#middleware-examples)
- [Middleware conventions](#middleware-conventions)
- [Advice for middleware authors](#advice-for-middleware-authors)
* [Globbing examples](#globbing-examples)
- [async](#async)
- [promise](#promise)
- [stream](#stream)
- [sync](#sync)
* [Events](#events)
- [Event examples](#event-examples)
* [FAQ](#faq)
* [TODO](#todo)
* [Community middleware](#community-middleware)
* [Related projects](#related-projects)
* [Running tests](#running-tests)
* [Contributing](#contributing)
* [Author](#author)
* [License](#license)
_(Table of contents generated by [verb](https://github.com/assemble/verb))_
<!-- tocstop -->
## Install
Install with [npm](https://www.npmjs.com/)
```sh
$ npm i glob-fs --save
```
## Usage
**Params**
All "read" methods take a glob pattern and an `options` object.
* `pattern` **{String}**: Glob pattern to use for matching. (multiple pattern support is planned)
* `options` **{Object}**: Options for `glob-fs` or middleware.
**Examples:**
```js
// sync
var files = glob.readdirSync('*.js', {});
// async
glob.readdir('*.js', function(err, files) {
console.log(files);
});
// stream
glob.readdirStream('*.js', {})
.on('data', function(file) {
console.log(file);
});
// promise
glob.readdirPromise('*.js')
.then(function(files) {
console.log(file);
});
```
## API
### [.readdir](lib/readers.js#L25)
Asynchronously glob files or directories that match the given `pattern`.
**Params**
* `pattern` **{String}**: Glob pattern
* `options` **{Object}**
* `cb` **{Function}**: Callback
**Example**
```js
var glob = require('glob-fs')({ gitignore: true });
glob.readdir('*.js', function (err, files) {
//=> do stuff with `files`
});
```
### [.readdirSync](lib/readers.js#L59)
Synchronously glob files or directories that match the given `pattern`.
**Params**
* `pattern` **{String}**: Glob pattern
* `options` **{Object}**
* `returns` **{Array}**: Returns an array of files.
**Example**
```js
var glob = require('glob-fs')({ gitignore: true });
var files = glob.readdirSync('*.js');
//=> do stuff with `files`
```
### [.readdirStream](lib/readers.js#L90)
Stream files or directories that match the given glob `pattern`.
**Params**
* `pattern` **{String}**: Glob pattern
* `options` **{Object}**
* `returns` **{Stream}**
**Example**
```js
var glob = require('glob-fs')({ gitignore: true });
glob.readdirStream('*.js')
.on('data', function (file) {
console.log(file.path);
})
.on('error', console.error)
.on('end', function () {
console.log('end');
});
```
### [Glob](index.js#L42)
Optionally create an instance of `Glob` with the given `options`.
**Params**
* `options` **{Object}**
**Example**
```js
var Glob = require('glob-fs').Glob;
var glob = new Glob();
```
### [.use](index.js#L178)
Add a middleware to be called in the order defined.
**Params**
* `fn` **{Function}**
* `returns` **{Object}**: Returns the `Glob` instance, for chaining.
**Example**
```js
var gitignore = require('glob-fs-gitignore');
var dotfiles = require('glob-fs-dotfiles');
var glob = require('glob-fs')({ foo: true })
.use(gitignore())
.use(dotfiles());
var files = glob.readdirSync('**');
```
### [.exclude](index.js#L219)
Thin wrapper around `.use()` for easily excluding files or directories that match the given `pattern`.
**Params**
* `pattern` **{String}**
* `options` **{Object}**
**Example**
```js
var gitignore = require('glob-fs-gitignore');
var dotfiles = require('glob-fs-dotfiles');
var glob = require('glob-fs')()
.exclude(/\.foo$/)
.exclude('*.bar')
.exclude('*.baz');
var files = glob.readdirSync('**');
//=> ['index.js', 'README.md', ...]
```
## Middleware
glob-fs uses middleware to add file matching and exclusion capabilities, or other features that may or may not eventually become core functionality.
**What is a middleware?**
A middleware is a function that "processes" files as they're read from the file system by glob-fs.
Additionally, middleware can:
* be chained
* `include` or `exclude` a file based on some condition, like whether or not one of its properties matches a regex or glob pattern.
* determine whether or not to continue recursing in a specific directory
* modifying an existing property to the `file` object
* add a new property to the `file` object
### Middleware examples
**Ignoring files**
In the following example, `notemp` is a complete and functional middleware for excluding any filepath that has the substring `temp`:
```js
var glob = require('glob-fs')();
function notemp(file) {
if (/temp/.test(file.path)) {
file.exclude = true;
}
return file;
}
glob.use(notemp)
.readdirStream('**/*.js')
.on('data', function(file) {
console.log(file.relative);
});
```
**Matching**
Pattern matching is done by default in glob-fs, but you get disable the built-in matchers or get more specific by adding a middleware that uses [micromatch][] or [minimatch](https://github.com/isaacs/minimatch#readme) for matching files.
```js
var glob = require('glob-fs')({ gitignore: true });
var mm = require('micromatch');
glob.use(function(file) {
if (mm.isMatch(file.relative, 'vendor/**')) file.exclude = true;
return file;
})
.readdirStream('**/*.js')
.on('data', function(file) {
console.log(file.relative);
});
```
**recursion**
Here is how a middleware might determine whether or not to recurse based on a certain pattern:
```js
var glob = require('glob-fs')();
// this specific check is already done by glob-fs, it's just used here as an example
function recurse(file) {
// `file.pattern` is an object with a `glob` (string) property
file.recurse = file.pattern.glob.indexOf('**') !== -1;
return file;
}
// use the middleware
glob.use(recurse)
.readdir('**/*.js', function(err, files) {
console.log(files);
});
```
**Built-in middleware**
Currently glob-fs includes and runs the following middleware automatically:
<!-- list automatically generated from deps. see .verb.md -->
* [glob-fs-dotfiles](https://github.com/jonschlinkert/glob-fs-dotfiles): glob-fs middleware for automatically ignoring dotfiles.
* [glob-fs-gitignore](https://github.com/jonschlinkert/glob-fs-gitignore): glob-fs middleware for automatically ignoring files specified in `.gitignore`
**Disabling built-ins**
To disable built-in middleware and prevent them from running, pass `builtins: false` on the global options. This will disable **all built-in middleware**.
Example:
```js
var glob = require('glob-fs')({builtins: false});
```
To disable a specific middleware from running, you can usually pass the name of the middleware on the options, like `dotfiles: false`, but it's best to check the readme of that middleware for specifics.
### Middleware conventions
* **Naming**: any middleware published to npm should be prefixed with `glob-fs-`, as in: `glob-fs-dotfiles`.
* **Keywords**: please add `glob-fs` to the keywords array in package.json
* **Options**: all middleware should return a function that takes an `options` object, as in the [Middleware Example](#middleware-example)
* **Return `file`**: all middleware should return the `file` object after processing.
### Advice for middleware authors
* A middleware should only do one specific thing.
* Multiple middleware libs can be bundled together to create a single middleware.
* Pattern matching should be extremely specific. Don't force downstream middleware to reverse your mistakes.
* As mentioned in the [middleware conventions](#middleware-conventions) section, **always return the `file` object**.
* A single conditional should only set `file.exclude` to `true`, or `file.include` to `true`, never both.
* It's completely okay to check `this.options`
* Middleware modules should be fully documented.
## Globbing examples
Note that the `gitignore` option is already `true` by default, it's just shown here as a placeholder for how options may be defined.
### async
```js
var glob = require('glob-fs')({ gitignore: true });
glob.readdir('**/*.js', function(err, files) {
console.log(files);
});
```
### promise
```js
var glob = require('glob-fs')({ gitignore: true });
glob.readdirPromise('**/*')
.then(function (files) {
console.log(files);
});
```
### stream
```js
var glob = require('glob-fs')({ gitignore: true });
glob.readdirStream('**/*')
.on('data', function (file) {
console.log(file.path);
})
```
### sync
```js
var glob = require('glob-fs')({ gitignore: true });
var files = glob.readdirSync('**/*.js');
console.log(files);
```
## Events
_(WIP)_
The following events are emitted with all "read" methods:
* `read`: emitted immediately before an iterator calls the first middleware.
* `include`: emits a `file` object when it's matched
* `exclude`: emits a `file` object when it's ignored/excluded
* `file`: emits a `file` object when the iterator pushes it into the results array. Only applies to `sync`, `async` and `promise`.
* `dir`: emits a `file` object when the iterator finds a directory
* `end` when the iterator is finished reading
* `error` on errors
### Event examples
**async**
```js
var glob = require('..')({ gitignore: true });
glob.on('dir', function (file) {
console.log(file);
});
glob.readdir('**/*.js', function (err, files) {
if (err) return console.error(err);
console.log(files.length);
});
```
**promise**
```js
var glob = require('glob-fs')({ gitignore: true });
glob.on('include', function (file) {
console.log('including:', file.path);
});
glob.on('exclude', function (file) {
console.log('excluding:', file.path);
});
glob.readdirPromise('**/*');
```
**sync**
Also has an example of a custom event, emitted from a middleware:
```js
var glob = require('glob-fs')({ gitignore: true })
.use(function (file) {
if (/\.js$/.test(file.path)) {
// custom event
this.emit('js', file);
}
return file;
});
glob.on('js', function (file) {
console.log('js file:', file.path);
});
glob.on('exclude', function (file) {
console.log('excluded:', i.excludes++);
});
glob.on('include', function (file) {
console.log('included:', i.includes++)
});
glob.on('end', function () {
console.log('total files:', this.files.length);
});
glob.readdirSync('**/*.js');
```
**stream**
```js
var glob = require('glob-fs')({ gitignore: true })
glob.readdirStream('**/*')
.on('data', function (file) {
console.log(file.path)
})
.on('error', console.error)
.on('end', function () {
console.log('end');
});
```
## FAQ
* when files are read from the file system, an object is created to keep a record of the file's `path`, `dirname`, and fs `stat` object and other pertinent information that makes it easier to make decisions about inclusion and exclusion later on.
* `file` objects are decorated with a `parse` method that is used to calculate the `file.relative` and `file.absolute` properties.
* the `file.parse()` method is called in the iterator, right after the call to ` fs.stats` and just before the call to the middleware handler (`.handle()`). This ensures that all middleware have access to necessary path information.
* `file.relative` is the file path that's actually pushed into the `files` array that is ultimately returned.
* `file.relative` is calculated using `path.relative(file.path, cwd)`, where `cwd` is passed on the options (globally, or on a middleware), and `file.path` is typically the absolute, actual file path to the file being globbed.
## TODO
**middleware**
* [x] middleware
* [x] middleware handler
* [ ] externalize middleware to modules (started, [prs welcome!](#contributing))
**events**
* [x] events
**tests**
* [x] unit tests (need to be moved)
**iterators**
* [x] sync iterator
* [x] async iterator
* [x] stream iterator
* [x] promise iterator
**read methods**
* [x] glob.readdir (async)
* [x] glob.readdirSync
* [x] glob.readdirStream
* [x] glob.readdirPromise
**patterns**
* [ ] Multiple pattern support. will need to change pattern handling, middleware handling. this is POC currently
* [ ] Negation patterns (might not do this, since it can be handled in middleware)
* [ ] matching method, memoized/cached/bound to a glob pattern or patterns, so it can be reused without having to recompile the regex.
**other**
* [ ] clean up `./lib`
* [ ] comparsion to [node-glob][]
## Community middleware
_(Add your project to the [.verb.md](./.verb.md) template do a PR!)_
<!-- remove these after we get some community middleware libs listed -->
* [glob-fs-dotfiles](https://github.com/jonschlinkert/glob-fs-dotfiles): glob-fs middleware for automatically ignoring dotfiles.
* [glob-fs-gitignore](https://github.com/jonschlinkert/glob-fs-gitignore): glob-fs middleware for automatically ignoring files specified in `.gitignore`
## Related projects
* [braces](https://github.com/jonschlinkert/braces): Fastest brace expansion for node.js, with the most complete support for the Bash 4.3 braces… [more](https://github.com/jonschlinkert/braces)
* [fill-range](https://github.com/jonschlinkert/fill-range): Fill in a range of numbers or letters, optionally passing an increment or multiplier to… [more](https://github.com/jonschlinkert/fill-range)
* [is-glob](https://github.com/jonschlinkert/is-glob): Returns `true` if the given string looks like a glob pattern.
* [micromatch](https://github.com/jonschlinkert/micromatch): Glob matching for javascript/node.js. A drop-in replacement and faster alternative to minimatch and multimatch. Just… [more](https://github.com/jonschlinkert/micromatch)
## Running tests
Install dev dependencies:
```sh
$ npm i -d && npm test
```
## Contributing
Pull requests and stars are always welcome. For bugs and feature requests, [please create an issue](https://github.com/jonschlinkert/glob-fs/issues/new)
## Author
**Jon Schlinkert**
+ [github/jonschlinkert](https://github.com/jonschlinkert)
+ [twitter/jonschlinkert](http://twitter.com/jonschlinkert)
## License
Copyright © 2015 Jon Schlinkert
Released under the MIT license.
***
_This file was generated by [verb-cli](https://github.com/assemble/verb-cli) on July 11, 2015._