robots-txt-parser
Version:
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
| Filename | Content Type | Size | |
|---|---|---|---|
| ../ | |||
| 2.31 kB | |||
| 524 B | |||
| 1.02 kB | |||
| 843 B | |||
| 725 B | |||
| 883 B | |||
| 1.55 kB | |||
| 332 B | |||
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
| Filename | Content Type | Size | |
|---|---|---|---|
| ../ | |||
| 2.31 kB | |||
| 524 B | |||
| 1.02 kB | |||
| 843 B | |||
| 725 B | |||
| 883 B | |||
| 1.55 kB | |||
| 332 B | |||