robots-txt-parser
Version:
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
77 lines (54 loc) • 2 kB
Markdown
# Changelog
All notable changes to this project will be documented in this file.
## 2.0.0 - 2.0.0 release (21/06/2021)
### Changed
- Rewrite of robots.js
- Additional testing.
- Fixed bugs with callbacks.
## 1.0.10 - Testing + cache usage (02/06/2021)
### Changed
- Fixed lack of cache usage.
- Additional testing.
## 1.0.9 - ESLint usage + related cleanup (23/05/2021)
### Changed
- Fixed Coveralls reporting.
- Fixed lcov output.
## 1.0.8 - ESLint usage + related cleanup (21/05/2021)
### Changed
- Added eslint + better code style to old project.
## 1.0.6 - Documentation cleanup pass (16/05/2021)
### Changed
- Fixed scroll-to links to work on both github + npm.
## 1.0.5 - Documentation cleanup pass (16/05/2021)
### Changed
- Cleaned up readme file.
- Documentation format improvements.
- Fixed bad example.
## [1.0.4] - Minor additions and fixes (13/5/2021)
### Added
- Additional testing on fetch command.
- ClearCache() function.
- Testing for clearCache().
### Changed
- Fixed no timeout on bad connection bug.
- Fixed bad coverage test command.
- Fixed error on connection to unavailable site.
### Removed
- Unnecessary dependency.
- Unnecessary test files.
## [1.0.3] - Minor additions and fixes (10/4/2021)
### Added
- Parsing for Host records.
### Changed
- Fixed bug with parsing robots.txt files stripped of line endings.
- Fixed no user agent lower casing.
- Fixed parser rule overwriting.
- Fixed parser fallback.
## [1.0.0] - Initial release (1-05-2017)
### Added
- Robots.txt parser.
- Automated fetching.
[Unreleased]: https://github.com/chrisakroyd/robots-txt-parser
[1.0.4]: https://github.com/chrisakroyd/robots-txt-parser/tree/c549401d6f035b0e05b3045cca0ad5f2f51811ff
[1.0.3]: https://github.com/chrisakroyd/robots-txt-parser/tree/f834932b97b667315ff28a9cf61a2f6c30d7e6d9
[1.0.0]: https://github.com/chrisakroyd/robots-txt-parser/tree/c062d954f834ca6b54729e607ba8f674ea01c919