got-resume-next
Version:
Fetch via HTTP/HTTPS using got with automatic resume after network failures
181 lines (107 loc) • 6.3 kB
Markdown
# Fork Notes
- Updated all dependencies
- Stripped it down to just the essential features (removed toFile, transform, etc. since you can do those yourself)
- Rewrote all of the streaming behavior
- Fixed issues with handling errors
- Promises to be responsive to PRs
All thanks and credit to the original author.
# got-resume.js
# Fetch via HTTP/HTTPS using got with automatic resume after network failures
## Current status
[](https://www.npmjs.com/package/got-resume)
[](http://travis-ci.org/overlookmotel/got-resume)
[](https://david-dm.org/overlookmotel/got-resume)
[](https://david-dm.org/overlookmotel/got-resume)
[](https://greenkeeper.io/)
[](https://coveralls.io/r/overlookmotel/got-resume)
## Usage
Use [got](https://www.npmjs.com/package/got) to make an HTTP request with automatic retries for network errors.
Designed for downloading large files. If transfer fails part-way through, will retry, resuming from point where previous attempt finished, using HTTP range headers.
### gotResume( [url], [options] ) -> Stream
```js
const stream = gotResume('http://google.com/')
stream.pipe(fs.createWriteStream('foo.html'))
stream.on('error', err => console.log('Failed!'))
stream.on('end', () => console.log('Finished!'))
```
### Options
#### url
Alternative way to provide URL.
```js
const stream = gotResume({ url: 'http://google.com/' })
```
#### attempts
Max number of attempts in a row yielding no data (i.e. failed connection, empty response) before aborting.
Set to `0` for no limit. Default `10`.
```js
const stream = gotResume('http://google.com/', { attempts: 0 })
```
#### attemptsTotal
Max number of total attempts before aborting.
Set to `0` for no limit. Default `0`.
#### backoff
Function to determine wait in milliseconds before retry. Called with arguments `(attempt, transfer)`.
`attempt` is what attempt number for current chunk (reset to zero when a new chunk is successfully received).
`transfer` is the internal `Transfer` object (see below).
If function returns `false`, the transfer is cancelled. If using this mechanism, `options.attempts` should be set to `0` so it does not interfere.
If not provided, default backoff function starts with 1000ms and doubles each time:
```js
function backoff(attempt) {
return Math.pow(2, attempt - 1) * 1000
}
```
#### length
Length of response expected in bytes. If undefined, `length` will be determined from HTTP `content-length` header.
If server does not provide `content-length` header, and `options.length` is not set, transfer will be considered complete when first successful request completes.
If `options.length` is set, only that number of bytes will be fetched (i.e. file will be truncated).
#### needLength
Set to true if you require the length of the transfer to be retrieved at start of the transfer. Default: false
Explanation: By default got will use transfer encoding (e.g. gzip). This makes the `content-length` HTTP header unreliable. Setting `options.needLength` disables encoding so length should be retrieved accurately (if server provides it).
#### pre
An async function that is run before each chunk request. Must return a `Promise`. Request will commence once promise resolves.
Useful where some authentication requires being set up before the transfer HTTP request, or where resource has a different URL each time (e.g. some file transfer services).
`pre` function is called with `Transfer` object (see below). To set URL for next chunk, `pre` should set `transfer.url`. To alter `got` options, should set `transfer.gotOptions`.
```js
function pre(transfer) {
transfer.gotOptions.headers['user-agent'] = 'Stealth 2.0'
return Promise.resolve()
}
```
#### log
Function to receive logging information e.g. HTTP responses.
```js
const stream = gotResume('http://google.com/', { log: console.log })
```
#### got
Options to pass to `got`. See [got documentation](https://www.npmjs.com/package/got) for details.
```js
const stream = gotResume('http://google.com/', { got: { method: 'POST' } })
```
### Events
#### error
Emitted with a `gotResume.TransferError` on stream when transfer fails and has exhausted retries.
#### end
Emitted when transfer completes.
NB Is also emitted after `error` event if transfer fails.
#### progress
Emitted when data received. Emitted with progress object of form `{transferred: 100, total: 3000}`.
#### request
Emitted with HTTP request object when first HTTP request made to server.
NB Not emitted again for each retry HTTP request. You cannot abort the transfer with `request.abort()` as the request may be finished if a retry has happened.
#### response
Emitted when first successful HTTP response is received. NB Not emitted again for each retry HTTP request.
Useful for e.g. determining length of transfer:
```js
const stream = gotResume('http://google.com/')
stream.on('response', res => console.log('Length: ', stream.transfer.length))
```
### Cancellation
The stream returned by `gotStream()` has an additional method `.cancel()`. Calling `.cancel()` will abort the transfer and cause the stream to emit an `error` event with a `gotResume.CancelError`.
If the transfer is complete before `.cancel()` is called, no `error` event will be emitted.
If `options.pre` function is supplied and `.cancel()` is called while `options.pre` is running, `.cancel()` method on the promise returned by `options.pre` will be called if it exists. Otherwise the transfer will abort once the promise resolves.
### Transfer object
A transfer in progress is represented internally as an instance of `gotResume.Transfer` class.
Transfer object is stored as `stream.transfer` and also passed to `options.pre` function.
## Tests
Use `npm test` to run the tests. Use `npm run cover` to check coverage.
Few tests so far but seems to work fine!