reshuffle-aws-connectors
Version:
A set of Reshuffle connectors for AWS services
627 lines (437 loc) • 12.4 kB
Markdown
# reshuffle-aws-connectors
[Code](https://github.com/reshufflehq/reshuffle-aws-connectors) |
[npm](https://www.npmjs.com/package/reshuffle-aws-connectors) |
[Code sample](https://github.com/reshufflehq/reshuffle/blob/master/examples/aws/s3-list-files.js)
`npm install reshuffle-aws-connectors`
### Reshuffle AWS S3 Connector
This [Reshuffle](https://github.com/reshufflehq/reshuffle) connector can be used to manage AWS S3 buckets and objects. Full
details on the S3 API can be found
[here](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html).
The following example creates an API endpoint to list all files in an S3
bucket:
```js
const { HttpConnector, Reshuffle } = require('reshuffle')
const { AWSS3Connector } = require('reshuffle-aws-connectors')
const app = new Reshuffle()
const s3Connector = new AWSS3Connector(app, {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
bucket: process.env.AWS_DEFAULT_BUCKET,
})
const httpConnector = new HttpConnector(app)
httpConnector.on({ method: 'GET', path: '/list' }, async (event) => {
const keys = await s3Connector.listObjectKeys()
event.res.json(keys)
})
app.start(8000)
```
#### Table of Contents
[Configuration](#configuration) Configuration options
_Connector events_:
[bucketInitialized](#bucketInitialized) Bucket traching initialized
[bucketChanged](#bucketChanged) Bucket content changes
[objectAdded](#objectAdded) Object added to bucket
[objectModified](#objectModified) Object modified in bucket
[objectRemoved](#objectRemoved) Object removed from bucket
_Connector actions_:
[listBuckets](#listBuckets) Get a list of bucket info objects
[listBucketNames](#listBucketNames) Get a list of bucket names
[createBucket](#createBucket) Create a new bucket
[deleteBucket](#deleteBucket) Delete a bucket
[listObjects](#listObjects) Get a list of object info objects
[listObjectKeys](#listObjectKeys) Get a list of object keys
[copyObject](#copyObject) Create a copy of an existing object
[deleteObject](#deleteObject) Delete an object
[getObject](#getObject) Get the contents of an object
[putObject](#putObject) Create a new object
[getSignedURL](#getSignedURL) Get a signed URL for a single operation
[getSignedObjectGetURL](#getSignedObjectGetURL) Get a signed download URL
[getSignedObjectPutURL](#getSignedObjectPutURL) Get a signed upload URL
[getS3URL](#getS3URL) Get an S3 object URL
[getWebURL](#getWebURL) Get an web object URL
_SDK_:
[sdk](#sdk) Get direct SDK access
##### <a name="configuration"></a>Configuration options
```js
const app = new Reshuffle()
const awsS3Connector = new AWSS3Connector(app, {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
bucket: process.env.AWS_DEFAULT_BUCKET,
})
```
#### Connector events
##### <a name="bucketInitialized"></a>Bucket Initialized event
_Example:_
```js
async (objects) => {
console.log(objects)
}
```
This event is fired when the connector starts tracking a specific S3
bucket. More technically, it is fired when the connector first reads
the content of an S3 bucket and does not have a previous record of its
object in its internal database.
For example, if a set of scripts is used to synchronize the contents of
one bucket to another, this event can be used to read the contents of the
target bucket and copy over the missing objects. This prevents the need to
copy over every object in case of a database failure.
When this event is fired, neither the [bucketChanged](#bucketChanged)
event nor the individual [objectAdded](#objectAdded) events are fired for
the same objects. Subsequent additions or modification to the tracked
bucket will generate those events.
##### <a name="bucketChanged"></a>Bucket Changed event
_Event parameters:_
```
bucket: string - S3 bucket name
```
_Handler inputs:_
```
objects: object - Bucket state
```
_Example:_
```js
async (objects) => {
console.log('All keys:', Object.keys(objects).join(', '))
}
```
This event is triggered when one or more of the objects in an S3 buckets
change: a new object is created, the content of an existing object is
modified or an object is removed.
This event consolidates multiple changes. For each of this changes, the
appropriate `objectAdded`, `objectModified` or `objectRemoved` is also
fired. Those events are fired one per each object changed.
The `objects` argument has the following format:
```ts
{
'key 1': {
key: string, // equals to 'key 1' in this case
lastModified: Date,
eTag: '"..."' // 32 character hex string
size: number // In bytes
},
'key 2': { ... },
...
}
```
##### <a name="objectAdded"></a>Object Added event
_Event parameters:_
```
bucket: string - S3 bucket name
```
_Handler inputs:_
```
object: object - Object info
```
_Example:_
```js
async (object) => {
console.log('New object added:')
console.log(' Key:', object.key)
console.log(' Modified:', object.lastModified)
console.log(' eTag:', object.eTag)
console.log(' Size:', object.size, 'bytes')
}
```
This event is triggered once for each new object added to the bucket.
The `object` info argument has the following format:
```ts
{
key: string,
lastModified: Date,
eTag: '"..."' // 32 character hex string
size: number // Size in bytes
}
```
##### <a name="objectModified"></a>Object Modified event
_Event parameters:_
```
bucket: string - S3 bucket name
```
_Handler inputs:_
```
object: object - Object info
```
_Example:_
```js
async (object) => {
console.log('New object added:')
console.log(' Key:', object.key)
console.log(' Modified:', object.lastModified)
console.log(' eTag:', object.eTag)
console.log(' Size:', object.size, 'bytes')
}
```
This event is triggered once whenever the content of an object in the
bucket is modified.
The `object` info argument has the following format:
```ts
{
key: string,
lastModified: Date,
eTag: '"..."' // 32 character hex string
size: number // Size in bytes
}
```
##### <a name="objectRemoved"></a>Object Removed event
_Event parameters:_
```
bucket: string - S3 bucket name
```
_Handler inputs:_
```
object: object - Object info
```
_Example:_
```js
async (object) => {
console.log('New object added:')
console.log(' Key:', object.key)
console.log(' Modified:', object.lastModified)
console.log(' eTag:', object.eTag)
console.log(' Size:', object.size, 'bytes')
}
```
This event is triggered once whenever an object is removed from
the bucket.
The `object` info argument has the following format:
```ts
{
key: string,
lastModified: Date,
eTag: '"..."' // 32 character hex string
size: number // Size in bytes
}
```
#### Connector actions
##### <a name="listBuckets"></a>List Buckets action
_Definition:_
```ts
() => object[]
```
_Usage:_
```js
const buckets = await awsS3Connector.listBuckets()
```
Get a list of
[bucket information objects](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#listBuckets-property)
for accessible buckets.
##### <a name="listBucketNames"></a>List Bucket Names action
_Definition:_
```ts
() => string[]
```
_Usage:_
```js
const names = await awsS3Connector.listBucketNames()
```
Get a list of accessible bucket names.
##### <a name="createBucket"></a>Create Bucket action
_Definition:_
```ts
(
bucket: string,
region?: string,
) => void
```
_Usage:_
```js
await awsS3Connector.createBucket('my-bucket-name', 'us-west-1')
```
Create a new bucket.
##### <a name="deleteBucket"></a>Delete Bucket action
_Definition:_
```ts
(
bucket: string,
) => void
```
_Usage:_
```js
await awsS3Connector.deleteBucket('my-bucket-name')
```
Delete a bucket.
##### <a name="listObjects"></a>List Objects action
_Definition:_
```ts
(
bucket: string,
) => object[]
```
_Usage:_
```js
const objects = await awsS3Connector.listObjects('my-bucket-name')
```
Get a list of
[object information objects](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#listObjects-property)
for objects in the specified bucket.
##### <a name="listObjectKeys"></a>List Object Keys action
_Definition:_
```ts
(
bucket: string,
) => string[]
```
_Usage:_
```js
const keys = await awsS3Connector.listObjectKeys()
```
Get a list of object keys in the specified bucket.
##### <a name="copyObject"></a>Copy Object action
_Definition:_
```ts
(
sourceBucket: string,
sourceKey: string,
targetBucket: string,
targetKey: string,
) => object
```
_Usage:_
```js
const result = await awsS3Connector.copyObject(
'old-bucket',
'original.jpg',
'new-bucket',
'copy.jpg',
)
```
Create a new copy of an existing object. Returns a
[copy result](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#copyObject-property).
##### <a name="deleteObject"></a>Delete Object action
_Definition:_
```ts
(
bucket: string,
key: string,
) => void
```
_Usage:_
```js
await awsS3Connector.deleteObject('my-bucket-name', 'no-longer-needed.txt')
```
Delete an object from the specified bucket.
##### <a name="getObject"></a>Get Object action
_Definition:_
```ts
(
bucket: string,
key: string,
) => object
```
_Usage:_
```js
const info = await awsS3Connector.getObject('my-bucket-name', 'image.png')
```
Get information about an object, including its contents, as defined
[here](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getObject-property).
##### <a name="putObject"></a>Put Object action
_Definition:_
```ts
(
bucket: string,
key: string,
buffer: Buffer,
) => object
```
_Usage:_
```js
const info = await awsS3Connector.putObject(
'my-bucket-name',
'hello.txt',
Buffer.from('Hello, world!'),
)
```
Returns information about the new object, as defined
[here](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putObject-property).
##### <a name="getSignedURL"></a>Get Signed URL action
_Definition:_
```ts
(
operation: string,
key: string,
expires?: number = 60,
) => string
```
_Usage:_
```js
const url = await awsS3Connector.getSignedURL('getObject', 'me.png')
```
Get a pre-signed URL for a single operation. The URL can be used to
access an object without requiring any credentials.
The URL is valid for a limited time, as specified by `expires` in seconds.
##### <a name="getSignedObjectGetURL"></a>Get Signed Object Get URL action
_Definition:_
```ts
(
key: string,
expires?: number = 60,
) => string
```
_Usage:_
```js
const url = await awsS3Connector.getSignedObjectGetURL('me.png')
```
Get a pre-signed URL for downloading an object. The URL can be used to
download the content of an object without requiring any credentials.
The URL is valid for a limited time, as specified by `expires` in seconds.
##### <a name="getSignedObjectPutURL"></a>Get Signed Object Put URL action
_Definition:_
```ts
(
key: string,
expires?: number = 60,
) => string
```
_Usage:_
```js
const url = await awsS3Connector.getSignedObjectPutURL('you.png')
```
Get a pre-signed URL for uploading an object. The URL can be used with a PUT
HTTP request to create a new object without requiring any credentials.
The URL is valid for a limited time, as specified by `expires` in seconds.
##### <a name="getS3URL"></a>Get S3 URL action
_Definition:_
```ts
(
key: string,
bucket?: string,
) => string
```
_Usage:_
```js
const url = await awsS3Connector.getS3URL('image.png')
```
Get an S3 object URL in the form "s3://<bucket>//<key>". If `bucket` is omitted
then the valude from the connector options is used.
##### <a name="getWebURL"></a>Get Web URL action
_Definition:_
```ts
(
key: string,
bucket?: string,
) => string
```
_Usage:_
```js
const url = await awsS3Connector.getWebURL('image.png')
```
Get an HTTP URL for accessing objet `key` in `bucket`. If `bucket` is omitted
then the valude from the connector options is used.
This URL is only valid if the specified S3 bucket is configured for web access.
The action does not configure the bucket for web access nor does it validate
that such access is enabled.
#### SDK
##### <a name="sdk"></a>SDK action
_Definition:_
```ts
(
options ?: object,
) => object
```
_Usage:_
```js
const s3 = await awsS3Connector.sdk()
```
Get the underlying SDK object. You can specify additional options to override
or add to the required fields in the connector's configuration.