@d3oxy/s3-pilot/README.md

A TypeScript wrapper for AWS S3 and S3-compatible services (R2, MinIO, DigitalOcean Spaces) with a simplified single-client, single-bucket architecture.

# S3Pilot

S3Pilot is a TypeScript library that abstracts AWS S3 and S3-compatible service operations, making it easier to interact with buckets and objects. It provides a cleaner API to manage file uploads, deletions, signed URL generation, and secure file downloads.

## Features

-   Simple single-client, single-bucket architecture - one instance per bucket.
-   **S3-Compatible Services**: Works with Cloudflare R2, DigitalOcean Spaces, MinIO, and other S3-compatible storage providers.
-   Easily upload, delete, rename, and manage files in S3 buckets.
-   Bulk delete support for deleting multiple files efficiently.
-   Generate signed URLs for private access to objects.
-   Secure file download methods to avoid CORS issues.
-   Enhanced signed URLs with download-specific headers.
-   Streaming support for large files.
-   Optional bucket validation on initialization.
-   Supports custom key prefixes and folders.

## Installation

```bash
# Using pnpm
pnpm install @d3oxy/s3-pilot

# Using npm
npm install @d3oxy/s3-pilot

# Using bun
bun add @d3oxy/s3-pilot

# Using yarn
yarn add @d3oxy/s3-pilot
```

## Usage

### Initialize S3Pilot

First, import the `S3Pilot` class into your TypeScript project:

```typescript
import { S3Pilot } from "@d3oxy/s3-pilot";
```

### Configuration

Create a new instance of `S3Pilot` for each bucket you want to work with:

```typescript
// One instance per bucket
const bucket1 = new S3Pilot({
    region: "us-east-1",
    accessKeyId: "AWS_ACCESS_KEY_ID",
    secretAccessKey: "AWS_SECRET_ACCESS_KEY",
    bucket: "my-bucket-1",
    keyPrefix: process.env.NODE_ENV === "development" ? "dev" : undefined,
});

// Create another instance for a different bucket
const bucket2 = new S3Pilot({
    region: "us-west-2",
    accessKeyId: "AWS_ACCESS_KEY_ID",
    secretAccessKey: "AWS_SECRET_ACCESS_KEY",
    bucket: "my-bucket-2",
});
```

### S3-Compatible Services

S3Pilot works with any S3-compatible storage service. Use the `endpoint` option to specify the service URL and `publicBaseUrl` for generating public URLs.

#### Cloudflare R2

```typescript
const r2 = new S3Pilot({
    region: "auto", // R2 uses "auto" for region
    accessKeyId: "R2_ACCESS_KEY_ID",
    secretAccessKey: "R2_SECRET_ACCESS_KEY",
    bucket: "my-r2-bucket",
    endpoint: "https://ACCOUNT_ID.r2.cloudflarestorage.com",
    publicBaseUrl: "https://pub-xxx.r2.dev", // Your R2 public bucket URL or custom domain
});

// All methods work the same!
await r2.uploadFile({ filename: "test.txt", file: "Hello R2!", contentType: "text/plain" });
const url = r2.getUrl("test.txt"); // Returns: https://pub-xxx.r2.dev/test.txt
```

#### DigitalOcean Spaces

```typescript
const spaces = new S3Pilot({
    region: "nyc3",
    accessKeyId: "DO_SPACES_KEY",
    secretAccessKey: "DO_SPACES_SECRET",
    bucket: "my-space",
    endpoint: "https://nyc3.digitaloceanspaces.com",
    publicBaseUrl: "https://my-space.nyc3.digitaloceanspaces.com",
});
```

#### MinIO

```typescript
const minio = new S3Pilot({
    region: "us-east-1", // MinIO default
    accessKeyId: "minio-access-key",
    secretAccessKey: "minio-secret-key",
    bucket: "my-bucket",
    endpoint: "http://localhost:9000",
    publicBaseUrl: "http://localhost:9000/my-bucket",
});
```

### Upload Files

To upload files to an S3 bucket, use the `uploadFile` method:

```typescript
(async () => {
    const response = await bucket1.uploadFile({
        filename: "example.jpg",
        file: Buffer.from("Your file data"),
        contentType: "image/jpeg",
    });
    console.log("Uploaded File URL:", response.url);
    console.log("File Key:", response.key);
})();
```

Upload to a specific folder:

```typescript
const response = await bucket1.uploadFile({
    filename: "example.jpg",
    folder: "images",
    file: Buffer.from("Your file data"),
    contentType: "image/jpeg",
});
```

### Download Files

#### Direct File Download (Recommended for Server-Side)

Use the `getFile` method to download file content as a Buffer. This is ideal for server-side processing or when you need to stream files through your API server to avoid CORS issues:

```typescript
(async () => {
    const fileResponse = await bucket1.getFile({
        key: "example.jpg",
    });

    console.log("File content:", fileResponse.buffer);
    console.log("Content type:", fileResponse.contentType);
    console.log("File size:", fileResponse.contentLength);

    // Use in your API response
    // res.setHeader('Content-Type', fileResponse.contentType);
    // res.setHeader('Content-Disposition', 'attachment; filename="example.jpg"');
    // res.send(fileResponse.buffer);
})();
```

#### Streaming Large Files

For large files, use the `getFileStream` method to stream content efficiently:

```typescript
(async () => {
    const streamResponse = await bucket1.getFileStream({
        key: "large-file.zip",
    });

    console.log("Content type:", streamResponse.contentType);
    console.log("File size:", streamResponse.contentLength);

    // Pipe the stream to your response
    // streamResponse.stream.pipe(res);
})();
```

### Generate Signed URLs

#### Basic Signed URL

To generate signed URLs for private access to S3 objects:

```typescript
(async () => {
    const signedUrl = await bucket1.generateSignedUrl({
        key: "example.jpg",
        expiresIn: 3600, // URL valid for 1 hour
    });
    console.log("Signed URL:", signedUrl);
})();
```

#### Enhanced Signed URL with Download Headers

Generate signed URLs with download-specific headers to force browser download behavior and avoid CORS issues:

```typescript
(async () => {
    const signedUrl = await bucket1.generateSignedUrl({
        key: "document.pdf",
        expiresIn: 3600, // 1 hour
        responseContentDisposition: 'attachment; filename="document.pdf"',
        responseContentType: "application/pdf",
        responseCacheControl: "no-cache",
    });
    console.log("Download URL:", signedUrl);
})();
```

### Delete Files

#### Delete Single File

To delete a single file from an S3 bucket, use the `deleteFile` method:

```typescript
(async () => {
    await bucket1.deleteFile({
        key: "example.jpg",
    });
    console.log("File deleted successfully.");
})();
```

#### Bulk Delete Files

To delete multiple files efficiently, use the `deleteFiles` method. This automatically batches requests (S3 allows up to 1000 objects per request):

```typescript
(async () => {
    const result = await bucket1.deleteFiles({
        keys: ["file1.jpg", "file2.jpg", "file3.jpg", "file4.jpg"],
    });

    console.log("Deleted files:", result.deleted);
    if (result.errors.length > 0) {
        console.log("Errors:", result.errors);
    }
})();
```

The `deleteFiles` method returns an object with:

-   `deleted`: Array of successfully deleted keys
-   `errors`: Array of errors (if any) with `key`, `code`, and `message` properties

### Move File

To move a file within the same bucket (change its key/path):

```typescript
(async () => {
    const result = await bucket1.moveFile({
        sourceKey: "uploads/temp/photo.jpg",
        destinationKey: "uploads/processed/photo.jpg",
    });
    console.log("Moved to:", result.url);
})();
```

### Move File to Another Bucket

To move a file from one bucket to another:

```typescript
const sourceBucket = new S3Pilot({ region: "us-east-1", bucket: "source-bucket", ... });
const destBucket = new S3Pilot({ region: "us-east-1", bucket: "dest-bucket", ... });

(async () => {
    const result = await sourceBucket.moveToBucket({
        sourceKey: "uploads/photo.jpg",
        destinationKey: "archive/photo.jpg",
        destination: destBucket,
    });
    console.log("Moved to destination bucket:", result.url);
})();
```

This method:
- First attempts a direct S3 copy (fastest, works if same credentials have access to both buckets)
- Falls back to download + upload if direct copy fails (works across different AWS accounts)

### Delete Folder

To delete a folder and all its contents:

```typescript
(async () => {
    await bucket1.deleteFolder({
        folder: "images",
    });
    console.log("Folder and all contents deleted successfully.");
})();
```

## File Download Use Cases

### 1. Server-Side Download (Recommended)

When you need to control access permissions or avoid CORS issues:

```typescript
// In your API endpoint
app.get("/download/:fileKey", async (req, res) => {
    try {
        const fileResponse = await bucket1.getFile({
            key: req.params.fileKey,
        });

        res.setHeader("Content-Type", fileResponse.contentType || "application/octet-stream");
        res.setHeader("Content-Disposition", `attachment; filename="${req.params.fileKey}"`);
        res.setHeader("Content-Length", fileResponse.contentLength?.toString() || "0");

        res.send(fileResponse.buffer);
    } catch (error) {
        res.status(404).json({ error: "File not found" });
    }
});
```

### 2. Enhanced Signed URL for Direct Download

When you want the browser to download directly from S3:

```typescript
// Generate a signed URL that forces download
const downloadUrl = await bucket1.generateSignedUrl({
    key: fileKey,
    expiresIn: 3600, // 1 hour
    responseContentDisposition: `attachment; filename="${fileName}"`,
    responseContentType: fileType,
});

// Redirect user to download URL
res.redirect(downloadUrl);
```

### 3. Streaming Large Files

For files larger than 100MB:

```typescript
app.get("/stream/:fileKey", async (req, res) => {
    try {
        const streamResponse = await bucket1.getFileStream({
            key: req.params.fileKey,
        });

        res.setHeader("Content-Type", streamResponse.contentType || "application/octet-stream");
        res.setHeader("Content-Disposition", `attachment; filename="${req.params.fileKey}"`);
        res.setHeader("Content-Length", streamResponse.contentLength?.toString() || "0");

        streamResponse.stream.pipe(res);
    } catch (error) {
        res.status(404).json({ error: "File not found" });
    }
});
```

## Security Considerations

-   **Private Buckets**: Always use private S3 buckets for sensitive files
-   **Signed URLs**: Use short expiration times (1 hour max) for signed URLs
-   **Access Control**: Implement proper authentication before generating download URLs
-   **CORS**: Configure S3 bucket CORS settings if using direct signed URL downloads
-   **File Validation**: Implement file type and size validation in your application layer before uploading files

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.