@d3oxy/s3-pilot/MIGRATION_GUIDE.md

A TypeScript wrapper for AWS S3 and S3-compatible services (R2, MinIO, DigitalOcean Spaces) with a simplified single-client, single-bucket architecture.

# S3 Pilot Migration Guide

This guide helps you migrate between different versions of S3 Pilot.

## Migrating from v3.x to v4.x

### Overview of Breaking Changes

S3 Pilot v4.x introduces **S3-compatible service support** and a **project restructure**:

-   **New `endpoint` config option**: Specify custom S3-compatible endpoints (R2, MinIO, Spaces)
-   **New `publicBaseUrl` config option**: Specify the base URL for generating public URLs
-   **`getUrl()` is now synchronous**: Returns a string directly instead of a Promise
-   **REMOVED: `getKeyFromUrl()` method**: This method has been removed. Parse URLs in your application layer if needed.
-   **Project restructure**: Source moved to `src/`, build output to `dist/`

### Migration Steps

#### Step 1: Update Method Calls for `getUrl()`

The `getUrl()` method is now **synchronous** and returns a string directly.

**Before (v3.x):**

```typescript
const url = await bucket1.getUrl("test.jpg");
```

**After (v4.x):**

```typescript
const url = bucket1.getUrl("test.jpg");
```

#### Step 2: Remove `getKeyFromUrl()` Usage

The `getKeyFromUrl()` method has been removed. If you need to extract keys from URLs, implement this in your application layer.

**Before (v3.x):**

```typescript
const key = await bucket1.getKeyFromUrl("https://bucket.s3.region.amazonaws.com/folder/file.jpg");
```

**After (v4.x):**

```typescript
// Implement URL parsing in your application layer
function getKeyFromUrl(url: string): string {
    const parsedUrl = new URL(url);
    return decodeURIComponent(parsedUrl.pathname.substring(1)); // Remove leading '/'
}

const key = getKeyFromUrl("https://bucket.s3.region.amazonaws.com/folder/file.jpg");
```

#### Step 3: Add S3-Compatible Service Support (Optional)

If you're using S3-compatible services like Cloudflare R2, DigitalOcean Spaces, or MinIO, you can now use the new `endpoint` and `publicBaseUrl` options.

**Before (v3.x) - Not possible without workarounds:**

```typescript
const bucket = new S3Pilot({
    region: "us-east-1",
    accessKeyId: "...",
    secretAccessKey: "...",
    bucket: "my-bucket",
    // Had to use additionalConfig for endpoint
    additionalConfig: {
        endpoint: "https://account.r2.cloudflarestorage.com",
    },
});
// getUrl() would still return AWS-style URLs
```

**After (v4.x) - Native S3-compatible support:**

```typescript
const bucket = new S3Pilot({
    region: "auto", // R2 uses "auto"
    accessKeyId: "...",
    secretAccessKey: "...",
    bucket: "my-bucket",
    endpoint: "https://account.r2.cloudflarestorage.com",
    publicBaseUrl: "https://pub-xxx.r2.dev", // Your public URL
});
// getUrl() now returns correct R2 URLs
const url = bucket.getUrl("test.jpg"); // https://pub-xxx.r2.dev/test.jpg
```

### Benefits of v4.x

1. **S3-Compatible Services**: Native support for Cloudflare R2, DigitalOcean Spaces, MinIO
2. **Cleaner API**: `getUrl()` is now synchronous (no unnecessary Promise)
3. **Flexible URL Generation**: Use `publicBaseUrl` for custom domains and CDNs
4. **Modern Project Structure**: Standard `src/` → `dist/` build pattern

---

## Migrating from v2.x to v3.x

### Overview of Breaking Changes

S3 Pilot v3.x introduces a **major API simplification**:

-   **Single-client, single-bucket architecture**: Each `S3Pilot` instance now represents one S3 client connected to exactly one bucket
-   **Removed multi-client management**: No more `clientName` parameters
-   **Removed bucket parameters**: Bucket is specified at initialization, not in each method call
-   **New bulk delete method**: `deleteFiles()` for efficient deletion of multiple files
-   **Simplified `moveFile()` method**: Now moves files within the same bucket
-   **New `moveToBucket()` method**: Moves files between buckets with automatic fallback for cross-account scenarios
-   **Removed extension validation**: File extension validation has been removed. You can upload files with any extension. Implement validation in your application layer if needed.

### Migration Steps

#### Step 1: Update Imports

**Before (v2.x):**

```typescript
import { S3ClientSettings, S3ClientsSetup, S3Pilot } from "@d3oxy/s3-pilot";
```

**After (v3.x):**

```typescript
import { S3Pilot } from "@d3oxy/s3-pilot";
```

#### Step 2: Update Initialization

**Before (v2.x):**

```typescript
const s3Pilot = new S3Pilot<
    S3ClientsSetup<{
        client1: S3ClientSettings<"bucket1" | "bucket2">;
        client2: S3ClientSettings<"bucket3">;
    }>
>({
    client1: {
        region: "us-east-1",
        accessKeyId: "...",
        secretAccessKey: "...",
        buckets: ["bucket1", "bucket2"],
    },
    client2: {
        region: "us-west-2",
        accessKeyId: "...",
        secretAccessKey: "...",
        buckets: ["bucket3"],
    },
});
```

**After (v3.x):**

```typescript
// Create one instance per bucket
const bucket1 = new S3Pilot({
    region: "us-east-1",
    accessKeyId: "...",
    secretAccessKey: "...",
    bucket: "bucket1",
});

const bucket2 = new S3Pilot({
    region: "us-east-1",
    accessKeyId: "...",
    secretAccessKey: "...",
    bucket: "bucket2",
});

const bucket3 = new S3Pilot({
    region: "us-west-2",
    accessKeyId: "...",
    secretAccessKey: "...",
    bucket: "bucket3",
});
```

#### Step 3: Update Method Calls

All methods now remove the `clientName` and `bucket` parameters:

**Before (v2.x):**

```typescript
// Upload
await s3Pilot.uploadFile("client1", "bucket1", {
    filename: "test.jpg",
    file: buffer,
    contentType: "image/jpeg",
});

// Download
const file = await s3Pilot.getFile("client1", "bucket1", {
    key: "test.jpg",
});

// Delete
await s3Pilot.deleteFile("client1", "bucket1", {
    key: "test.jpg",
});

// Generate signed URL
const url = await s3Pilot.generateSignedUrl("client1", "bucket1", {
    key: "test.jpg",
    expiresIn: 3600,
});
```

**After (v3.x):**

```typescript
// Upload
await bucket1.uploadFile({
    filename: "test.jpg",
    file: buffer,
    contentType: "image/jpeg",
});

// Download
const file = await bucket1.getFile({
    key: "test.jpg",
});

// Delete
await bucket1.deleteFile({
    key: "test.jpg",
});

// Generate signed URL
const url = await bucket1.generateSignedUrl({
    key: "test.jpg",
    expiresIn: 3600,
});
```

#### Step 4: Update Bulk Delete (New Feature)

**Before (v2.x):**

```typescript
// Had to delete files one by one
for (const key of keys) {
    await s3Pilot.deleteFile("client1", "bucket1", { key });
}
```

**After (v3.x):**

```typescript
// New bulk delete method
const result = await bucket1.deleteFiles({
    keys: ["file1.jpg", "file2.jpg", "file3.jpg"],
});

console.log("Deleted:", result.deleted);
if (result.errors.length > 0) {
    console.log("Errors:", result.errors);
}
```

#### Step 5: Handle Updated `moveFile()` Method

The `moveFile()` method now only supports **same-bucket moves**. For cross-bucket moves, use copy + delete.

**Before (v2.x) - Cross-bucket move:**

```typescript
await s3Pilot.moveFile("client1", "bucket1", "bucket2", {
    sourceKey: "old-key.jpg",
    destinationKey: "new-key.jpg",
});
```

**After (v3.x) - Same-bucket move (supported):**

```typescript
// Move within the same bucket - simplified API
await bucket1.moveFile({
    sourceKey: "uploads/temp/photo.jpg",
    destinationKey: "uploads/processed/photo.jpg",
});
```

**After (v3.x) - Cross-bucket move (new method):**

```typescript
const sourceBucket = new S3Pilot({ ...config1, bucket: "bucket1" });
const destBucket = new S3Pilot({ ...config2, bucket: "bucket2" });

// Use the new moveToBucket method
await sourceBucket.moveToBucket({
    sourceKey: "old-key.jpg",
    destinationKey: "new-key.jpg",
    destination: destBucket,
});
```

The `moveToBucket` method automatically:
- Tries direct S3 copy first (fastest, works if same credentials)
- Falls back to download + upload if direct copy fails (cross-account support)

#### Step 6: Handle Removed Extension Validation

**Before (v2.x):**

```typescript
const s3Pilot = new S3Pilot({
    client1: {
        region: "us-east-1",
        accessKeyId: "...",
        secretAccessKey: "...",
        buckets: ["bucket1"],
        enableDefaultAllowedExtensions: true,
        // or
        allowedExtensions: ["jpg", "png", "pdf"],
    },
});

// Extension validation was automatic
await s3Pilot.uploadFile("client1", "bucket1", {
    filename: "test.jpg", // Would validate extension
    file: buffer,
    contentType: "image/jpeg",
});
```

**After (v3.x):**

```typescript
const bucket1 = new S3Pilot({
    region: "us-east-1",
    accessKeyId: "...",
    secretAccessKey: "...",
    bucket: "bucket1",
    // Extension validation config options removed
});

// No extension validation - implement in your application layer
const allowedExtensions = ["jpg", "png", "pdf"];
const extension = filename.split(".").pop();

if (!extension || !allowedExtensions.includes(extension.toLowerCase())) {
    throw new Error("Invalid file extension");
}

await bucket1.uploadFile({
    filename: "test.jpg",
    file: buffer,
    contentType: "image/jpeg",
});
```

### Complete Migration Example

**Before (v2.x):**

```typescript
import { S3ClientSettings, S3ClientsSetup, S3Pilot } from "@d3oxy/s3-pilot";

const s3 = new S3Pilot<
    S3ClientsSetup<{
        main: S3ClientSettings<"uploads" | "downloads">;
    }>
>({
    main: {
        region: "us-east-1",
        accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
        buckets: ["uploads", "downloads"],
    },
});

// Upload to uploads bucket
await s3.uploadFile("main", "uploads", {
    filename: "photo.jpg",
    file: buffer,
    contentType: "image/jpeg",
});

// Download from downloads bucket
const file = await s3.getFile("main", "downloads", {
    key: "document.pdf",
});
```

**After (v3.x):**

```typescript
import { S3Pilot } from "@d3oxy/s3-pilot";

const uploadsBucket = new S3Pilot({
    region: "us-east-1",
    accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
    bucket: "uploads",
});

const downloadsBucket = new S3Pilot({
    region: "us-east-1",
    accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
    bucket: "downloads",
});

// Upload to uploads bucket
await uploadsBucket.uploadFile({
    filename: "photo.jpg",
    file: buffer,
    contentType: "image/jpeg",
});

// Download from downloads bucket
const file = await downloadsBucket.getFile({
    key: "document.pdf",
});
```

### Benefits of v3.x

1. **Simpler API**: No need to pass client and bucket names to every method
2. **Better Type Safety**: Each instance is strongly typed to its specific bucket
3. **Cleaner Code**: Less boilerplate, more readable
4. **Bulk Operations**: Efficient bulk delete support
5. **Better Performance**: Simpler internal structure reduces overhead

### Common Patterns

#### Pattern 1: Multiple Buckets with Same Credentials

```typescript
const createBucket = (bucketName: string) =>
    new S3Pilot({
        region: "us-east-1",
        accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
        bucket: bucketName,
    });

const uploads = createBucket("uploads");
const downloads = createBucket("downloads");
const temp = createBucket("temp");
```

#### Pattern 2: Factory Function for Reusability

```typescript
const createS3Pilot = (config: { region: string; bucket: string }) =>
    new S3Pilot({
        ...config,
        accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
    });

const bucket1 = createS3Pilot({ region: "us-east-1", bucket: "bucket1" });
const bucket2 = createS3Pilot({ region: "us-west-2", bucket: "bucket2" });
```

---

## Migrating from v2.4.x to v2.5.1+ (File Download Features)

This section covers migration for file download features added in v2.5.1+.

### Overview of Changes

### New Methods Added

1. **`getFile()`** - Download file content as Buffer (recommended for server-side)
2. **`getFileStream()`** - Download file content as ReadableStream (for large files)
3. **Enhanced `generateSignedUrl()`** - Support for download-specific headers

### Breaking Changes

-   `generateSignedUrl()` default expiration changed from 60 seconds to 3600 seconds (1 hour)

### Migration Scenarios

### Scenario 1: CORS Issues with Signed URLs

**Before (Causing CORS Issues):**

```typescript
// Current approach - causes CORS errors in browser
const signedUrl = await s3Pilot.generateSignedUrl("main", bucketName, {
    key: fileKey,
    expiresIn: 3600,
});

// Frontend tries to fetch this URL but gets CORS error
const response = await fetch(signedUrl);
```

**After (Recommended Solution):**

```typescript
// Option 1: Server-side download (recommended)
const fileResponse = await s3Pilot.getFile("main", bucketName, {
    key: fileKey,
});

// Stream file to client with proper headers
res.setHeader("Content-Type", fileResponse.contentType || "application/octet-stream");
res.setHeader("Content-Disposition", `attachment; filename="${fileName}"`);
res.send(fileResponse.buffer);

// Option 2: Enhanced signed URL with download headers
const signedUrl = await s3Pilot.generateSignedUrl("main", bucketName, {
    key: fileKey,
    expiresIn: 3600,
    responseContentDisposition: `attachment; filename="${fileName}"`,
    responseContentType: fileType,
});
```

### Scenario 2: Large File Downloads

**Before (Memory Issues):**

```typescript
// Downloading large files as buffer can cause memory issues
const fileResponse = await s3Pilot.getFile("main", bucketName, {
    key: largeFileKey,
});
```

**After (Streaming for Large Files):**

```typescript
// Use streaming for files larger than 100MB
const streamResponse = await s3Pilot.getFileStream("main", bucketName, {
    key: largeFileKey,
});

// Pipe stream directly to response
res.setHeader("Content-Type", streamResponse.contentType || "application/octet-stream");
res.setHeader("Content-Disposition", `attachment; filename="${fileName}"`);
streamResponse.stream.pipe(res);
```

### API Endpoint Migration Examples

### Express.js/Hono.js Migration

**Before:**

```typescript
app.get("/download/:fileKey", async (req, res) => {
    try {
        const signedUrl = await s3Pilot.generateSignedUrl("main", bucketName, {
            key: req.params.fileKey,
            expiresIn: 3600,
        });

        // This can cause CORS issues
        res.redirect(signedUrl);
    } catch (error) {
        res.status(404).json({ error: "File not found" });
    }
});
```

**After (Recommended):**

```typescript
app.get("/download/:fileKey", async (req, res) => {
    try {
        const fileResponse = await s3Pilot.getFile("main", bucketName, {
            key: req.params.fileKey,
        });

        res.setHeader("Content-Type", fileResponse.contentType || "application/octet-stream");
        res.setHeader("Content-Disposition", `attachment; filename="${req.params.fileKey}"`);
        res.setHeader("Content-Length", fileResponse.contentLength?.toString() || "0");

        res.send(fileResponse.buffer);
    } catch (error) {
        res.status(404).json({ error: "File not found" });
    }
});
```

## Version Compatibility

-   **v4.0.0+**: S3-compatible services support (R2, MinIO, Spaces), sync `getUrl()`, removed `getKeyFromUrl()`
-   **v3.0.0+**: Simplified single-client, single-bucket API
-   **v2.5.1+**: File download methods available (with multi-client API)
-   **v2.4.x and below**: Only basic signed URLs available

## Support

If you encounter issues during migration:

1. Check the [README](./README.md) for detailed documentation
2. Review the examples in the README
3. Open an issue on GitHub with your specific use case