oneie/one/things/plans/r2.md

Build apps, websites, and AI agents in English. Zero-interaction setup for AI agents (Claude Code, Cursor, Windsurf). Download to your computer, run in the cloud, deploy to the edge. Open source and free forever.

---
title: Cloudflare R2 Storage
dimension: things
category: plans
tags: storage, cloudflare, r2, video, audio, uploads
related_dimensions: connections, events, knowledge
scope: global
created: 2025-11-10
updated: 2025-11-10
version: 1.0.0
ai_context: |
  This document is part of the things dimension in the plans category.
  Location: one/things/plans/r2.md
  Purpose: Documents Cloudflare R2 storage implementation for video/audio uploads
  Related dimensions: connections, events, knowledge
  For AI agents: Read this to understand R2 storage integration.
---

# Cloudflare R2 Storage Implementation Plan

**Goal:** Implement Cloudflare R2 object storage for video and audio file uploads, leveraging the free tier (10GB storage, 1M uploads/month, unlimited free egress).

**Vision:** Enable creators to upload, manage, and serve video/audio content directly from the ONE Platform using Cloudflare R2 as the storage backend, with full integration into the 6-dimension ontology.

## Cloudflare R2 Free Tier Overview

### What's Included (Free)
- **Storage:** 10 GB/month
- **Class A Operations:** 1 million/month (uploads, lists, writes)
- **Class B Operations:** 10 million/month (downloads, reads)
- **Egress:** **FREE and UNLIMITED** (huge advantage over S3)
- **File Types:** No restrictions (videos, MP3s, images, any file type)

### Key Advantages
1. **Zero egress fees** - No bandwidth charges for downloads (S3 charges ~$0.09/GB)
2. **S3-compatible API** - Use existing S3 SDKs and tools
3. **Global CDN** - Cloudflare's edge network for fast delivery
4. **Simple pricing** - Only pay for storage and operations, never bandwidth

### When to Upgrade (Paid Tier)
- Storage >10GB: $0.015/GB-month (~$1.50 for 100GB)
- Class A >1M/month: $4.50 per million requests
- Class B >10M/month: $0.36 per million requests

**For most use cases, free tier is plenty to start.**

## 6-Dimension Ontology Mapping

### Things
- `media_file` - Uploaded video/audio files
  - Properties: `fileKey`, `fileName`, `fileSize`, `mimeType`, `duration`, `uploadedAt`
  - Metadata: `r2Bucket`, `r2Key`, `publicUrl`
- `media_folder` - Organization structure for files
  - Properties: `name`, `path`, `parentFolderId`
- `upload_session` - Track multi-part uploads
  - Properties: `status`, `bytesUploaded`, `totalBytes`, `resumeKey`

### Connections
- `creator` → `media_file` (relationshipType: "uploaded")
- `media_file` → `media_folder` (relationshipType: "stored_in")
- `course` → `media_file` (relationshipType: "contains_media")
- `post` → `media_file` (relationshipType: "attached_media")

### Events
- `media_uploaded` - File successfully uploaded to R2
- `media_deleted` - File removed from R2
- `media_viewed` - File accessed/streamed
- `upload_started` - Upload initiated
- `upload_failed` - Upload error occurred
- `upload_resumed` - Multi-part upload continued

### Knowledge
- Tags for categorization: `video`, `audio`, `podcast`, `course-content`, `tutorial`
- Vector embeddings for media content (transcriptions, descriptions)
- Search indexing for file discovery

### Groups
- All media files scoped by `groupId` (multi-tenant isolation)
- Organization-level storage quotas
- Team-based access control

### People
- **platform_owner** - Manage R2 configuration, view all uploads
- **org_owner** - Manage org storage, set quotas
- **org_user** - Upload files within quota
- **customer** - View/stream purchased content

## Technical Architecture

### Backend Layer (Convex + Effect.ts)

#### Service Layer (Effect.ts)

```typescript
// backend/convex/services/storage/r2.ts
export class R2Service extends Effect.Service<R2Service>()("R2Service", {
  effect: Effect.gen(function* () {
    const db = yield* ConvexDatabase;
    const config = yield* ConfigService; // R2 credentials

    return {
      // Core operations
      uploadFile: (args: UploadFileArgs) =>
        Effect.gen(function* () {
          // 1. Validate file (size, type)
          // 2. Generate unique R2 key
          // 3. Get presigned upload URL
          // 4. Return upload URL + metadata
        }),

      getPresignedUrl: (fileKey: string, expiresIn: number = 3600) =>
        Effect.gen(function* () {
          // Generate presigned URL for direct upload
          // Reduces server load (client → R2 directly)
        }),

      deleteFile: (fileKey: string) =>
        Effect.gen(function* () {
          // 1. Delete from R2
          // 2. Remove from database
          // 3. Log event
        }),

      listFiles: (prefix?: string) =>
        Effect.gen(function* () {
          // List files in bucket (with pagination)
        }),

      getFileUrl: (fileKey: string) =>
        Effect.gen(function* () {
          // Return public URL or presigned URL for private files
        }),

      // Multi-part upload for large files
      initiateMultipartUpload: (fileName: string) =>
        Effect.gen(function* () {
          /* ... */
        }),

      completeMultipartUpload: (uploadId: string, parts: Part[]) =>
        Effect.gen(function* () {
          /* ... */
        }),
    };
  }),
  dependencies: [ConvexDatabase.Default, ConfigService.Default],
}) {}
```

#### Convex Layer (Thin Wrappers)

```typescript
// backend/convex/mutations/storage.ts
export const requestUpload = confect.mutation({
  args: {
    fileName: v.string(),
    fileSize: v.number(),
    mimeType: v.string(),
    folderId: v.optional(v.id("things")),
  },
  handler: (ctx, args) =>
    Effect.gen(function* () {
      const r2Service = yield* R2Service;
      const identity = yield* getIdentity(ctx);

      // Check quota
      yield* checkStorageQuota(ctx, identity.groupId, args.fileSize);

      // Generate upload URL
      const uploadData = yield* r2Service.getPresignedUrl(args.fileName);

      // Create pending media_file record
      const fileId = yield* db.insert("things", {
        type: "media_file",
        groupId: identity.groupId,
        createdBy: identity.userId,
        metadata: {
          fileName: args.fileName,
          fileSize: args.fileSize,
          mimeType: args.mimeType,
          status: "uploading",
          r2Key: uploadData.key,
          uploadUrl: uploadData.url,
        },
      });

      // Log event
      yield* logEvent(ctx, {
        type: "upload_started",
        thingId: fileId,
      });

      return { fileId, uploadUrl: uploadData.url };
    }).pipe(Effect.provide(MainLayer)),
});

export const confirmUpload = confect.mutation({
  args: {
    fileId: v.id("things"),
  },
  handler: (ctx, args) =>
    Effect.gen(function* () {
      // Mark file as completed
      yield* db.patch(args.fileId, {
        "metadata.status": "completed",
        "metadata.uploadedAt": Date.now(),
      });

      // Log event
      yield* logEvent(ctx, {
        type: "media_uploaded",
        thingId: args.fileId,
      });

      return { success: true };
    }).pipe(Effect.provide(MainLayer)),
});

export const deleteFile = confect.mutation({
  args: {
    fileId: v.id("things"),
  },
  handler: (ctx, args) =>
    Effect.gen(function* () {
      const r2Service = yield* R2Service;

      // Get file
      const file = yield* db.get(args.fileId);
      if (!file || file.type !== "media_file") {
        return yield* Effect.fail(new Error("File not found"));
      }

      // Delete from R2
      yield* r2Service.deleteFile(file.metadata.r2Key);

      // Delete from database
      yield* db.delete(args.fileId);

      // Log event
      yield* logEvent(ctx, {
        type: "media_deleted",
        thingId: args.fileId,
      });

      return { success: true };
    }).pipe(Effect.provide(MainLayer)),
});
```

#### Queries

```typescript
// backend/convex/queries/storage.ts
export const listFiles = confect.query({
  args: {
    folderId: v.optional(v.id("things")),
    fileType: v.optional(v.union(v.literal("video"), v.literal("audio"))),
  },
  handler: (ctx, args) =>
    Effect.gen(function* () {
      const identity = yield* getIdentity(ctx);

      return yield* db
        .query("things")
        .withIndex("by_type", (q) =>
          q.eq("type", "media_file").eq("groupId", identity.groupId),
        )
        .filter((q) => {
          if (args.folderId) {
            return q.eq(q.field("metadata.folderId"), args.folderId);
          }
          if (args.fileType) {
            return q.eq(
              q.field("metadata.mimeType"),
              args.fileType === "video" ? "video/*" : "audio/*",
            );
          }
          return true;
        })
        .collect();
    }).pipe(Effect.provide(MainLayer)),
});

export const getFileUrl = confect.query({
  args: {
    fileId: v.id("things"),
  },
  handler: (ctx, args) =>
    Effect.gen(function* () {
      const r2Service = yield* R2Service;
      const file = yield* db.get(args.fileId);

      if (!file || file.type !== "media_file") {
        return yield* Effect.fail(new Error("File not found"));
      }

      const url = yield* r2Service.getFileUrl(file.metadata.r2Key);

      return { url, fileName: file.metadata.fileName };
    }).pipe(Effect.provide(MainLayer)),
});
```

### Frontend Layer (Astro + React)

#### Upload Component

```typescript
// web/src/components/features/storage/FileUploader.tsx
import { useMutation } from "convex/react";
import { api } from "@/convex/_generated/api";
import { Button } from "@/components/ui/button";
import { Progress } from "@/components/ui/progress";
import { Upload } from "lucide-react";
import { useState } from "react";

export function FileUploader({ folderId }: { folderId?: Id<"things"> }) {
  const requestUpload = useMutation(api.storage.requestUpload);
  const confirmUpload = useMutation(api.storage.confirmUpload);

  const [uploading, setUploading] = useState(false);
  const [progress, setProgress] = useState(0);

  const handleFileSelect = async (e: React.ChangeEvent<HTMLInputElement>) => {
    const file = e.target.files?.[0];
    if (!file) return;

    setUploading(true);
    setProgress(0);

    try {
      // 1. Request presigned upload URL
      const { fileId, uploadUrl } = await requestUpload({
        fileName: file.name,
        fileSize: file.size,
        mimeType: file.type,
        folderId,
      });

      // 2. Upload directly to R2
      const xhr = new XMLHttpRequest();
      xhr.upload.onprogress = (e) => {
        if (e.lengthComputable) {
          setProgress((e.loaded / e.total) * 100);
        }
      };

      await new Promise((resolve, reject) => {
        xhr.onload = () => (xhr.status === 200 ? resolve(null) : reject());
        xhr.onerror = reject;
        xhr.open("PUT", uploadUrl);
        xhr.setRequestHeader("Content-Type", file.type);
        xhr.send(file);
      });

      // 3. Confirm upload completion
      await confirmUpload({ fileId });

      toast.success(`${file.name} uploaded successfully!`);
    } catch (error) {
      console.error("Upload failed:", error);
      toast.error("Upload failed");
    } finally {
      setUploading(false);
      setProgress(0);
    }
  };

  return (
    <div className="space-y-4">
      <Button asChild disabled={uploading}>
        <label>
          <Upload className="mr-2 h-4 w-4" />
          {uploading ? "Uploading..." : "Upload File"}
          <input
            type="file"
            className="sr-only"
            accept="video/*,audio/*"
            onChange={handleFileSelect}
            disabled={uploading}
          />
        </label>
      </Button>

      {uploading && (
        <div className="space-y-2">
          <Progress value={progress} />
          <p className="text-sm text-muted-foreground">{progress.toFixed(0)}%</p>
        </div>
      )}
    </div>
  );
}
```

#### Media Library Component

```typescript
// web/src/components/features/storage/MediaLibrary.tsx
import { useQuery, useMutation } from "convex/react";
import { api } from "@/convex/_generated/api";
import { Button } from "@/components/ui/button";
import { Trash2, Play, Download } from "lucide-react";

export function MediaLibrary() {
  const files = useQuery(api.storage.listFiles);
  const deleteFile = useMutation(api.storage.deleteFile);
  const getUrl = useMutation(api.storage.getFileUrl);

  const handlePlay = async (fileId: Id<"things">) => {
    const { url } = await getUrl({ fileId });
    // Open in video/audio player
    window.open(url, "_blank");
  };

  const handleDelete = async (fileId: Id<"things">) => {
    if (confirm("Delete this file?")) {
      await deleteFile({ fileId });
      toast.success("File deleted");
    }
  };

  return (
    <div className="grid grid-cols-1 md:grid-cols-3 gap-4">
      {files?.map((file) => (
        <div key={file._id} className="border rounded-lg p-4 space-y-2">
          <h3 className="font-medium truncate">{file.metadata.fileName}</h3>
          <p className="text-sm text-muted-foreground">
            {formatBytes(file.metadata.fileSize)}
          </p>

          <div className="flex gap-2">
            <Button size="sm" onClick={() => handlePlay(file._id)}>
              <Play className="h-4 w-4" />
            </Button>
            <Button
              size="sm"
              variant="destructive"
              onClick={() => handleDelete(file._id)}
            >
              <Trash2 className="h-4 w-4" />
            </Button>
          </div>
        </div>
      ))}
    </div>
  );
}
```

## File Structure

### Backend

```
backend/convex/
├── services/
│   └── storage/
│       ├── r2.ts              # R2 service (Effect.ts)
│       ├── quota.ts           # Storage quota management
│       └── multipart.ts       # Multi-part upload service
├── mutations/
│   └── storage.ts             # Upload, delete mutations
├── queries/
│   └── storage.ts             # List, get file queries
├── actions/
│   └── storage.ts             # R2 API interactions
└── http/
    └── webhooks/
        └── r2.ts              # R2 event notifications (optional)
```

### Frontend

```
web/src/
├── components/
│   └── features/
│       └── storage/
│           ├── FileUploader.tsx       # Upload component
│           ├── MediaLibrary.tsx       # File browser
│           ├── VideoPlayer.tsx        # Video player
│           ├── AudioPlayer.tsx        # Audio player
│           ├── FolderTree.tsx         # Folder navigation
│           └── StorageQuota.tsx       # Quota display
└── pages/
    └── media/
        ├── index.astro               # Media library page
        ├── upload.astro              # Upload page
        └── [id].astro                # File detail page
```

## Configuration

### Environment Variables

```bash
# .env.local (backend)
R2_ACCOUNT_ID=your_account_id
R2_ACCESS_KEY_ID=your_access_key
R2_SECRET_ACCESS_KEY=your_secret_key
R2_BUCKET_NAME=one-platform-media
R2_PUBLIC_URL=https://media.one.ie  # Custom domain (optional)
```

### R2 Setup Steps

1. **Create R2 Bucket**
   - Go to Cloudflare Dashboard → R2
   - Create bucket: `one-platform-media`
   - Enable public access (or use presigned URLs)

2. **Generate API Tokens**
   - R2 → Manage R2 API Tokens
   - Create token with read/write permissions
   - Copy Account ID, Access Key, Secret Key

3. **Custom Domain (Optional)**
   - Connect custom domain: `media.one.ie`
   - Add CNAME: `media.one.ie` → `<bucket>.r2.cloudflarestorage.com`
   - Free SSL via Cloudflare

4. **CORS Configuration**
   ```json
   [
     {
       "AllowedOrigins": ["https://one.ie", "http://localhost:4321"],
       "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
       "AllowedHeaders": ["*"],
       "ExposeHeaders": ["ETag"],
       "MaxAgeSeconds": 3600
     }
   ]
   ```

## Implementation Phases

### Phase 1: Basic Upload (Week 1)

**Goal:** Single file upload to R2 with presigned URLs

- [ ] Set up R2 bucket and credentials
- [ ] Create `R2Service` (Effect.ts)
- [ ] Implement `requestUpload` mutation
- [ ] Build `FileUploader` component
- [ ] Test video/audio uploads
- [ ] Verify files in R2 dashboard

**Success Metrics:**
- Upload video files <100MB
- Upload audio files <50MB
- 95%+ upload success rate

### Phase 2: Media Library (Week 2)

**Goal:** Browse, play, and delete uploaded files

- [ ] Create `listFiles` query
- [ ] Build `MediaLibrary` component
- [ ] Implement file deletion
- [ ] Add video/audio preview
- [ ] Create folder organization

**Success Metrics:**
- List 100+ files without lag
- Preview videos in-browser
- Organize files in folders

### Phase 3: Advanced Features (Week 3)

**Goal:** Multi-part uploads, quotas, analytics

- [ ] Multi-part upload for large files (>100MB)
- [ ] Storage quota per organization
- [ ] Usage analytics (storage, bandwidth)
- [ ] Resumable uploads (pause/resume)
- [ ] Thumbnail generation (via Cloudflare Images)

**Success Metrics:**
- Upload files >1GB
- Enforce storage quotas
- Track bandwidth usage
- Resume interrupted uploads

### Phase 4: Streaming & CDN (Week 4)

**Goal:** Optimized video/audio delivery

- [ ] HLS/DASH streaming support
- [ ] Adaptive bitrate streaming
- [ ] CDN caching optimization
- [ ] Download acceleration
- [ ] Mobile-optimized delivery

**Success Metrics:**
- Stream 4K video smoothly
- <2s initial playback time
- Cache hit rate >80%

## Integration with ONE Platform Features

### Course Content
- Upload video lessons to R2
- Attach audio files to course modules
- Stream content to enrolled students
- Track video completion (events)

### Podcasts
- Upload podcast episodes (MP3/M4A)
- Generate RSS feed with R2 URLs
- Track listens and downloads
- Auto-transcription (Whisper API)

### Creator Content
- Upload content for sale
- Protect files with presigned URLs
- Track views per customer
- Revenue attribution per file

### Marketing Assets
- Store product demo videos
- Host webinar recordings
- Share promotional audio
- Track engagement metrics

## Cost Optimization

### Free Tier Strategy
- Compress videos before upload (H.264, medium quality)
- Use MP3 128kbps for audio (vs. lossless)
- Delete unused files regularly
- Archive old content to cheaper storage

### Monitoring
- Track storage usage daily
- Alert at 80% quota (8GB)
- Monitor Class A operations
- Estimate monthly costs

### Scaling Beyond Free Tier
- At 10GB: Upgrade to paid ($1.50/month for 100GB)
- At 1M uploads/month: Review upload patterns
- Consider Cloudflare Stream for video (separate product)

## Security Considerations

1. **Access Control**
   - Use presigned URLs (expiring tokens)
   - Validate file types server-side
   - Scan for malware (optional: VirusTotal API)
   - Rate limit uploads per user

2. **Data Privacy**
   - Encrypt sensitive files (AES-256)
   - GDPR compliance (delete on request)
   - Private files use presigned URLs only
   - Audit access logs

3. **Quota Enforcement**
   - Check quota before upload
   - Block uploads if exceeded
   - Notify users at 80% usage
   - Auto-delete old files (configurable)

4. **File Validation**
   - Max file size: 5GB
   - Allowed types: video/*, audio/*
   - Sanitize file names
   - Virus scanning (optional)

## Testing Strategy

### Unit Tests
```typescript
// tests/unit/services/r2.test.ts
describe("R2Service", () => {
  it("should generate presigned upload URL", async () => {
    const result = await Effect.runPromise(
      Effect.gen(function* () {
        const service = yield* R2Service;
        return yield* service.getPresignedUrl("test-video.mp4");
      }).pipe(Effect.provide(TestLayer)),
    );

    expect(result.url).toContain("https://");
    expect(result.key).toMatch(/^media\/.+\.mp4$/);
  });

  it("should enforce storage quota", async () => {
    // Test that upload fails when quota exceeded
  });
});
```

### Integration Tests
```typescript
// tests/integration/upload-flow.test.ts
describe("Upload Flow", () => {
  it("should complete full upload lifecycle", async () => {
    // 1. Request upload URL
    // 2. Upload file to R2
    // 3. Confirm upload
    // 4. Verify file in database
    // 5. Verify file in R2
    // 6. Delete file
  });
});
```

### E2E Tests
- Upload 10MB video file
- Upload 5MB audio file
- Delete uploaded file
- Exceed storage quota (fails gracefully)
- Play uploaded video in browser

## Migration Path

### From Current State
1. ✅ No existing storage system (greenfield)
2. ⏭️ Set up R2 bucket
3. ⏭️ Create R2Service
4. ⏭️ Build upload UI
5. ⏭️ Add media library

### Future Enhancements
- Cloudflare Stream (video transcoding, $1/1000 minutes)
- Cloudflare Images (thumbnail generation, $5/100k images)
- AI transcription (Whisper API)
- Auto-captioning for videos
- Content moderation (AI)

## Resources

**Documentation:**
- [Cloudflare R2 Docs](https://developers.cloudflare.com/r2/)
- [R2 API Reference](https://developers.cloudflare.com/r2/api/s3/)
- [AWS S3 SDK](https://aws.amazon.com/sdk-for-javascript/) (S3-compatible)

**Tools:**
- [s3cmd](https://s3tools.org/s3cmd) - Command-line R2 client
- [Cyberduck](https://cyberduck.io/) - GUI R2 browser
- [Rclone](https://rclone.org/) - File sync to R2

**Examples:**
- `backend/convex/services/` - Effect.ts service patterns
- `web/src/components/ui/` - shadcn/ui components

## Success Metrics

### Phase 1: Basic Upload
- [ ] Upload video files successfully
- [ ] Upload audio files successfully
- [ ] Files accessible via public URL
- [ ] 95%+ upload success rate

### Phase 2: Media Library
- [ ] Browse uploaded files
- [ ] Play videos in browser
- [ ] Delete files
- [ ] Organize in folders

### Phase 3: Advanced Features
- [ ] Upload files >1GB (multi-part)
- [ ] Enforce storage quotas
- [ ] Resume interrupted uploads
- [ ] Track usage analytics

### Phase 4: Streaming & CDN
- [ ] Stream 4K video smoothly
- [ ] Adaptive bitrate working
- [ ] Cache hit rate >80%
- [ ] <2s initial playback time

---

**Success = Production-ready media storage with 10GB free tier, unlimited bandwidth, and seamless integration with the ONE Platform's 6-dimension ontology.**