@firesystem/indexeddb/README.md

IndexedDB implementation of Virtual File System

# @firesystem/indexeddb

IndexedDB implementation of the Virtual File System (VFS) interface. Provides persistent storage in web browsers with an intuitive file system API.

## Features

- 🌐 **Browser Native** - Uses IndexedDB for persistent storage
- 📦 **Large Storage** - Store gigabytes of data (browser limits apply)
- 🔄 **Full VFS API** - Implements complete @firesystem/core interface
- 🚀 **Async Performance** - Non-blocking operations
- 🔍 **Indexing Support** - Fast file lookups and queries
- 💾 **Persistent** - Data survives browser restarts
- ⚡ **Native Events** - Built-in reactive event system for all operations
- 📊 **Progress Tracking** - Monitor initialization and file loading
- 🔔 **Real-time Updates** - Get notified of all file system changes
- 🔌 **Workspace Ready** - Optional integration with @workspace-fs/core
- 🔄 **Multi-tab Sync** - Changes sync across browser tabs

## Installation

```bash
npm install @firesystem/indexeddb
# or
yarn add @firesystem/indexeddb
# or
pnpm add @firesystem/indexeddb
```

## Usage

```typescript
import { IndexedDBFileSystem } from "@firesystem/indexeddb";

// Create a new file system instance
const fs = new IndexedDBFileSystem({
  dbName: "my-app-storage", // Optional, defaults to "vfs"
  version: 1, // Optional, defaults to 1
});

// Use it like any other VFS implementation
await fs.writeFile("/data.json", { message: "Hello, IndexedDB!" });
const file = await fs.readFile("/data.json");
console.log(file.content); // { message: "Hello, IndexedDB!" }

// NEW: Native event support
fs.events.on("file:written", ({ path, size }) => {
  console.log(`File ${path} saved to IndexedDB (${size} bytes)`);
});
```

## Configuration Options

```typescript
interface IndexedDBConfig {
  dbName?: string; // Database name (default: "vfs")
  version?: number; // Database version (default: 1)
  storeName?: string; // Object store name (default: "files")
  enableExternalSync?: boolean; // Enable cross-tab sync and external change detection (default: true)
  deepChangeDetection?: boolean; // Check content even if date hasn't changed (default: false)
}

const fs = new IndexedDBFileSystem({
  dbName: "my-app",
  version: 1,
  storeName: "documents",
  enableExternalSync: true, // Enable external change detection
});
```

## Event System

IndexedDBFileSystem includes a native event system that emits events for all operations:

### Basic Events

```typescript
import { IndexedDBFileSystem, FileSystemEvents } from "@firesystem/indexeddb";

const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Listen to file operations
fs.events.on(FileSystemEvents.FILE_WRITTEN, ({ path, size }) => {
  console.log(`File written: ${path} (${size} bytes)`);
});

fs.events.on(FileSystemEvents.FILE_DELETED, ({ path }) => {
  console.log(`File deleted: ${path}`);
});

// Listen to directory operations
fs.events.on(FileSystemEvents.DIR_CREATED, ({ path }) => {
  console.log(`Directory created: ${path}`);
});

// Operation tracking with timing
fs.events.on(FileSystemEvents.OPERATION_END, ({ operation, duration }) => {
  console.log(`${operation} completed in ${duration}ms`);
});
```

### Initialization with Progress

Load all existing files from IndexedDB with progress tracking:

```typescript
const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Track initialization progress
fs.events.on(FileSystemEvents.INIT_PROGRESS, ({ loaded, total, phase }) => {
  console.log(`${phase}: ${loaded}/${total} files loaded`);
});

// Get notified of each file during initialization
fs.events.on(FileSystemEvents.FILE_READ, ({ path, size }) => {
  console.log(`Loaded: ${path} (${size} bytes)`);
});

fs.events.on(FileSystemEvents.DIR_READ, ({ path }) => {
  console.log(`Found directory: ${path}`);
});

// Initialize and load all existing files
await fs.initialize();
```

### Integration with State Management

```typescript
// Example with React and Zustand
import { create } from 'zustand';
import { IndexedDBFileSystem, FileSystemEvents } from '@firesystem/indexeddb';

const useFileStore = create((set) => ({
  files: [],
  isLoading: true,
  progress: { loaded: 0, total: 0 },

  initializeFS: async () => {
    const fs = new IndexedDBFileSystem({ dbName: "my-app" });

    // Update progress
    fs.events.on(FileSystemEvents.INIT_PROGRESS, ({ loaded, total }) => {
      set({ progress: { loaded, total } });
    });

    // Populate files as they load
    fs.events.on(FileSystemEvents.FILE_READ, ({ path, size }) => {
      set(state => ({
        files: [...state.files, { path, size, type: 'file' }]
      }));
    });

    fs.events.on(FileSystemEvents.DIR_READ, ({ path }) => {
      set(state => ({
        files: [...state.files, { path, type: 'directory' }]
      }));
    });

    // Mark as loaded when done
    fs.events.on(FileSystemEvents.INITIALIZED, () => {
      set({ isLoading: false });
    });

    // Initialize and load all files from IndexedDB
    await fs.initialize();
  }
}));

// In your component
function FileExplorer() {
  const { files, isLoading, progress, initializeFS } = useFileStore();

  useEffect(() => {
    initializeFS(); // Load all files from IndexedDB
  }, []);

  if (isLoading) {
    return <div>Loading files: {progress.loaded}/{progress.total}</div>;
  }

  return <FileTree files={files} />;
}
```

### Storage Events

```typescript
// Monitor storage operations
fs.events.on(FileSystemEvents.STORAGE_CLEARING, () => {
  console.log("Clearing all files...");
});

fs.events.on(FileSystemEvents.STORAGE_CLEARED, ({ duration }) => {
  console.log(`Storage cleared in ${duration}ms`);
});

fs.events.on(FileSystemEvents.STORAGE_SIZE_CALCULATED, ({ size }) => {
  console.log(`Total storage used: ${size} bytes`);
});
```

## Workspace Integration

IndexedDBFileSystem can be integrated with `@workspace-fs/core` through the `IndexedDBWorkspaceProvider`:

### Standalone Usage (Default)

```typescript
import { IndexedDBFileSystem } from "@firesystem/indexeddb";

// Use directly without workspace
const fs = new IndexedDBFileSystem({ dbName: "my-app" });
await fs.initialize();
await fs.writeFile("/data.json", { message: "Hello!" });
```

### With Workspace

```typescript
import { WorkspaceFileSystem } from "@workspace-fs/core";
import { indexedDBProvider } from "@firesystem/indexeddb/provider";

// Register the provider
const workspace = new WorkspaceFileSystem();
workspace.registerProvider(indexedDBProvider);

// Create persistent projects
const project = await workspace.loadProject({
  id: "my-project",
  name: "My Project",
  source: {
    type: "indexeddb",
    config: {
      dbName: "project-storage",
      version: 1,
      enableExternalSync: true,
    },
  },
});

// Work with the project - data persists across sessions
await workspace.writeFile("/src/app.js", "// Application code");
await workspace.writeFile("/data/users.json", JSON.stringify([]));

// Switch between multiple persistent projects
const devProject = await workspace.loadProject({
  id: "dev",
  name: "Development",
  source: { type: "indexeddb", config: { dbName: "dev-db" } },
});

const prodProject = await workspace.loadProject({
  id: "prod",
  name: "Production",
  source: { type: "indexeddb", config: { dbName: "prod-db" } },
});

await workspace.setActiveProject("dev");
```

### Provider Configuration

The IndexedDB provider accepts all standard IndexedDBFileSystem configuration options:

```typescript
interface IndexedDBConfig {
  // Database name (required for persistence)
  dbName?: string;

  // Database version (default: 1)
  version?: number;

  // Object store name (default: "files")
  storeName?: string;

  // Enable cross-tab sync (default: true)
  enableExternalSync?: boolean;

  // Deep change detection (default: false)
  deepChangeDetection?: boolean;
}
```

## External Change Detection

IndexedDBFileSystem can detect changes made to the database from external sources (other tabs, windows, or direct IndexedDB access):

### Cross-Tab Synchronization

Changes made in one tab are automatically synchronized to other tabs using BroadcastChannel:

```typescript
// Tab 1
const fs1 = new IndexedDBFileSystem({ dbName: "my-app" });
fs1.watch("**/*", (event) => {
  console.log(`Change detected: ${event.type} ${event.path}`);
});

// Tab 2
const fs2 = new IndexedDBFileSystem({ dbName: "my-app" });
await fs2.writeFile("/data.json", { updated: true });
// Tab 1 will receive the change notification automatically
```

### Polling for External Changes

To detect changes made directly to IndexedDB (outside of FileSystem API):

```typescript
const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Start watching for external changes (default: 250ms polling)
await fs.startWatching(); // Poll every 250ms
// or with custom interval
await fs.startWatching(1000); // Poll every 1 second

// Your watch handlers will now receive notifications for external changes
fs.watch("**/*", (event) => {
  console.log(`External change: ${event.type} ${event.path}`);
});

// Stop watching when done
await fs.stopWatching();
```

### Deep Change Detection

By default, changes are detected by file modification date and size. To detect content changes even when date/size remain the same:

```typescript
const fs = new IndexedDBFileSystem({
  dbName: "my-app",
  deepChangeDetection: true, // Enable content hash comparison
});

// Now changes will be detected even if:
// - File content changes but date doesn't update
// - Metadata changes but file size remains same
await fs.startWatching();
```

**Note:** Deep change detection has performance implications as it requires calculating hashes for all file contents. Use it only when necessary.

### Disable External Sync

If you don't need external change detection:

```typescript
const fs = new IndexedDBFileSystem({
  dbName: "my-app",
  enableExternalSync: false, // Disable all external sync features
});
```

## Browser Storage Limits

IndexedDB storage limits vary by browser:

- **Chrome/Edge**: Up to 60% of total disk space
- **Firefox**: Up to 50% of free disk space
- **Safari**: Up to 1GB initially, can request more

Check available storage:

```typescript
if ("storage" in navigator && "estimate" in navigator.storage) {
  const estimate = await navigator.storage.estimate();
  console.log(`Using ${estimate.usage} of ${estimate.quota} bytes`);
}
```

## Use Cases

### 1. Offline Web Apps

```typescript
const fs = new IndexedDBFileSystem({ dbName: "offline-app" });

// Save user documents locally
await fs.writeFile("/documents/report.pdf", pdfBlob);
await fs.writeFile("/documents/data.xlsx", excelBlob);

// Access offline
const files = await fs.readDir("/documents");
```

### 2. Browser-Based IDEs

```typescript
const fs = new IndexedDBFileSystem({ dbName: "web-ide" });

// Save project files
await fs.mkdir("/src");
await fs.writeFile("/src/index.js", sourceCode);
await fs.writeFile("/package.json", packageJson);

// Watch for changes
fs.watch("**/*.js", (event) => {
  console.log(`File ${event.type}: ${event.path}`);
});
```

### 3. Client-Side Caching

```typescript
const fs = new IndexedDBFileSystem({ dbName: "app-cache" });

// Cache API responses
await fs.writeFile("/cache/users.json", await fetchUsers());

// Read from cache
try {
  const cached = await fs.readFile("/cache/users.json");
  return cached.content;
} catch {
  // Cache miss, fetch fresh data
  const fresh = await fetchUsers();
  await fs.writeFile("/cache/users.json", fresh);
  return fresh;
}
```

## Error Handling

```typescript
try {
  await fs.writeFile("/data.json", data);
} catch (error) {
  if (error.name === "QuotaExceededError") {
    console.error("Storage quota exceeded");
    // Handle cleanup or request more storage
  }
}
```

## Performance Tips

1. **Batch Operations**: Group multiple operations when possible
2. **Large Files**: Consider chunking very large files
3. **Indexing**: Use meaningful paths for better organization
4. **Event Listeners**: Remove unused listeners to prevent memory leaks
5. **Initialization**: Call `initialize()` once to load all files efficiently

### Event Performance

```typescript
// Dispose of event listeners when done
const disposable = fs.events.on(FileSystemEvents.FILE_WRITTEN, handler);

// Later...
disposable.dispose(); // Remove the listener

// Or remove all listeners for an event
fs.events.removeAllListeners(FileSystemEvents.FILE_WRITTEN);
```

### Cleanup

Always dispose of the file system instance when done to clean up resources:

```typescript
const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Use the file system...

// Clean up when done
fs.dispose(); // Closes DB connection, stops watching, cleans up listeners
```

## Testing

When testing code that uses IndexedDBFileSystem in Node.js environments, you have two options:

### Option 1: Using the testing module (Recommended)

```typescript
// Import from the testing module which handles the polyfill automatically
import { IndexedDBFileSystem } from "@firesystem/indexeddb/testing";

describe("My app", () => {
  it("should work with IndexedDB", async () => {
    const fs = new IndexedDBFileSystem({ dbName: "test" });
    await fs.writeFile("/test.txt", "Hello");
    // ...
  });
});
```

Note: You still need to install `fake-indexeddb` as a dev dependency:

```bash
npm install --save-dev fake-indexeddb
```

### Option 2: Manual setup

```bash
npm install --save-dev fake-indexeddb
# or
yarn add -D fake-indexeddb
# or
pnpm add -D fake-indexeddb
```

Then import it before your tests:

```typescript
// In your test setup file or at the top of test files
import "fake-indexeddb/auto";

// Now you can test IndexedDBFileSystem
import { IndexedDBFileSystem } from "@firesystem/indexeddb";

describe("My app", () => {
  it("should work with IndexedDB", async () => {
    const fs = new IndexedDBFileSystem({ dbName: "test" });
    await fs.writeFile("/test.txt", "Hello");
    // ...
  });
});
```

## API Reference

See [@firesystem/core](../core) for the complete VFS API documentation.

## License

MIT © Anderson D. Rosa