@sailboat-computer/data-storage/README.md

Shared data storage library for sailboat computer v3

# Sailboat Computer Data Storage

A unified data storage solution for the Sailboat Computer system with resilience features and local storage fallback.

## Features

- **Unified Storage Manager**: A single interface for all storage operations
- **Multiple Storage Tiers**: Support for hot, warm, and cold storage tiers
- **Local Storage Fallback**: Automatic fallback to local storage when remote storage is unavailable
- **Synchronization**: Automatic synchronization between local and remote storage
- **Resilience Features**:
  - Circuit breakers to prevent cascading failures
  - Retry logic with exponential backoff
  - Timeouts for operations
  - Bulkhead pattern for isolation
- **Metrics**: Detailed metrics for storage operations and resilience features

## Installation

```bash
npm install @sailboat-computer/data-storage
```

## Usage

### Basic Usage

```typescript
import {
  createUnifiedStorageManager,
  StorageTier,
  StorageConfig
} from '@sailboat-computer/data-storage';

// Create storage configuration
const config: StorageConfig = {
  providers: {
    hot: {
      type: 'redis',
      config: {
        host: 'localhost',
        port: 6379,
        password: '',
        db: 0,
        tls: false
      }
    },
    // Add warm and cold storage providers as needed
    batching: {
      timeBased: {
        enabled: true,
        intervalMs: 5000
      },
      sizeBased: {
        enabled: true,
        maxBatchSize: 100
      },
      priorityOverride: true
    }
  }
};

// Create unified storage manager options
const options = {
  localStorage: {
    directory: './data',
    maxSizeBytes: 100 * 1024 * 1024, // 100MB
    encrypt: false,
    compressionLevel: 6
  },
  sync: {
    enabled: true,
    intervalMs: 60000, // 1 minute
    batchSize: 100
  },
  resilience: {
    circuitBreaker: {
      failureThreshold: 3,
      resetTimeoutMs: 30000 // 30 seconds
    },
    retry: {
      maxRetries: 3,
      baseDelayMs: 1000 // 1 second
    },
    timeout: {
      defaultTimeoutMs: 5000 // 5 seconds
    }
  },
  logger: {
    level: 'info',
    console: true
  }
};

// Create and initialize storage manager
const storageManager = createUnifiedStorageManager(config, options);
await storageManager.initialize();

// Store data
const id = await storageManager.store(
  { sensorId: 'temp-1', value: 25.5 },
  'sensor-readings',
  {
    timestamp: new Date(),
    tags: { location: 'cabin' },
    tier: StorageTier.HOT
  }
);

// Retrieve data
const results = await storageManager.retrieve({
  category: 'sensor-readings',
  timeRange: {
    start: new Date(Date.now() - 24 * 60 * 60 * 1000).toISOString(),
    end: new Date().toISOString()
  }
});

// Close storage manager when done
await storageManager.close();
```

### Advanced Features

#### Local Storage Status

```typescript
const localStatus = await storageManager.getLocalStorageStatus();
console.log('Local storage status:', localStatus);
```

#### Force Synchronization

```typescript
const syncResult = await storageManager.forceSynchronization();
console.log('Synchronization result:', syncResult);
```

#### Resilience Metrics

```typescript
const metrics = storageManager.getResilienceMetrics();
console.log('Resilience metrics:', metrics);
```

## API Reference

### StorageManager

The main interface for storage operations.

```typescript
interface StorageManager {
  initialize(): Promise<void>;
  close(): Promise<void>;
  store(data: any, category: string, options?: StorageOptions): Promise<string>;
  retrieve(query: DataQuery, tier?: StorageTier): Promise<StoredData[]>;
  update(data: StoredData): Promise<void>;
  delete(id: string): Promise<void>;
  storeBatch(items: Array<{ data: any; category: string; options?: StorageOptions }>): Promise<string[]>;
  retrieveBatch(queries: DataQuery[], tier?: StorageTier): Promise<StoredData[][]>;
  cleanup(category: string, tier: StorageTier, retentionDays: number): Promise<CleanupResult>;
  migrate(id: string, fromTier: StorageTier, toTier: StorageTier): Promise<void>;
  getStatus(): Promise<StorageStatus>;
}
```

### UnifiedStorageManager

Extends the StorageManager interface with additional features.

```typescript
interface UnifiedStorageManager extends StorageManager {
  getLocalStorageStatus(): Promise<LocalStorageStatus>;
  forceSynchronization(): Promise<SyncResult>;
  clearLocalStorage(): Promise<ClearResult>;
  getResilienceMetrics(): ResilienceMetrics;
}
```

### StorageTier

```typescript
enum StorageTier {
  HOT = 'hot',
  WARM = 'warm',
  COLD = 'cold'
}
```

### DataQuery

```typescript
interface DataQuery {
  category?: string;
  timeRange?: {
    start?: string;
    end?: string;
  };
  sort?: {
    field: string;
    order: 'asc' | 'desc';
  };
}
```

### StorageOptions

```typescript
interface StorageOptions {
  timestamp?: Date;
  tags?: Record<string, string>;
  tier?: StorageTier;
  priority?: 'low' | 'normal' | 'high' | 'critical';
  ttl?: number; // Time to live in seconds
}
```

## Examples

See the [examples](./examples) directory for more usage examples.

## License

MIT