@dataql/react-native/DATABASE_CONFIG_GUIDE.md

DataQL React Native SDK with offline-first capabilities and clean API

# Database Configuration Guide

This guide explains how to use the new extended database configuration in DataQL React Native client.

## Overview

The DataQL React Native client now supports comprehensive database configuration that gets passed through the DataQL Worker to the Lambda functions for actual database connections. The architecture flow is:

**Client → Worker → Lambda → Database**

The client configuration specifies which database the worker/lambda should connect to, supporting:

- **MongoDB** - Document database
- **SQLite** - Local file-based database
- **PostgreSQL** - Relational database
- **Cloudflare D1** - Serverless SQLite

## DataQL Database Configuration Guide

This guide explains how to configure database connections in DataQL React Native applications.

### Architecture Overview

DataQL uses a distributed architecture: **Client → Worker → Lambda → Database**

- **MongoDB**: Handled by Lambda with automatic optimizations
- **Other databases** (PostgreSQL, SQLite, D1): Handled by Workers
- **Database isolation**: Each client gets their own database (no sharing)

### Configuration Modes

**Database name is always required** - each client gets their own database for data isolation.

1. **Minimal Configuration**: Database name only (uses main database URL)
2. **Full Configuration**: Custom database URL + database name

```typescript
// Option 1: Minimal config - uses main database URL with your database name
const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: {
    name: "my-app-db", // Required - your dedicated database
    // Type auto-detected as MongoDB from main database URL
  },
});

// Option 2: Full configuration - custom database URL + your database name
const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: {
    name: "my-app-db", // Required - your dedicated database
    url: "mongodb://user:pass@custom-server.com:27017", // Type: MongoDB (auto-detected)
  },
});

// Option 3: Legacy mode (no databaseConfig) - uses session database name
const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  // Uses database name from session initialization
});
```

## DatabaseConfig Interface

DataQL provides a **simplified, user-facing** database configuration interface that makes it easy to connect to different databases. The system automatically handles all internal optimizations like connection pooling, timeouts, SSL settings, and retry logic.

### User-Facing Configuration

```typescript
interface DatabaseConfig {
  /** Database name (required - each client gets their own database) */
  name: string;
  /** Database connection URL (optional - uses main database URL if not provided. Database type auto-detected from URL protocol) */
  url?: string;
}
```

**Note**:

- **Database credentials should be included in the URL** (e.g., `mongodb://user:pass@host/db` or `postgresql://user:pass@host:port/db`).
- The `url` field is optional. If not provided, the system will use the main database URL with your specified database name.
- **Database name is always required** - each client gets their own isolated database.
- **Database type is auto-detected** from URL protocol (mongodb://, postgresql://, sqlite://, file:, etc.)

### What the System Handles Automatically

When you provide a `DatabaseConfig`, the DataQL system automatically applies production-ready defaults for:

- **Connection Pooling**: Optimized pool sizes (min: 2, max: 10)
- **Timeouts**: 30-second connection and socket timeouts
- **SSL/TLS**: Secure connections enabled by default for MongoDB and PostgreSQL
- **Retry Logic**: Automatic retries (3 attempts with 1-second delays)
- **Pool Management**: Connection acquisition and idle timeouts

You don't need to worry about these details - the system is configured with best practices out of the box.

## Architecture Flow

**Important**: The client does not connect directly to databases. The flow is:

```
Client → Worker → Lambda → Database
```

1. **Client**: Stores operations locally for immediate response
2. **Client**: Sends database config with requests to Worker/Lambda
3. **Worker**: Extracts database config and forwards to Lambda
4. **Lambda**: Uses database config for actual database connections
5. **Lambda**: Returns results back through the chain

This architecture ensures:

- ✅ Offline-first operation with local storage
- ✅ Transparent sync when connectivity is available
- ✅ No direct database connections from client devices
- ✅ Centralized database access control and security

## Configuration Examples

### MongoDB Configuration

```typescript
// URL with credentials (recommended)
const mongoConfig: DatabaseConfig = {
  name: "my-app-db",
  url: "mongodb://dbuser:dbpass@localhost:27017", // Type: MongoDB (auto-detected)
};

// Or minimal configuration (uses main database URL)
const mongoConfigSimple: DatabaseConfig = {
  name: "my-app-db",
  // No URL - uses main database URL (MONGO_URI) with your database name
  // Type: MongoDB (auto-detected from main database)
};

const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: mongoConfig,
});
```

### PostgreSQL Configuration

```typescript
// URL with credentials (recommended)
const postgresConfig: DatabaseConfig = {
  name: "my-app-db",
  url: "postgresql://dbuser:dbpass@localhost:5432/my-app-db", // Type: PostgreSQL (auto-detected)
};

// Or minimal configuration (uses main database URL)
const postgresConfigSimple: DatabaseConfig = {
  name: "my-app-db",
  // No URL - uses main database URL with your database name
  // Note: Main database is typically MongoDB, so this would fallback to MongoDB
};

const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: postgresConfig,
});
```

### SQLite Configuration (for testing)

```typescript
const sqliteConfig: DatabaseConfig = {
  type: "sqlite",
  name: "app.db",
  url: "file:./app.db",
  // No username/password needed for SQLite
};

const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: sqliteConfig,
});
```

### Cloudflare D1 Configuration

```typescript
const d1Config: DatabaseConfig = {
  type: "d1",
  name: "my-d1-database",
  url: "https://api.cloudflare.com/client/v4/accounts/{account_id}/d1/database/{database_id}",
  options: {
    // D1-specific options
    accountId: "your-account-id",
    databaseId: "your-database-id",
  },
};

const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: d1Config,
});
```

## Multiple Database Support

You can easily switch between different databases by changing the configuration:

```typescript
// Development - use local MongoDB
const devConfig: DatabaseConfig = {
  type: "mongodb",
  name: "myapp-dev",
  url: "mongodb://localhost:27017",
};

// Production - use MongoDB Atlas
const prodConfig: DatabaseConfig = {
  type: "mongodb",
  name: "myapp-prod",
  username: process.env.DB_USER,
  password: process.env.DB_PASS,
  url: process.env.MONGODB_URL!,
};

// Use the appropriate config based on environment
const client = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig:
    process.env.NODE_ENV === "production" ? prodConfig : devConfig,
});
```

## Current Support Status

- ✅ **MongoDB**: Full support with connection pooling and SSL
- 🚧 **PostgreSQL**: Interface ready, implementation pending
- 🚧 **SQLite**: Interface ready, implementation pending
- 🚧 **Cloudflare D1**: Interface ready, implementation pending

## Security Best Practices

1. **Never hardcode credentials** in your source code
2. **Use environment variables** for sensitive configuration
3. **Enable SSL/TLS** for production databases (handled automatically)
4. **Use connection pooling** for optimal performance (handled automatically)
5. **Implement proper retry logic** for network resilience (handled automatically)

The DataQL system handles items 3-5 automatically when you provide a `DatabaseConfig`.

## Configuration Interface

```typescript
interface DatabaseConfig {
  /** Database type */
  type: "mongodb" | "sqlite" | "d1" | "postgres";
  /** Database name */
  name: string;
  /** Database username (optional) */
  username?: string;
  /** Database password (optional) */
  password?: string;
  /** Database connection URL */
  url: string;
  /** Optional connection pool settings (for lambda) */
  pool?: {
    min?: number;
    max?: number;
  };
  /** SSL/TLS configuration (for lambda connections) */
  ssl?:
    | boolean
    | {
        rejectUnauthorized?: boolean;
        ca?: string;
        cert?: string;
        key?: string;
      };
  /** Additional database-specific options (for lambda) */
  options?: Record<string, any>;
}
```

## Basic Usage

```typescript
import { DataQLClient, DatabaseConfig } from "@dataql/react-native";

// Define database configuration that will be used by worker/lambda
const databaseConfig: DatabaseConfig = {
  type: "mongodb",
  name: "myapp_production",
  username: "myapp_user",
  password: "secure_password_123",
  url: "mongodb://localhost:27017/myapp_production",
};

// Create client configuration
const config = {
  schemas: yourSchemas,
  database: databaseConfig, // Passed to worker/lambda
  syncConfig: {
    serverUrl: "https://dataql-worker.finhub.workers.dev", // Worker endpoint
  },
  // ... other configuration options
};

// Initialize client
const client = new DataQLClient(config);
await client.initialize();
```

## Database-Specific Configurations

### MongoDB (for Lambda Connection)

```typescript
const mongoConfig: DatabaseConfig = {
  type: "mongodb",
  name: "myapp_production",
  username: "mongodb_user",
  password: "mongodb_password",
  url: "mongodb+srv://user:pass@cluster.mongodb.net/database",
};
```

### PostgreSQL (for Lambda Connection)

```typescript
const postgresConfig: DatabaseConfig = {
  type: "postgres",
  name: "myapp_production",
  username: "postgres_user",
  password: "postgres_password",
  url: "postgresql://user:pass@localhost:5432/database",
};
```

### SQLite (for Lambda Connection)

```typescript
const sqliteConfig: DatabaseConfig = {
  type: "sqlite",
  name: "myapp_local.db",
  url: "file:./data/myapp_local.db",
};
```

### Cloudflare D1 (for Lambda Connection)

```typescript
const d1Config: DatabaseConfig = {
  type: "d1",
  name: "myapp-d1-database",
  username: "cloudflare_token",
  password: "your_cloudflare_api_token",
  url: "https://api.cloudflare.com/client/v4/accounts/account-id/d1/database/db-id",
};
```

## Environment-Based Configuration

```typescript
function getDatabaseConfig(env: string): DatabaseConfig {
  switch (env) {
    case "development":
      return {
        type: "sqlite",
        name: "myapp_dev.db",
        url: "file:./data/myapp_dev.db",
      };

    case "staging":
      return {
        type: "postgres",
        name: "myapp_staging",
        username: "staging_user",
        password: "staging_password",
        url: "postgresql://staging_user:staging_password@staging-db.com:5432/myapp_staging",
      };

    case "production":
      return {
        type: "mongodb",
        name: "myapp_production",
        username: "prod_user",
        password: "prod_password",
        url: "mongodb+srv://prod_user:prod_password@production-cluster.mongodb.net/myapp_production",
      };

    default:
      return getDatabaseConfig("development");
  }
}

// Use in client configuration
const config = {
  schemas: yourSchemas,
  database: getDatabaseConfig(process.env.NODE_ENV || "development"),
  syncConfig: {
    serverUrl: "https://dataql-worker.finhub.workers.dev",
  },
};
```

## Complete Client Configuration

```typescript
import {
  DataQLClient,
  DatabaseConfig,
  DataQLReactNativeConfig,
} from "@dataql/react-native";

const databaseConfig: DatabaseConfig = {
  type: "mongodb",
  name: "myapp_production",
  username: "myapp_user",
  password: "secure_password_123",
  url: "mongodb://localhost:27017/myapp_production",
};

const clientConfig: DataQLReactNativeConfig = {
  // Required: Your data schemas
  schemas: yourSchemas,

  // Optional: App authentication token
  appToken: "your-app-token",

  // Local SQLite database name (for offline storage)
  databaseName: "dataql_local.db",

  // Database configuration (passed to worker/lambda)
  database: databaseConfig,

  // Sync configuration
  syncConfig: {
    serverUrl: "https://dataql-worker.finhub.workers.dev", // Worker endpoint
    autoSync: true,
    syncInterval: 30000, // 30 seconds
    retryCount: 3,
    batchSize: 50,
  },

  // Additional options
  enableChangeListener: true,
  debug: true,
  env: "dev",
};

const client = new DataQLClient(clientConfig);
await client.initialize();
```

## Utility Functions

### Create Database Configuration

```typescript
import { createDatabaseConfig } from "@dataql/react-native";

const config = createDatabaseConfig(
  "postgres",
  "myapp_db",
  "postgresql://user:pass@localhost:5432/myapp_db",
  {
    username: "db_user",
    password: "db_password",
    ssl: true,
    pool: {
      min: 2,
      max: 10,
    },
  }
);
```

### Validate Database Configuration

```typescript
import { validateDatabaseConfig } from "@dataql/react-native";

const validation = validateDatabaseConfig(databaseConfig);

if (!validation.valid) {
  console.error("Database configuration errors:", validation.errors);
  // Handle validation errors
} else {
  // Configuration is valid - will be sent to worker/lambda
  const client = new DataQLClient({
    database: databaseConfig,
    ...otherConfig,
  });
}
```

## Important Notes

### Architecture Flow

The DataQL architecture ensures secure and scalable database access:

1. **Client Side**: Stores data locally in SQLite for offline functionality
2. **Worker Layer**: Receives client requests and database configuration
3. **Lambda Functions**: Establish secure connections to production databases
4. **Database Layer**: Actual data storage (MongoDB, PostgreSQL, etc.)

### Transparent Operations

Remember that [the DataQL API should not distinguish between offline and online operations](memory:7036908233781557754) - it should be transparent to the user. The client will automatically:

1. Store operations locally for immediate response
2. Send requests with database config to worker/lambda
3. Sync data when connectivity is available
4. Handle conflicts and maintain data consistency

### Security Considerations

- Database credentials are transmitted securely to worker/lambda
- Client never directly connects to production databases
- Worker/lambda handles all database authentication and connection pooling
- Use environment variables for sensitive credentials
- Implement proper authentication between client and worker

### Performance Tips

- Configure appropriate connection pool settings for lambda functions
- Use read replicas for read-heavy workloads in lambda
- Consider caching strategies at the worker level
- Monitor lambda connection usage and adjust pool settings

## Example Implementation

See `examples/database-config-usage.tsx` for a complete working example that demonstrates:

- Multiple database type configurations for worker/lambda
- Environment-specific configuration
- Client initialization and testing
- Error handling and validation

## Migration from Previous Versions

If you're upgrading from a previous version:

1. Add the `database` property to your client configuration
2. Update your worker/lambda to handle the new database configurations
3. Test the new database connections in development
4. Deploy worker and lambda updates before client updates

The existing `databaseName` property is still supported for local SQLite database naming and works alongside the new `database` configuration that gets passed to worker/lambda.

### Database Isolation

Each client gets their own dedicated database for complete data isolation:

```typescript
// Client A gets their own database
const clientA = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: {
    name: "client-a-database", // Dedicated database for Client A
    // Type auto-detected from main database URL (MongoDB)
  },
});

// Client B gets their own separate database
const clientB = new DataQLClient({
  // Worker URL is now hardcoded to https://dataql-worker.finhub.workers.dev
  databaseConfig: {
    name: "client-b-database", // Dedicated database for Client B
    // Type auto-detected from main database URL (MongoDB)
  },
});
```

**Connection Strategy:**

- **With URL**: Creates/connects to database using provided URL + database name
- **Without URL**: Uses main database URL (MONGO_URI) + your database name
- **No sharing**: Each database name gets its own isolated data space