oneie
Version:
Build apps, websites, and AI agents in English. Zero-interaction setup for AI agents (Claude Code, Cursor, Windsurf). Download to your computer, run in the cloud, deploy to the edge. Open source and free forever.
1,769 lines (1,448 loc) • 88.2 kB
Markdown
---
title: 2 2 Config System
dimension: things
category: features
tags: backend, frontend
related_dimensions: connections, events, groups, knowledge, people
scope: global
created: 2025-11-03
updated: 2025-11-03
version: 1.0.0
ai_context: |
This document is part of the things dimension in the features category.
Location: one/things/features/2-2-config-system.md
Purpose: Documents feature 2-2: configuration system
Related dimensions: connections, events, groups, knowledge, people
For AI agents: Read this to understand 2 2 config system.
---
# Feature 2-2: Configuration System
**Feature ID:** `feature_2_2_config_system`
**Plan:** `plan_2_backend_agnostic_frontend`
**Owner:** Backend Specialist
**Status:** Complete Specification
**Priority:** P0 (Critical - Required for Provider Switching)
**Effort:** 3 days
**Dependencies:** Feature 2-1 (DataProvider Interface)
---
## Executive Summary
The Configuration System enables ONE Platform to support multiple backend providers (Convex, WordPress, Notion, Supabase) with runtime provider switching, multi-tenant isolation, encrypted credential storage, and <30-second switchover times. Organizations configure their backend provider via environment variables or runtime API, with all credentials encrypted at rest and provider factories instantiating the correct implementation on demand.
**Key Features:**
- Environment-based configuration with Zod validation
- Runtime provider switching per organization
- Encrypted API key storage using AES-256-GCM
- Provider factory pattern with lazy initialization
- Multi-tenant isolation (each org has independent provider)
- Provider registration system for extensibility
- Event logging for all configuration changes
- Fast switchover (<30 seconds)
---
## 1. Ontology Mapping (6 Dimensions)
### 1.1 Organizations
**Purpose:** Each organization has independent backend provider configuration.
**Schema Addition:**
```typescript
// entities table (type: "organization")
properties: {
// ... existing org properties
backendProvider?: {
type: "convex" | "wordpress" | "notion" | "supabase",
configId?: Id<"entities">, // Reference to external_connection
switchedAt?: number, // Last provider switch timestamp
previousProvider?: string, // For rollback
}
}
```
**Usage:**
- `org.properties.backendProvider.type` determines which provider to use
- `org.properties.backendProvider.configId` links to encrypted credentials
- Platform owner can switch providers for testing
- Org owners can switch their own provider (with proper permissions)
### 1.2 People
**Purpose:** Only org_owner and platform_owner can change backend providers.
**Roles with Config Access:**
- `platform_owner`: Can configure ANY organization's provider (for support/migration)
- `org_owner`: Can configure THEIR organization's provider
- `org_user`: Read-only access to provider type (not credentials)
- `customer`: No access to provider configuration
**Permission Check Pattern:**
```typescript
// Before allowing provider switch
const person = await getPerson(userId);
const org = await getOrganization(orgId);
// Check authorization
if (person.role !== "platform_owner") {
// Must be org_owner of THIS org
const membership = await getMembership(userId, orgId);
if (!membership || membership.metadata.role !== "org_owner") {
throw new Error("Forbidden: Only org owners can change providers");
}
}
```
### 1.3 Things (Entities)
**New Entity Type: `external_connection`**
```typescript
type: "external_connection"
name: "Convex Production Backend" | "WordPress Staging" | etc.
properties: {
platform: "convex" | "wordpress" | "notion" | "supabase" | "custom",
connectionType: "backend_provider", // Distinguishes from other external connections
// Connection details (NOT encrypted here - see encryption section)
baseUrl?: string, // API base URL
apiKey?: string, // Encrypted API key (AES-256-GCM)
apiKeyIv?: string, // Initialization vector for encryption
apiKeyAuthTag?: string, // Auth tag for GCM mode
// Additional provider-specific config
config: {
// Convex
deploymentUrl?: string,
deploymentName?: string,
// WordPress
wpJsonEndpoint?: string,
username?: string,
applicationPassword?: string, // Encrypted
// Notion
notionToken?: string, // Encrypted
databaseId?: string,
// Supabase
supabaseUrl?: string,
supabaseAnonKey?: string, // Encrypted
supabaseServiceKey?: string, // Encrypted (admin operations)
},
// Metadata
status: "active" | "inactive" | "error",
lastConnectedAt?: number,
lastError?: string,
healthCheckUrl?: string,
rateLimits?: {
requestsPerMinute: number,
requestsPerDay: number,
},
}
status: "active" | "inactive"
createdAt: number
updatedAt: number
```
**Why This Design:**
- Reuses existing `external_connection` entity type (no new type needed)
- Encrypted credentials stored in properties (never in plaintext)
- Supports multiple configs per organization (dev, staging, prod)
- Can health-check provider availability
- Rate limits prevent quota exhaustion
### 1.4 Connections
**Organization → Backend Config:**
```typescript
{
fromEntityId: organizationId,
toEntityId: externalConnectionId,
relationshipType: "configured_by",
metadata: {
configType: "backend_provider",
activeProvider: true, // Only one can be active per org
switchedAt: Date.now(),
switchedBy: personId,
},
validFrom: Date.now(),
validTo?: undefined, // Current active config has no end date
createdAt: Date.now(),
}
```
**Person → Backend Config (Who Created):**
```typescript
{
fromEntityId: personId,
toEntityId: externalConnectionId,
relationshipType: "created_by",
metadata: {
role: "platform_owner" | "org_owner",
},
createdAt: Date.now(),
}
```
### 1.5 Events
**Configuration Events:**
```typescript
// settings_updated (existing consolidated event)
{
type: "settings_updated",
actorId: personId,
targetId: organizationId,
timestamp: Date.now(),
metadata: {
settingType: "backend_provider",
fromProvider: "convex",
toProvider: "wordpress",
configId: externalConnectionId,
switchDuration: 28500, // milliseconds (<30 seconds target)
}
}
// communication_event (for provider health checks)
{
type: "communication_event",
actorId: systemAgentId,
targetId: externalConnectionId,
timestamp: Date.now(),
metadata: {
protocol: "http",
messageType: "health_check",
endpoint: "https://shocking-falcon-870.convex.cloud/_system/health",
status: 200,
responseTime: 142, // ms
}
}
// entity_created (when new config added)
{
type: "entity_created",
actorId: personId,
targetId: externalConnectionId,
timestamp: Date.now(),
metadata: {
entityType: "external_connection",
platform: "wordpress",
organizationId: orgId,
}
}
// entity_archived (when config deactivated)
{
type: "entity_archived",
actorId: personId,
targetId: externalConnectionId,
timestamp: Date.now(),
metadata: {
reason: "provider_switch",
newConfigId: newExternalConnectionId,
}
}
```
### 1.6 Knowledge
**Configuration Labels:**
```typescript
// Label knowledge items for provider configs
{
knowledgeType: "label",
text: "backend:convex",
labels: ["capability:realtime", "capability:auth", "capability:storage"],
createdAt: Date.now(),
}
{
knowledgeType: "label",
text: "backend:wordpress",
labels: ["capability:cms", "capability:rest_api", "capability:plugins"],
createdAt: Date.now(),
}
// Link labels to external_connection entities via thingKnowledge
{
thingId: externalConnectionId,
knowledgeId: labelId,
role: "label",
createdAt: Date.now(),
}
```
**Why Labels:**
- Enables capability-based provider discovery
- Supports queries like "find backends with realtime capability"
- Allows filtering in UI (show only providers with auth support)
- Future-proof for new provider types
---
## 2. User Stories with Acceptance Criteria
### Story 2.1: Platform Owner Configures Default Provider
**As a** platform owner (Anthony)
**I want to** set the default backend provider via environment variables
**So that** new organizations automatically use the configured backend
**Acceptance Criteria:**
- [ ] `.env.local` contains `BACKEND_PROVIDER=convex` variable
- [ ] `.env.local` contains provider-specific credentials (e.g., `CONVEX_DEPLOYMENT_URL`)
- [ ] Zod schema validates all required environment variables on startup
- [ ] Missing or invalid config throws error with helpful message
- [ ] Default provider is used when org has no custom config
- [ ] Provider switch creates `settings_updated` event with platform_owner actorId
**Example Config:**
```bash
# .env.local
BACKEND_PROVIDER=convex
CONVEX_DEPLOYMENT_URL=https://shocking-falcon-870.convex.cloud
CONVEX_DEPLOYMENT_NAME=prod:shocking-falcon-870
# Alternative for WordPress
# BACKEND_PROVIDER=wordpress
# WORDPRESS_URL=https://example.com/wp-json
# WORDPRESS_USERNAME=admin
# WORDPRESS_APP_PASSWORD=xxxx-xxxx-xxxx-xxxx
```
### Story 2.2: Org Owner Switches Backend Provider
**As an** org owner
**I want to** switch my organization's backend from Convex to WordPress
**So that** I can use my existing WordPress infrastructure
**Acceptance Criteria:**
- [ ] Org owner can access "Backend Settings" page
- [ ] Page lists available provider types (Convex, WordPress, Notion, Supabase)
- [ ] Org owner enters WordPress credentials in form
- [ ] System validates credentials by testing connection
- [ ] On success, system creates `external_connection` entity with encrypted credentials
- [ ] System updates organization's `backendProvider` property
- [ ] System creates `settings_updated` event
- [ ] Switch completes in <30 seconds
- [ ] Next API call uses new provider automatically
**Security Requirements:**
- [ ] API keys encrypted at rest using AES-256-GCM
- [ ] Credentials never logged or exposed in events
- [ ] Only org_owner and platform_owner can see credentials
- [ ] Failed credential validation does not save config
### Story 2.3: Multi-Tenant Isolation
**As the** platform
**I want to** ensure each organization uses its own provider independently
**So that** Org A using WordPress doesn't affect Org B using Convex
**Acceptance Criteria:**
- [ ] Each organization has independent `backendProvider` config
- [ ] Provider registry maintains per-org provider instances
- [ ] Queries automatically route to correct provider based on org context
- [ ] Provider failure in Org A does not affect Org B
- [ ] Event logs clearly show which org triggered which provider operation
- [ ] Health checks run per organization, not globally
**Example:**
```typescript
// Org A uses Convex
const orgAProvider = await getProviderForOrg("org_a");
await orgAProvider.query("entities", { type: "creator" }); // Convex
// Org B uses WordPress
const orgBProvider = await getProviderForOrg("org_b");
await orgBProvider.query("entities", { type: "creator" }); // WordPress REST API
```
### Story 2.4: Provider Health Monitoring
**As a** platform owner
**I want to** monitor health of all configured providers
**So that** I can detect and fix provider issues before users are affected
**Acceptance Criteria:**
- [ ] Health check runs every 5 minutes per active provider
- [ ] Health check creates `communication_event` with status
- [ ] Dashboard shows provider health status (online, degraded, offline)
- [ ] Failed health checks trigger alerts
- [ ] Health metrics include response time and error rate
- [ ] Admins can manually trigger health check
**Health Check Endpoint Examples:**
- Convex: `/_system/health`
- WordPress: `/wp-json/`
- Notion: `/v1/users/me`
- Supabase: `/rest/v1/`
### Story 2.5: Provider Factory Registration
**As a** developer
**I want to** register a new backend provider (e.g., Supabase)
**So that** users can choose from more backend options
**Acceptance Criteria:**
- [ ] Provider implements `DataProvider` interface
- [ ] Provider registered via `registerProvider()` function
- [ ] Factory creates provider instance with validated config
- [ ] New provider appears in UI dropdown automatically
- [ ] Provider-specific Zod schema validates config
- [ ] Provider factory lazy-initializes (only creates instance when needed)
**Registration Pattern:**
```typescript
// backend/convex/providers/supabase.ts
export const SupabaseProviderFactory: ProviderFactory = {
type: "supabase",
create: (config: ProviderConfig) => new SupabaseProvider(config),
validate: SupabaseConfigSchema,
capabilities: ["auth", "storage", "realtime", "edge_functions"],
};
// Register in registry
registerProvider(SupabaseProviderFactory);
```
### Story 2.6: Configuration Rollback
**As an** org owner
**I want to** rollback to previous provider if new one fails
**So that** my organization isn't stuck with a broken backend
**Acceptance Criteria:**
- [ ] Organization properties store `previousProvider` reference
- [ ] UI shows "Rollback to Convex" button when provider changed recently
- [ ] Rollback creates new connection with `validFrom` = now
- [ ] Old provider marked `inactive` but not deleted
- [ ] Rollback completes in <10 seconds
- [ ] Rollback creates `settings_updated` event with rollback metadata
**Example Rollback:**
```typescript
await rollbackProvider({
orgId: "org_123",
targetConfigId: previousConfigId, // Points to old external_connection
});
// Event logged:
{
type: "settings_updated",
actorId: orgOwnerId,
targetId: orgId,
metadata: {
settingType: "backend_provider_rollback",
fromProvider: "wordpress",
toProvider: "convex",
reason: "connection_failure",
}
}
```
### Story 2.7: Encrypted Credential Management
**As a** security engineer
**I want to** ensure API keys are never stored in plaintext
**So that** compromised database doesn't expose credentials
**Acceptance Criteria:**
- [ ] Encryption key stored in environment variable `ENCRYPTION_KEY`
- [ ] AES-256-GCM used for encryption (authenticated encryption)
- [ ] Each credential has unique IV (initialization vector)
- [ ] Auth tag validates integrity on decryption
- [ ] Failed decryption throws error and logs security event
- [ ] Credentials never appear in logs, events, or error messages
- [ ] Encryption key rotation supported (future feature)
**Encryption Flow:**
```typescript
// On save
const encrypted = await encrypt(apiKey, process.env.ENCRYPTION_KEY);
await db.patch(externalConnectionId, {
properties: {
...properties,
apiKey: encrypted.ciphertext,
apiKeyIv: encrypted.iv,
apiKeyAuthTag: encrypted.authTag,
},
});
// On load
const decrypted = await decrypt({
ciphertext: properties.apiKey,
iv: properties.apiKeyIv,
authTag: properties.apiKeyAuthTag,
key: process.env.ENCRYPTION_KEY,
});
```
### Story 2.8: Configuration Validation
**As a** developer
**I want to** validate provider configs with Zod schemas
**So that** invalid configs are caught early with helpful errors
**Acceptance Criteria:**
- [ ] Each provider type has dedicated Zod schema
- [ ] Schema validates required fields (e.g., `CONVEX_DEPLOYMENT_URL`)
- [ ] Schema validates field formats (URLs, API key patterns)
- [ ] Validation errors show field name and expected format
- [ ] Runtime config override validates before applying
- [ ] Invalid config prevents provider initialization
**Schema Examples:**
```typescript
// Convex config schema
export const ConvexConfigSchema = z.object({
type: z.literal("convex"),
deploymentUrl: z.string().url(),
deploymentName: z.string().regex(/^(dev|prod):.+$/),
});
// WordPress config schema
export const WordPressConfigSchema = z.object({
type: z.literal("wordpress"),
baseUrl: z.string().url(),
username: z.string().min(1),
applicationPassword: z.string().regex(/^[a-zA-Z0-9 ]{24}$/),
});
```
### Story 2.9: Provider Switching Performance
**As an** org owner
**I want** provider switches to complete quickly
**So that** my team isn't waiting for backend changes
**Acceptance Criteria:**
- [ ] Provider switch completes in <30 seconds (99th percentile)
- [ ] Switch includes: validate → encrypt → save → initialize → health check
- [ ] Progress indicator shows switch steps in UI
- [ ] Failed switch rolls back automatically
- [ ] Switch duration logged in `settings_updated` event
- [ ] Performance metrics tracked (P50, P95, P99)
**Target Timings:**
- Validate credentials: <2s
- Encrypt and save: <1s
- Initialize provider: <5s
- Health check: <3s
- Total: <15s (average), <30s (P99)
### Story 2.10: Configuration UI
**As an** org owner
**I want** a simple UI to manage backend configuration
**So that** I don't need technical knowledge to switch providers
**Acceptance Criteria:**
- [ ] Settings page at `/org/[orgId]/settings/backend`
- [ ] Current provider shown with status badge (online/offline)
- [ ] Dropdown to select new provider type
- [ ] Dynamic form based on selected provider (WordPress shows username field, Convex doesn't)
- [ ] "Test Connection" button validates credentials before saving
- [ ] Success notification shows "Provider switched to WordPress"
- [ ] Error messages are user-friendly (not raw error logs)
- [ ] Credentials masked in UI (show **\*\*\*** for API keys)
**UI Flow:**
1. Org owner visits `/org/acme/settings/backend`
2. Sees "Current Provider: Convex (Online)"
3. Clicks "Change Provider"
4. Selects "WordPress" from dropdown
5. Form shows: Base URL, Username, Application Password fields
6. Fills in credentials
7. Clicks "Test Connection" → success notification
8. Clicks "Save and Switch" → progress indicator
9. After 15 seconds → "Successfully switched to WordPress"
10. Page shows "Current Provider: WordPress (Online)"
---
## 3. Implementation Steps (50 Steps)
### Phase 1: Schema and Types (Steps 1-10)
**Step 1:** Define `ProviderConfig` type
```typescript
// backend/convex/providers/types.ts
export type ProviderType = "convex" | "wordpress" | "notion" | "supabase";
export interface ProviderConfig {
type: ProviderType;
url: string;
credentials: Record<string, string>;
options?: Record<string, any>;
}
export interface EncryptedCredentials {
ciphertext: string;
iv: string;
authTag: string;
}
```
**Step 2:** Add Zod schemas for each provider
```typescript
// backend/convex/providers/schemas.ts
import { z } from "zod";
export const ConvexConfigSchema = z.object({
type: z.literal("convex"),
deploymentUrl: z.string().url(),
deploymentName: z.string(),
});
export const WordPressConfigSchema = z.object({
type: z.literal("wordpress"),
baseUrl: z.string().url().endsWith("/wp-json"),
username: z.string().min(1),
applicationPassword: z.string().length(27), // WordPress app password format
});
export const NotionConfigSchema = z.object({
type: z.literal("notion"),
token: z.string().startsWith("secret_"),
databaseId: z.string(),
});
export const SupabaseConfigSchema = z.object({
type: z.literal("supabase"),
url: z.string().url(),
anonKey: z.string(),
serviceKey: z.string().optional(),
});
export const ProviderConfigSchema = z.discriminatedUnion("type", [
ConvexConfigSchema,
WordPressConfigSchema,
NotionConfigSchema,
SupabaseConfigSchema,
]);
```
**Step 3:** Update organization schema
```typescript
// backend/convex/schema.ts
entities: defineTable({
type: v.string(),
name: v.string(),
properties: v.any(), // Includes backendProvider: { type, configId, switchedAt }
status: v.optional(v.union(/* ... */)),
createdAt: v.number(),
updatedAt: v.number(),
});
```
**Step 4:** Add indexes for external_connection queries
```typescript
// backend/convex/schema.ts
entities: defineTable({
/* ... */
})
.index("by_type", ["type"])
.index("by_type_status", ["type", "status"]); // NEW: Fast external_connection queries
```
**Step 5:** Define `ProviderFactory` interface
```typescript
// backend/convex/providers/factory.ts
import { z } from "zod";
import type { DataProvider } from "./interface";
export interface ProviderFactory {
type: ProviderType;
create: (config: ProviderConfig) => Promise<DataProvider>;
validate: z.ZodSchema;
capabilities: string[];
healthCheckUrl?: (config: ProviderConfig) => string;
}
```
**Step 6:** Create encryption utility
```typescript
// backend/convex/lib/encryption.ts
import { createCipheriv, createDecipheriv, randomBytes } from "crypto";
const ALGORITHM = "aes-256-gcm";
const KEY_LENGTH = 32; // 256 bits
const IV_LENGTH = 16; // 128 bits
const AUTH_TAG_LENGTH = 16;
export async function encrypt(
plaintext: string,
key: string,
): Promise<EncryptedCredentials> {
const keyBuffer = Buffer.from(key, "hex");
if (keyBuffer.length !== KEY_LENGTH) {
throw new Error("Encryption key must be 32 bytes (64 hex chars)");
}
const iv = randomBytes(IV_LENGTH);
const cipher = createCipheriv(ALGORITHM, keyBuffer, iv);
let ciphertext = cipher.update(plaintext, "utf8", "hex");
ciphertext += cipher.final("hex");
const authTag = cipher.getAuthTag();
return {
ciphertext,
iv: iv.toString("hex"),
authTag: authTag.toString("hex"),
};
}
export async function decrypt(
encrypted: EncryptedCredentials,
key: string,
): Promise<string> {
const keyBuffer = Buffer.from(key, "hex");
const ivBuffer = Buffer.from(encrypted.iv, "hex");
const authTagBuffer = Buffer.from(encrypted.authTag, "hex");
const decipher = createDecipheriv(ALGORITHM, keyBuffer, ivBuffer);
decipher.setAuthTag(authTagBuffer);
let plaintext = decipher.update(encrypted.ciphertext, "hex", "utf8");
plaintext += decipher.final("utf8");
return plaintext;
}
```
**Step 7:** Create environment variable loader
```typescript
// backend/convex/config/loader.ts
import { ProviderConfigSchema } from "../providers/schemas";
export function loadProviderConfig(): ProviderConfig {
const type = process.env.BACKEND_PROVIDER as ProviderType;
if (!type) {
throw new Error("BACKEND_PROVIDER environment variable required");
}
const rawConfig = {
type,
...(type === "convex" && {
deploymentUrl: process.env.CONVEX_DEPLOYMENT_URL,
deploymentName: process.env.CONVEX_DEPLOYMENT_NAME,
}),
...(type === "wordpress" && {
baseUrl: process.env.WORDPRESS_URL,
username: process.env.WORDPRESS_USERNAME,
applicationPassword: process.env.WORDPRESS_APP_PASSWORD,
}),
...(type === "notion" && {
token: process.env.NOTION_TOKEN,
databaseId: process.env.NOTION_DATABASE_ID,
}),
...(type === "supabase" && {
url: process.env.SUPABASE_URL,
anonKey: process.env.SUPABASE_ANON_KEY,
serviceKey: process.env.SUPABASE_SERVICE_KEY,
}),
};
// Validate with Zod
return ProviderConfigSchema.parse(rawConfig);
}
```
**Step 8:** Add encryption key validation
```typescript
// backend/convex/config/loader.ts (continued)
export function validateEncryptionKey(): void {
const key = process.env.ENCRYPTION_KEY;
if (!key) {
throw new Error("ENCRYPTION_KEY environment variable required");
}
const keyBuffer = Buffer.from(key, "hex");
if (keyBuffer.length !== 32) {
throw new Error("ENCRYPTION_KEY must be 32 bytes (64 hex characters)");
}
}
// Generate key helper (for docs)
export function generateEncryptionKey(): string {
return randomBytes(32).toString("hex");
}
```
**Step 9:** Create provider configuration service
```typescript
// backend/convex/services/config/provider.ts
import { Effect, Context } from "effect";
import type { Id } from "../../_generated/dataModel";
export class ProviderConfigService extends Context.Tag("ProviderConfigService")<
ProviderConfigService,
{
getForOrganization: (
orgId: Id<"entities">,
) => Effect.Effect<
ProviderConfig | null,
ConfigNotFoundError | DecryptionError
>;
saveForOrganization: (
orgId: Id<"entities">,
config: ProviderConfig,
actorId: Id<"entities">,
) => Effect.Effect<
Id<"entities">, // externalConnectionId
ValidationError | EncryptionError | UnauthorizedError
>;
testConnection: (
config: ProviderConfig,
) => Effect.Effect<
{ success: true; responseTime: number },
ConnectionTestError
>;
}
>() {}
```
**Step 10:** Define error types
```typescript
// backend/convex/services/config/errors.ts
export class ConfigNotFoundError {
readonly _tag = "ConfigNotFoundError";
constructor(readonly orgId: string) {}
}
export class DecryptionError {
readonly _tag = "DecryptionError";
constructor(readonly reason: string) {}
}
export class ValidationError {
readonly _tag = "ValidationError";
constructor(readonly errors: z.ZodError) {}
}
export class EncryptionError {
readonly _tag = "EncryptionError";
constructor(readonly reason: string) {}
}
export class UnauthorizedError {
readonly _tag = "UnauthorizedError";
constructor(
readonly userId: string,
readonly requiredRole: string,
) {}
}
export class ConnectionTestError {
readonly _tag = "ConnectionTestError";
constructor(
readonly provider: string,
readonly reason: string,
readonly httpStatus?: number,
) {}
}
```
### Phase 2: Provider Factory (Steps 11-20)
**Step 11:** Create provider registry
```typescript
// backend/convex/providers/registry.ts
const registeredProviders = new Map<ProviderType, ProviderFactory>();
export function registerProvider(factory: ProviderFactory): void {
registeredProviders.set(factory.type, factory);
}
export function getProviderFactory(
type: ProviderType,
): ProviderFactory | undefined {
return registeredProviders.get(type);
}
export function listProviderTypes(): ProviderType[] {
return Array.from(registeredProviders.keys());
}
export function getProviderCapabilities(type: ProviderType): string[] {
return registeredProviders.get(type)?.capabilities ?? [];
}
```
**Step 12:** Implement Convex provider factory
```typescript
// backend/convex/providers/convex/factory.ts
import { ConvexProvider } from "./provider";
import { ConvexConfigSchema } from "../schemas";
export const ConvexProviderFactory: ProviderFactory = {
type: "convex",
create: async (config: ProviderConfig) => {
const validated = ConvexConfigSchema.parse(config);
return new ConvexProvider({
deploymentUrl: validated.deploymentUrl,
deploymentName: validated.deploymentName,
});
},
validate: ConvexConfigSchema,
capabilities: ["auth", "realtime", "storage", "functions", "search"],
healthCheckUrl: (config) => `${config.url}/_system/health`,
};
// Auto-register
registerProvider(ConvexProviderFactory);
```
**Step 13:** Implement WordPress provider factory
```typescript
// backend/convex/providers/wordpress/factory.ts
import { WordPressProvider } from "./provider";
import { WordPressConfigSchema } from "../schemas";
export const WordPressProviderFactory: ProviderFactory = {
type: "wordpress",
create: async (config: ProviderConfig) => {
const validated = WordPressConfigSchema.parse(config);
return new WordPressProvider({
baseUrl: validated.baseUrl,
username: validated.username,
applicationPassword: validated.applicationPassword,
});
},
validate: WordPressConfigSchema,
capabilities: ["cms", "rest_api", "plugins", "media_library"],
healthCheckUrl: (config) => config.baseUrl,
};
registerProvider(WordPressProviderFactory);
```
**Step 14:** Create provider instance cache
```typescript
// backend/convex/providers/cache.ts
const providerCache = new Map<string, DataProvider>();
export function getCachedProvider(orgId: string): DataProvider | undefined {
return providerCache.get(orgId);
}
export function setCachedProvider(orgId: string, provider: DataProvider): void {
providerCache.set(orgId, provider);
}
export function clearCachedProvider(orgId: string): void {
providerCache.delete(orgId);
}
export function clearAllProviders(): void {
providerCache.clear();
}
```
**Step 15:** Implement lazy provider initialization
```typescript
// backend/convex/providers/initializer.ts
import { Effect } from "effect";
import { getCachedProvider, setCachedProvider } from "./cache";
import { getProviderFactory } from "./registry";
export const initializeProvider = (
orgId: Id<"entities">,
config: ProviderConfig,
): Effect.Effect<DataProvider, ProviderInitError> =>
Effect.gen(function* () {
// Check cache first
const cached = getCachedProvider(orgId);
if (cached) return cached;
// Get factory
const factory = getProviderFactory(config.type);
if (!factory) {
return yield* Effect.fail(
new ProviderInitError(`Unknown provider type: ${config.type}`),
);
}
// Create provider instance
const provider = yield* Effect.tryPromise({
try: () => factory.create(config),
catch: (error) =>
new ProviderInitError(`Failed to initialize ${config.type}: ${error}`),
});
// Cache for future use
setCachedProvider(orgId, provider);
return provider;
});
```
**Step 16:** Create provider switcher service
```typescript
// backend/convex/services/config/switcher.ts
export const switchProvider = (
orgId: Id<"entities">,
newConfig: ProviderConfig,
actorId: Id<"entities">,
): Effect.Effect<
{ switchDuration: number; configId: Id<"entities"> },
ValidationError | EncryptionError | UnauthorizedError | ConnectionTestError
> =>
Effect.gen(function* () {
const startTime = Date.now();
// 1. Validate config
const factory = getProviderFactory(newConfig.type);
if (!factory) {
return yield* Effect.fail(
new ValidationError(`Unknown provider: ${newConfig.type}`),
);
}
const validated = factory.validate.parse(newConfig);
// 2. Test connection
yield* testConnection(validated);
// 3. Encrypt credentials
const encrypted = yield* encryptCredentials(validated);
// 4. Save config as external_connection
const configId = yield* saveConfig(orgId, encrypted, actorId);
// 5. Update organization
yield* updateOrgProvider(orgId, newConfig.type, configId);
// 6. Clear cached provider
clearCachedProvider(orgId);
// 7. Log event
const switchDuration = Date.now() - startTime;
yield* logProviderSwitch(orgId, actorId, newConfig.type, switchDuration);
return { switchDuration, configId };
});
```
**Step 17:** Implement connection test
```typescript
// backend/convex/services/config/tester.ts
export const testConnection = (
config: ProviderConfig,
): Effect.Effect<
{ success: true; responseTime: number },
ConnectionTestError
> =>
Effect.gen(function* () {
const factory = getProviderFactory(config.type);
if (!factory?.healthCheckUrl) {
// No health check available, assume success
return { success: true, responseTime: 0 };
}
const url = factory.healthCheckUrl(config);
const startTime = Date.now();
const response = yield* Effect.tryPromise({
try: () => fetch(url, { method: "GET", timeout: 5000 }),
catch: (error) =>
new ConnectionTestError(
config.type,
`Health check failed: ${error}`,
undefined,
),
});
if (!response.ok) {
return yield* Effect.fail(
new ConnectionTestError(
config.type,
`Health check returned ${response.status}`,
response.status,
),
);
}
const responseTime = Date.now() - startTime;
return { success: true, responseTime };
});
```
**Step 18:** Implement credential encryption helper
```typescript
// backend/convex/services/config/encryption.ts
export const encryptCredentials = (
config: ProviderConfig,
): Effect.Effect<EncryptedConfig, EncryptionError> =>
Effect.gen(function* () {
const key = process.env.ENCRYPTION_KEY;
if (!key) {
return yield* Effect.fail(
new EncryptionError("ENCRYPTION_KEY not configured"),
);
}
const credentials = extractCredentials(config);
const encrypted: Record<string, EncryptedCredentials> = {};
for (const [field, value] of Object.entries(credentials)) {
encrypted[field] = yield* Effect.tryPromise({
try: () => encrypt(value, key),
catch: (error) =>
new EncryptionError(`Failed to encrypt ${field}: ${error}`),
});
}
return { ...config, encryptedCredentials: encrypted };
});
function extractCredentials(config: ProviderConfig): Record<string, string> {
switch (config.type) {
case "convex":
return {}; // Convex uses public URLs
case "wordpress":
return { applicationPassword: config.applicationPassword };
case "notion":
return { token: config.token };
case "supabase":
return {
anonKey: config.anonKey,
...(config.serviceKey && { serviceKey: config.serviceKey }),
};
default:
return {};
}
}
```
**Step 19:** Implement config save mutation
```typescript
// backend/convex/mutations/config.ts
export const saveProviderConfig = mutation({
args: {
orgId: v.id("entities"),
config: v.any(), // Validated by service layer
},
handler: async (ctx, args) => {
// Get actor
const identity = await ctx.auth.getUserIdentity();
if (!identity) throw new Error("Not authenticated");
const actor = await ctx.db
.query("entities")
.filter((q) => q.eq(q.field("properties.email"), identity.email))
.first();
if (!actor) throw new Error("User not found");
// Use Effect service
return await Effect.runPromise(
switchProvider(args.orgId, args.config, actor._id).pipe(
Effect.provide(ConfigServiceLayer),
),
);
},
});
```
**Step 20:** Register all provider factories on startup
```typescript
// backend/convex/providers/index.ts
import "./convex/factory";
import "./wordpress/factory";
import "./notion/factory";
import "./supabase/factory";
export * from "./registry";
export * from "./factory";
export * from "./initializer";
```
### Phase 3: Multi-Tenant Provider Access (Steps 21-30)
**Step 21:** Create organization context service
```typescript
// backend/convex/services/context/organization.ts
export class OrganizationContext extends Context.Tag("OrganizationContext")<
OrganizationContext,
{
getCurrentOrg: () => Effect.Effect<Id<"entities">, OrgNotFoundError>;
getOrgProvider: (
orgId: Id<"entities">,
) => Effect.Effect<DataProvider, ConfigNotFoundError | ProviderInitError>;
}
>() {}
```
**Step 22:** Implement provider resolver
```typescript
// backend/convex/services/context/resolver.ts
export const resolveProviderForOrg = (
orgId: Id<"entities">,
): Effect.Effect<DataProvider, ConfigNotFoundError | ProviderInitError> =>
Effect.gen(function* () {
// Check cache
const cached = getCachedProvider(orgId);
if (cached) return cached;
// Load org
const org = yield* Effect.tryPromise(() => db.get(orgId));
if (!org) {
return yield* Effect.fail(new ConfigNotFoundError(orgId));
}
// Check if org has custom provider
const customConfig = org.properties.backendProvider;
if (customConfig?.configId) {
const config = yield* loadExternalConnection(customConfig.configId);
const decrypted = yield* decryptConfig(config);
return yield* initializeProvider(orgId, decrypted);
}
// Fall back to default provider
const defaultConfig = loadProviderConfig();
return yield* initializeProvider(orgId, defaultConfig);
});
```
**Step 23:** Implement config loader from external_connection
```typescript
// backend/convex/services/config/loader.ts
export const loadExternalConnection = (
configId: Id<"entities">,
): Effect.Effect<ProviderConfig, ConfigNotFoundError | DecryptionError> =>
Effect.gen(function* () {
const config = yield* Effect.tryPromise(() => db.get(configId));
if (!config || config.type !== "external_connection") {
return yield* Effect.fail(new ConfigNotFoundError(configId));
}
if (config.status !== "active") {
return yield* Effect.fail(
new ConfigNotFoundError(`Config ${configId} is inactive`),
);
}
// Decrypt credentials
const decrypted = yield* decryptConfigCredentials(config.properties);
return {
type: config.properties.platform,
...config.properties.config,
...decrypted,
};
});
```
**Step 24:** Create scoped query wrapper
```typescript
// backend/convex/queries/scoped.ts
export const createScopedQuery = <Args, Result>(
fn: (ctx: QueryCtx, args: Args, provider: DataProvider) => Promise<Result>,
) => {
return query({
handler: async (ctx, args) => {
// Get user identity
const identity = await ctx.auth.getUserIdentity();
if (!identity) throw new Error("Not authenticated");
// Get user's organization
const user = await ctx.db
.query("entities")
.filter((q) => q.eq(q.field("properties.email"), identity.email))
.first();
if (!user?.properties.organizationId) {
throw new Error("User has no organization");
}
// Resolve provider for org
const provider = await Effect.runPromise(
resolveProviderForOrg(user.properties.organizationId).pipe(
Effect.provide(ConfigServiceLayer),
),
);
// Execute query with scoped provider
return await fn(ctx, args, provider);
},
});
};
```
**Step 25:** Example scoped query usage
```typescript
// backend/convex/queries/entities.ts
export const list = createScopedQuery(
async (ctx, args: { type: string }, provider) => {
// Provider is automatically scoped to user's org
return await provider.query("entities", {
where: { type: args.type },
});
},
);
```
**Step 26:** Create scoped mutation wrapper
```typescript
// backend/convex/mutations/scoped.ts
export const createScopedMutation = <Args, Result>(
fn: (
ctx: MutationCtx,
args: Args,
provider: DataProvider,
orgId: Id<"entities">,
) => Promise<Result>,
) => {
return mutation({
handler: async (ctx, args) => {
const identity = await ctx.auth.getUserIdentity();
if (!identity) throw new Error("Not authenticated");
const user = await ctx.db
.query("entities")
.filter((q) => q.eq(q.field("properties.email"), identity.email))
.first();
if (!user?.properties.organizationId) {
throw new Error("User has no organization");
}
const provider = await Effect.runPromise(
resolveProviderForOrg(user.properties.organizationId).pipe(
Effect.provide(ConfigServiceLayer),
),
);
return await fn(ctx, args, provider, user.properties.organizationId);
},
});
};
```
**Step 27:** Implement provider health check
```typescript
// backend/convex/services/health/checker.ts
export const checkProviderHealth = (
configId: Id<"entities">,
): Effect.Effect<
{ status: "online" | "offline"; responseTime: number },
ConnectionTestError
> =>
Effect.gen(function* () {
const config = yield* loadExternalConnection(configId);
const result = yield* testConnection(config);
return {
status: "online",
responseTime: result.responseTime,
};
});
```
**Step 28:** Create scheduled health check action
```typescript
// backend/convex/crons/health.ts
export const healthCheckCron = cronJobs.interval(
"provider-health-check",
{ minutes: 5 },
internal.health.checkAllProviders,
);
export const checkAllProviders = internalAction({
handler: async (ctx) => {
// Get all active external_connection entities
const configs = await ctx.runQuery(
internal.queries.config.listActiveConfigs,
);
for (const config of configs) {
try {
const health = await Effect.runPromise(
checkProviderHealth(config._id).pipe(
Effect.provide(HealthCheckLayer),
),
);
// Log health event
await ctx.runMutation(internal.mutations.events.logHealthCheck, {
configId: config._id,
status: health.status,
responseTime: health.responseTime,
});
} catch (error) {
// Log failure event
await ctx.runMutation(internal.mutations.events.logHealthCheck, {
configId: config._id,
status: "offline",
error: String(error),
});
}
}
},
});
```
**Step 29:** Implement provider usage tracking
```typescript
// backend/convex/services/usage/tracker.ts
export const trackProviderUsage = (
orgId: Id<"entities">,
operation: string,
duration: number,
): Effect.Effect<void, never> =>
Effect.gen(function* () {
// Create usage event
yield* Effect.tryPromise(() =>
db.insert("events", {
type: "communication_event",
actorId: orgId,
targetId: undefined,
timestamp: Date.now(),
metadata: {
protocol: "provider_usage",
operation,
duration,
},
}),
);
// Update organization usage metrics (if needed)
// This enables billing based on provider operations
});
```
**Step 30:** Add provider metrics query
```typescript
// backend/convex/queries/metrics.ts
export const getProviderMetrics = query({
args: { orgId: v.id("entities"), days: v.number() },
handler: async (ctx, args) => {
const since = Date.now() - args.days * 24 * 60 * 60 * 1000;
const events = await ctx.db
.query("events")
.withIndex("by_timestamp", (q) => q.gte("timestamp", since))
.filter((q) =>
q.and(
q.eq(q.field("type"), "communication_event"),
q.eq(q.field("actorId"), args.orgId),
q.eq(q.field("metadata.protocol"), "provider_usage"),
),
)
.collect();
return {
totalOperations: events.length,
averageDuration:
events.reduce((sum, e) => sum + (e.metadata.duration || 0), 0) /
events.length,
operationsByType: groupBy(events, (e) => e.metadata.operation),
};
},
});
```
### Phase 4: Frontend Integration (Steps 31-40)
**Step 31:** Create provider config form component
```typescript
// frontend/src/components/features/config/ProviderConfigForm.tsx
import { useState } from "react";
import { useMutation } from "convex/react";
import { api } from "@/convex/_generated/api";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Select } from "@/components/ui/select";
import { Card } from "@/components/ui/card";
export function ProviderConfigForm({ orgId }: { orgId: Id<"entities"> }) {
const [providerType, setProviderType] = useState<ProviderType>("convex");
const [config, setConfig] = useState<Record<string, string>>({});
const [testing, setTesting] = useState(false);
const saveConfig = useMutation(api.mutations.config.saveProviderConfig);
const testConnection = useMutation(api.mutations.config.testConnection);
const handleTest = async () => {
setTesting(true);
try {
const result = await testConnection({ orgId, config: { type: providerType, ...config } });
alert(`Connection successful! Response time: ${result.responseTime}ms`);
} catch (error) {
alert(`Connection failed: ${error.message}`);
} finally {
setTesting(false);
}
};
const handleSave = async () => {
try {
await saveConfig({ orgId, config: { type: providerType, ...config } });
alert("Provider configuration saved successfully!");
} catch (error) {
alert(`Failed to save configuration: ${error.message}`);
}
};
return (
<Card className="p-6">
<h2 className="text-2xl font-bold mb-4">Backend Provider Configuration</h2>
<div className="space-y-4">
<Select
value={providerType}
onValueChange={(value) => setProviderType(value as ProviderType)}
>
<option value="convex">Convex</option>
<option value="wordpress">WordPress</option>
<option value="notion">Notion</option>
<option value="supabase">Supabase</option>
</Select>
{providerType === "wordpress" && (
<>
<Input
placeholder="WordPress URL"
value={config.baseUrl || ""}
onChange={(e) => setConfig({ ...config, baseUrl: e.target.value })}
/>
<Input
placeholder="Username"
value={config.username || ""}
onChange={(e) => setConfig({ ...config, username: e.target.value })}
/>
<Input
type="password"
placeholder="Application Password"
value={config.applicationPassword || ""}
onChange={(e) => setConfig({ ...config, applicationPassword: e.target.value })}
/>
</>
)}
{providerType === "notion" && (
<>
<Input
type="password"
placeholder="Notion Token"
value={config.token || ""}
onChange={(e) => setConfig({ ...config, token: e.target.value })}
/>
<Input
placeholder="Database ID"
value={config.databaseId || ""}
onChange={(e) => setConfig({ ...config, databaseId: e.target.value })}
/>
</>
)}
<div className="flex gap-2">
<Button onClick={handleTest} disabled={testing}>
{testing ? "Testing..." : "Test Connection"}
</Button>
<Button onClick={handleSave} variant="primary">
Save Configuration
</Button>
</div>
</div>
</Card>
);
}
```
**Step 32:** Create provider status badge
```typescript
// frontend/src/components/features/config/ProviderStatusBadge.tsx
import { useQuery } from "convex/react";
import { api } from "@/convex/_generated/api";
import { Badge } from "@/components/ui/badge";
export function ProviderStatusBadge({ orgId }: { orgId: Id<"entities"> }) {
const status = useQuery(api.queries.config.getProviderStatus, { orgId });
if (status === undefined) return <Badge>Loading...</Badge>;
return (
<Badge variant={status.online ? "success" : "destructive"}>
{status.type} ({status.online ? "Online" : "Offline"})
</Badge>
);
}
```
**Step 33:** Create settings page
```typescript
// frontend/src/pages/org/[orgId]/settings/backend.astro
---
import Layout from "@/layouts/Layout.astro";
import ProviderConfigForm from "@/components/features/config/ProviderConfigForm";
import ProviderStatusBadge from "@/components/features/config/ProviderStatusBadge";
const { orgId } = Astro.params;
---
<Layout title="Backend Settings">
<div class="container mx-auto py-8">
<div class="flex items-center justify-between mb-6">
<h1 class="text-3xl font-bold">Backend Provider Settings</h1>
<ProviderStatusBadge client:load orgId={orgId} />
</div>
<ProviderConfigForm client:load orgId={orgId} />
</div>
</Layout>
```
**Step 34:** Add provider metrics dashboard
```typescript
// frontend/src/components/features/config/ProviderMetrics.tsx
import { useQuery } from "convex/react";
import { api } from "@/convex/_generated/api";
import { Card } from "@/components/ui/card";
import { LineChart, Line, XAxis, YAxis, Tooltip } from "recharts";
export function ProviderMetrics({ orgId }: { orgId: Id<"entities"> }) {
const metrics = useQuery(api.queries.metrics.getProviderMetrics, {
orgId,
days: 7,
});
if (metrics === undefined) return <div>Loading metrics...</div>;
return (
<Card className="p-6">
<h3 className="text-xl font-bold mb-4">Provider Usage (Last 7 Days)</h3>
<div className="grid grid-cols-2 gap-4 mb-6">
<div>
<p className="text-sm text-gray-500">Total Operations</p>
<p className="text-2xl font-bold">{metrics.totalOperations}</p>
</div>
<div>
<p className="text-sm text-gray-500">Average Response Time</p>
<p className="text-2xl font-bold">{metrics.averageDuration.toFixed(0)}ms</p>
</div>
</div>
<LineChart width={600} height={300} data={metrics.operationsByDay}>
<XAxis dataKey="day" />
<YAxis />
<Tooltip />
<Line type="monotone" dataKey="count" stroke="#8884d8" />
</LineChart>
</Card>
);
}
```
**Step 35:** Implement rollback UI
```typescript
// frontend/src/components/features/config/ProviderRollback.tsx
import { useMutation, useQuery } from "convex/react";
import { api } from "@/convex/_generated/api";
import { Button } from "@/components/ui/button";
import { Alert } from "@/components/ui/alert";
export function ProviderRollback({ orgId }: { orgId: Id<"entities"> }) {
const org = useQuery(api.queries.entities.get, { id: orgId });
const rollback = useMutation(api.mutations.config.rollbackProvider);
if (!org?.properties.backendProvider?.previousProvider) {
return null; // No previous provider to rollback to
}
const handleRollback = async () => {
if (!confirm("Are you sure you want to rollback to the previous provider?")) {
return;
}
try {
await rollback({ orgId });
alert("Successfully rolled back to previous provider!");
} catch (error) {
alert(`Rollback failed: ${error.message}`);
}
};
return (
<Alert variant="warning" className="mt-4">
<p className="mb-2">
You recently switched from {org.properties.backendProvider.previousProvider}.
</p>
<Button onClick={handleRollback} variant="outline">
Rollback to {org.properties.backendProvider.previousProvider}
</Button>
</Alert>
);
}
```
**Step 36:** Add provider switch confirmation dialog
```typescript
// frontend/src/components/fe